Slide and aquatic organism microscopic scanning image splicing method and device

Abstract

The embodiment of the invention provides a slide, an aquatic organism microscopic scanning image splicing method and equipment, and an intelligent image splicing method for aquatic organism microscopic scanning imaging. The slide comprises a slide body and reticulate pattern marking lines; the reticulate pattern identification lines comprise a plurality of first identification lines and a plurality of second identification lines with different gray features. The line spacing between the adjacent first identification lines and the line spacing between the adjacent second identification lines are both smaller than a threshold distance. In the microscopic image splicing process, at least two splicing references of the two to-be-spliced microscopic images can be determined based on the first identification line and the second identification line, and the splicing relation of the two adjacent to-be-spliced microscopic images can be determined based on the at least two splicing references. And accurate and rapid splicing of the microscopic images to be spliced can be realized based on the splicing reference.

Description

Technical Field

[0001] The present invention relates to the field of microscopic observation technology, and in particular to a method and device for splicing glass slides and microscopic scanning images of aquatic organisms. Background Art

[0002] Microscopic examination of aquatic organisms is an important part of water ecological assessment. Because aquatic organisms are small in size and have complex taxonomy, before microscopic examination of aquatic organisms, it is necessary to use a microscopic scanning imaging system to take pictures of different positions on the slide to obtain microscopic magnified images, stitch the microscopic magnified images to form an overall stitched image, and then analyze and identify the overall stitched image and count the biomass.

[0003] During the aforementioned operation, stitching the microscopic magnified images to form an overall stitched image is a difficult point in the entire operation process. Specifically, because the positions of aquatic organisms on the glass slide are random and vary in density, it may be difficult to find reference objects for correct stitching from the microscopic magnified images of adjacent areas during the image stitching process, resulting in poor image stitching quality. In order to overcome the aforementioned problem, the existing technology can only increase the overlapping area of ​​the microscopic magnified images as much as possible, and increasing the overlapping area increases the time spent on shooting a unit glass slide and the number of microscopic magnified images obtained, which further increases the resource overhead and workload in the later image stitching process. Summary of the invention

[0004] In order to solve the problem of high cost of stitching microscopic images due to the lack of reference objects, the embodiments of the present disclosure provide a method and a computing device for stitching microscopic scanned images of glass slides and aquatic organisms.

[0005] In a first aspect, an embodiment of the present disclosure provides a glass slide, comprising: a glass slide body, and a reticulated identification line disposed at a microscopic observation area of ​​the glass slide body;

[0006] The reticulated identification lines include a plurality of first identification lines running through the microscopic observation area along a first direction, and second identification lines running through the microscopic observation area along a second direction, wherein the first direction is different from the second direction;

[0007] At least adjacent first identification lines are set to have different light processing characteristics, so that the grayscales of the lines after microscopic imaging under the same lighting conditions are different; at least adjacent second identification lines are set to have different light processing characteristics, so that the grayscales of the lines after microscopic imaging under the same lighting conditions are also different;

[0008] The line spacing between adjacent first identification lines and the line spacing between adjacent second identification lines are both less than a threshold distance; the threshold distance is a distance to ensure that the microscopic images to be stitched formed after microscopic imaging have at least two first identification lines and at least one second identification line.

[0009] Optionally, each first identification line has different light processing characteristics, and the line grayscale of each first identification line changes sequentially after microscopic imaging; and / or,

[0010] Each of the second identification lines has different light processing characteristics, and the line grayscale of each of the second identification lines changes sequentially after microscopic imaging.

[0011] Optionally, the first identification line and / or the second identification line are etching lines formed by etching in the microscopic observation area; at least the etching depths and / or etching widths of adjacent first identification lines are different; and / or at least the etching depths and / or etching widths of adjacent second identification lines are different;

[0012] or,

[0013] The first marking line and / or the second marking line are deposited lines formed by deposition on the microscopic observation area; at least adjacent first marking lines have different deposition thicknesses and / or different light absorption characteristics; and / or at least adjacent second marking lines have different deposition thicknesses and / or different light absorption characteristics.

[0014] Optionally, each of the first identification lines is arranged on the slide body at equal intervals;

[0015] And / or, each of the second identification lines is arranged on the slide body at equal intervals.

[0016] In a second aspect, the present disclosure provides a method for stitching microscopic scanning images of aquatic organisms, comprising:

[0017] Identify the first identification line and the second identification line in each microscopic image to be stitched, and determine the position coordinates of the stitching reference in the microscopic image to be stitched based on the positions of the identified first identification line and the second identification line, and determine the two-dimensional position mark of each stitching reference according to the pixel grayscale of the first identification line and the second identification line; wherein the microscopic image to be stitched is obtained by photographing the sample prepared based on the glass slide as described above by a microscopic imaging system;

[0018] Based on the overlap of stitching reference pixels with the same two-dimensional position mark in adjacent microscopic images to be stitched, stitching and fusion processing is performed on adjacent microscopic images to obtain a stitched microscopic image.

[0019] Optionally, before performing the stitching and fusing process on the adjacent microscopic images to be stitched, the method further comprises: determining the number of interval pixels between two stitching references in the microscopic images to be stitched;

[0020] Determining a zoom factor based on the number of spaced pixels and a corresponding number of standard pixels; wherein the number of standard pixels is determined based on microscopic imaging of the two stitched references at a standard magnification, or microscopic imaging of a similar reference at the same distance from the two stitched references;

[0021] Performing scaling processing on the corresponding microscopic image to be stitched according to the scaling factor to determine a scaled microscopic image; or,

[0022] or performing scaling processing on the local pixel area of ​​the microscopic image to be stitched including the two stitching references according to the scaling factor to obtain a local scaled image; and stitching the partial scaled images to obtain a scaled microscopic image;

[0023] The stitching and fusing process of the adjacent microscopic images to be stitched includes: stitching and fusing the zoomed microscopic images corresponding to the adjacent images to be stitched.

[0024] Optionally, before performing stitching and fusing processing on the zoomed microscopic images corresponding to adjacent images to be stitched, the method further includes:

[0025] Based on the position coordinates of the stitching reference in each image to be stitched, the corresponding zoomed microscopic image is rotationally corrected;

[0026] The stitching and fusing process of the adjacent zoomed images to be stitched includes: stitching and fusing the zoomed microscopic images that have been subjected to rotation correction;

[0027] Alternatively, rotation correction is performed on each image to be stitched based on the position coordinates of the stitching reference in each image to be stitched;

[0028] The step of scaling the corresponding microscopic images to be spliced ​​according to the scaling factor to determine the scaled microscopic image includes: scaling the corresponding microscopic images to be spliced ​​that have undergone rotation correction according to the scaling factor to determine the scaled microscopic image.

[0029] Optionally, the adjacent microscopic images to be stitched are stitched and fused based on the overlap of stitching reference pixels having the same two-dimensional position identifier in adjacent microscopic images to be stitched, to obtain a stitched microscopic image, including:

[0030] Based on the overlap of pixels in adjacent images to be stitched, the pixels to be fused and the pixels not to be fused in the adjacent images to be stitched are determined, wherein the pixels to be fused are the pixels in the adjacent images to be stitched that represent the same imaging object;

[0031] Calculate the fused grayscale of the pixel to be fused as the pixel grayscale of the fused pixel;

[0032] The pixels that do not need to be fused and the pixels that need to be fused are spliced ​​according to the positional relationship between the pixels that do not need to be fused and the pixels that need to be fused to obtain the spliced ​​microscopic image.

[0033] Optionally, the splicing reference is an intersection point of the first identification line and the second identification line;

[0034] The determining the position coordinates of the stitching reference in the to-be-stitched microscopic image based on the position coordinates of the identified first identification line and the second identification line comprises: determining the position coordinates of the intersection point of the lines based on the position coordinates of the identified first identification line and the second identification line or the coordinates of the straight lines on which they are located;

[0035] Determining the two-dimensional position marker of each splicing reference according to the pixel grayscale of the first identification line and the second identification line includes: determining the first dimension position marker according to the pixel grayscale of the first identification line forming the line intersection point, and determining the second dimension position marker according to the pixel grayscale of the second identification line forming the line intersection point;

[0036] The first dimensional position identifier and the second dimensional position identifier are combined to obtain a two-dimensional position identifier of the line intersection point.

[0037] In a third aspect, an embodiment of the present disclosure provides a computing device, characterized in that it includes a processor and a memory, wherein the memory is used to store a computer program; when the computer program is loaded by the processor, the processor executes the stitching method of microscopic scanning images of aquatic organisms as described above.

[0038] When microscopic imaging is performed using the glass slide provided by the embodiment of the present disclosure, the pixel coordinates of at least two stitching references in two adjacent microscopic images to be stitched can be determined based on the first identification line and the second identification line. The stitching relationship between the two adjacent microscopic images to be stitched can be determined based on the pixel coordinates of the at least two stitching references, and then accurate and fast stitching of the microscopic images to be stitched can be achieved based on the stitching references. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor, including

[0041] Figure 1 is a schematic diagram of the structure of a glass slide provided by an embodiment of the present disclosure;

[0042] Figure 2 is a top view of the microscopic imaging area;

[0043] Figure 3 is a flow chart of a method for stitching microscopic scanning images of aquatic organisms provided in an embodiment of the present disclosure;

[0044] Figure 4 is a microscopic image to be stitched collected in a specific implementation;

[0045] Figure 5 is another microscopic image to be stitched collected during the specific implementation;

[0046] Figure 6 Is Figure 4 and Figure 5 A microscopic image formed by stitching two microscopic images to be stitched;

[0047] Figure 7 is a schematic diagram of a microscopic image stitching device provided in an embodiment of the present disclosure;

[0048] Figure 8 It is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0050] The term "including" and its variations used in this document are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. In this document, relational terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0051] In order to solve the various high cost problems caused by the lack of reference objects in the existing microbial microscopic image stitching, the embodiments of the present disclosure provide a new glass slide for microbial microscopic imaging, and a method for stitching microscopic scanning images of aquatic organisms based on the aforementioned glass slide.

[0052] Figure 1 Schematic diagram of the structure of the glass slide provided by the embodiment of the present disclosure. Figure 1 As shown, the glass slide 100 provided in the embodiment of the present disclosure is a glass slide, which includes a glass slide body 101, and a microscopic imaging area 102 is arranged on the glass slide body 101. The microscopic imaging area 102 is an area for carrying a water sample to realize subsequent microscopic imaging.

[0053] Figure 2 is a top view of the microscopic imaging area. Figure 2 As shown, the microscopic imaging area 102 is provided with a plurality of first identification lines 1021 and second identification lines 1022, wherein Figure 2 The line that is horizontally arranged and runs through the microscopic imaging area 102 is the first identification line 1021, the line that is vertically arranged and runs through the microscopic imaging area 102 is the second identification line 1022, and the black line at the edge is not the first identification line 1021 and the second identification line 1022, but the edge line of the microscopic imaging area 102 (which can be used to indicate the start or end of microscopic imaging shooting).

[0054] like Figure 2 As shown, each first identification line 1021 is set to have a different pixel grayscale when microscopic imaging is performed under the same lighting conditions, and similarly each second identification line 1022 is also set to have a different pixel grayscale when microscopic imaging is performed under the same lighting conditions. Each first identification line 1021 is set to have a different pixel grayscale when microscopic imaging, and each second identification line 1022 is set to have a different pixel grayscale when microscopic imaging, so that even if the adjacent relationship of the images to be spliced ​​cannot be determined later, it is still possible to achieve splicing of all the microscopic images to be spliced.

[0055] In other embodiments, only adjacent first marking lines may be set to have different pixel grayscales during microscopic imaging, and adjacent second marking lines 1022 may be set to have different pixel grayscales during microscopic imaging. In this case, it is necessary to reasonably set the moving step length during microscopic imaging, and record the adjacent relationship of each microscopic image to be spliced, so as to ensure that image splicing can be achieved.

[0056] In addition, in the embodiment of the present disclosure, the line spacing between adjacent first identification lines 1021 and the line spacing between adjacent second identification lines 1022 are both less than the threshold distance. The aforementioned threshold distance is a distance to ensure that all microscopic images to be stitched formed after microscopic imaging have at least two first identification lines 1021 and second identification lines 1022, and the threshold distance can be determined according to the actual imaging area size corresponding to the microscopic imaging field of view.

[0057] The preceding paragraph only defines the spacing between adjacent first identification lines 1021 and the spacing between adjacent second identification lines 1022, and does not define the spacing between the first identification lines 1021 to be the same as the spacing between the second identification lines. In addition, it does not define the spacing between adjacent first identification lines 1021 to be the same, nor does it define the spacing between adjacent first identification lines 1021 to be the same. In other words, in actual applications, the spacing between adjacent first identification lines 1021 only needs to be less than the threshold distance, and is not required to be the same; similarly, in actual applications, the spacing between adjacent second identification lines 1022 only needs to be less than the threshold distance, and is not required to be the same.

[0058] However, considering the convenience of use (consistency of reference distance), in actual application, the line spacing between adjacent first identification lines 1021 is set to be the same, and the line spacing between adjacent second identification lines 1022 is also set to be the same. Figure 2 As shown, in some embodiments, the line spacing of the first identification lines 1021 and the line spacing of the second identification lines 1022 are set to be the same. When the line spacing of the first identification lines 1021 and the line spacing of the second identification lines 1022 are set to be the same, it can be determined whether the magnifications of the images to be spliced ​​(or different areas of the images to be spliced) are consistent according to the number of pixels of the first identification lines 1021 or the second identification lines 1022 actually imaged.

[0059] See also Figure 2 In a specific implementation, the first identification line 1021 and the second identification line 1022 are set as solid lines with continuous imaging features. In other embodiments, the first identification line 1021 and the second identification line 1022 can also be set as discontinuous lines (as long as the two first identification lines 1021 and the two second identification lines 1022 can be imaged simultaneously in a single microscopic imaging).

[0060] like Figure 2 As shown, in the embodiment of the present disclosure, the first identification line 1021 and the second identification line 1022 are set to be perpendicular to each other. In other embodiments, the first identification line 1021 and the second identification line 1022 can also be set to other intersection angles, which does not affect the subsequent determination of the stitching reference based on them, and the stitching of adjacent images to be stitched based on the stitching reference.

[0061] As analyzed above, in the embodiment of the present disclosure, the glass slide 100 is configured to include a first identification line 1021 and a second identification line 1022, and the identification lines to be stitched formed by the microscopic imaging all have two first identification lines 1021 and a second identification line 1022, so that at least two stitching references of two microscopic images to be stitched can be determined based on the first identification lines 1021 and the second identification lines 1022, and based on the aforementioned at least two stitching references, the stitching relationship between two adjacent microscopic images to be stitched can also be determined, and then the microscopic images to be stitched can be quickly and accurately stitched based on the stitching references.

[0062] In some embodiments, the aforementioned first identification line 1021 and the second identification line 1022 are both etched lines set on the glass body. In a specific implementation, the etched lines can be formed by laser etching. In order to at least make the pixel grayscales of adjacent first identification lines 1021 different under the same illumination, and at least make the pixel grayscales of adjacent second identification lines 1022 different under the same illumination, the etching depths or etching widths of adjacent first identification lines 1021 are different, and the etching depths or etching widths of adjacent second identification lines 1022 are different. More preferably, in order to make the pixel grayscales of all first identification lines 1021 different during microscopic imaging, and to make the pixel grayscales of all second identification lines 1022 different during microscopic imaging, the etching depths of each first identification line 1021 can be set to be different from each other but the etching widths are the same, and the etching depths of each second identification line 1022 can be set to be different from each other but the etching widths are the same.

[0063] It should be noted here that the aforementioned etching width is much smaller than the size of the microorganisms in the prepared water sample to avoid interference with the observation of the morphology of the microorganisms (this is also the reason why the first identification line 1021 and the second identification line 1022 are not set to have a varying etching width).

[0064] In other embodiments, the first identification line 1021 and the second identification line 1022 may also be deposited lines disposed on the glass body, and the deposited lines may be deposited by a coating process or a glue coating process used in the semiconductor field. In order to make the pixel grayscale different during microscopic imaging under the same lighting conditions, at least the deposition thickness of adjacent first identification lines 1021 is different or the light absorption characteristics are different, and at least the deposition thickness of adjacent second identification lines 1022 is different or the light absorption characteristics are different. The aforementioned different light absorption characteristics may be achieved by changing the type of material of the deposit or the surface shape of the deposit.

[0065] like Figure 2As shown, in the embodiment of the present disclosure, the first identification line 1021 and the second identification line 1022 are set as straight line segments. In other embodiments, the first identification line 1021 and the second identification line 1022 are set as arcs, wavy lines or other types of segments, which does not hinder the realization of their identification functions.

[0066] The above mentioned the line spacing between the first identification line 1021 and the second identification line 1022. In practical applications, the setting of the line spacing should not only consider the stitching of adjacent microscopic images, but also consider the shooting efficiency and the focus adjustment of the microscope. Taking the above factors into consideration, the line spacing is set to 1 / 3-1 / 2 of the side length of the imaging field of view at the standard magnification.

[0067] In the foregoing, it is mentioned that the glass slide 100 is a glass slide. In a specific implementation, the glass slide 100 is not limited to a glass slide, but can also be a cover glass.

[0068] In addition to providing the aforementioned glass slide structure, the disclosed embodiment also provides a method for stitching microscopic scanning images of aquatic organisms. The image to be stitched processed by the method for stitching microscopic scanning images of aquatic organisms is obtained by photographing a sample prepared with the aforementioned glass slide as a carrier by a microscopic imaging system.

[0069] Figure 3 FIG. 1 is a flow chart of a method for stitching microscopic scanning images of aquatic organisms provided in an embodiment of the present disclosure. Figure 3 As shown, the stitching method of aquatic organism microscopic scanning images includes S110-S120.

[0070] S110: Identify a first identification line and a second identification line in the microscopic image to be stitched, and determine the position coordinates of at least two stitching references based on the pixel coordinates of the identified first identification line and the second identification line, and determine the two-dimensional relative position identification of each stitching reference according to the pixel grayscale of the first identification line and the second identification line.

[0071] In a specific embodiment, an image processing algorithm such as Hough transform can be used to process the pixel content in the stitched microscopic image to determine the first identification line and the second identification line. As analyzed above, because the first identification line and the second identification line are straight lines in conventional applications, the pixels where the first identification line and the second identification line are located can be easily determined by Hough transform.

[0072] In actual applications, during the microscopic imaging process, the microscopic imaging system and the glass slide make horizontal relative movement or vertical movement in the same direction. Therefore, after the glass slide is relatively fixed, the pointing directions of the first identification line and the second identification line will not change much during the microscopic imaging process. Therefore, the first identification line and the second identification line and their position coordinates in the microscopic image to be stitched can be identified more accurately.

[0073] After the position coordinates of the first identification line and the second identification line in the microscopic image are identified, the pixel grayscale of the pixels in the aforementioned position coordinate area is subsequently extracted to determine the pixel grayscale of the first identification line and the second identification line. According to the pre-set correspondence between the pixel grayscale and the position identifier, the corresponding line position identifier can be determined according to the pixel grayscale of the first identification line and the second identification line. The aforementioned line position identifier is used to indicate which first identification line or which second identification line in the slide the first identification line or the second identification line is.

[0074] In the disclosed embodiment, the stitching reference is a reference jointly determined by the first identification line and the second identification line, and is used to determine the image stitching benchmark. In a specific implementation, the stitching reference can be the intersection of the first identification line and the second identification line, or the first identification line and the second identification line themselves. Considering that using a point as a reference has the advantage of convenient processing, the stitching reference in practical applications often uses the aforementioned intersection point.

[0075] In a specific implementation, after determining the pixel coordinates of the first identification line and the pixel coordinates of the second identification line, the straight line expressions of the first identification line and the second identification line can be determined. Based on the straight line expressions of the first identification line and the second identification line, the position coordinates of the splicing reference can be determined.

[0076] After determining the line position marks of the first identification line and the second identification line in the foregoing, the two-dimensional relative position mark of the intersection point determined by the first identification line and the second identification line can be determined. Specifically, the two-dimensional relative position mark is obtained by combining the line position marks of the first identification line and the second identification line, and it represents the intersection point formed by the intersection of the first identification line and the second identification line.

[0077] In a specific implementation, in order to achieve the stitching of two images, at least two stitching references should be determined, so as to use the at least two stitching references to determine the relative position relationship and relative angle relationship of the two images to be stitched.

[0078] S120: Based on the overlap of stitching reference pixels with the same relative position identifiers in adjacent microscopic images to be stitched, stitching and fusing the adjacent microscopic images to be stitched is performed to obtain a stitched microscopic image.

[0079] After determining the stitching reference of the microscopic images to be stitched, the adjacent relationship of the microscopic images to be stitched can also be determined.

[0080] In some embodiments, the glass slide used has different grayscales for each first identification line and different grayscales for each second identification line, and the corresponding line position identification is an absolute identification. The corresponding two-dimensional relative position identification of each stitching reference can be used as an absolute representation of the relative position relationship of the images to be stitched, and the adjacent microscopic images to be stitched can be determined based on the two-dimensional relative position identification of the stitching reference. Subsequently, the stitching reference pixels with the same relative position identification in the adjacent microscopic images to be stitched can be aligned as a reference to perform stitching and fusion processing on the adjacent microscopic images to be stitched to obtain a stitched microscopic image.

[0081] In some other embodiments, the grayscales of adjacent first identification lines and adjacent second identification lines in the glass slide are different. In this case, the adjacent relationship of each image to be stitched can be determined based on the acquisition of the image to be stitched and the movement path of the lens of the microscopic imaging system relative to the glass slide, and then the adjacent images to be stitched can be determined. Subsequently, the stitching reference pixels with the same relative position identification in the adjacent microscopic images to be stitched can be aligned as a reference to perform stitching and fusion processing on the adjacent microscopic images to be stitched to obtain a stitched microscopic image.

[0082] Figure 4 is a microscopic image to be stitched collected during the specific implementation. Figure 5 It is another microscopic image to be stitched collected in the specific implementation. After the pixel coordinates of reference point 1 and reference point 2 (reference point 1 and reference point 2 are formed by the intersection of the first identification line and the second identification line of different grayscales) on the two microscopic images to be stitched are determined by the aforementioned S110 process, the reference point 1 and the reference point 2 in the two images are then aligned as a reference, and the images can be stitched. Figure 6 Is Figure 4 and Figure 5 The microscopic image formed by stitching two microscopic images to be stitched together shows that the two microscopic images to be stitched together are correctly stitched together according to the above method. It should be noted here that Figure 4 and Figure 5 The two microscopic images to be stitched are two images with the same magnification.

[0083] As analyzed above, by using the aforementioned glass slide to prepare the sample and using a microscope imaging system to photograph the aforementioned glass slide to obtain the microscopic image to be stitched, the first identification line and the second identification line in the microscopic image to be stitched can be used to determine the stitching reference, and then based on the stitching reference, the correspondence between the pixels in the two adjacent microscopic images to be stitched can be quickly determined, thereby realizing rapid alignment and stitching of the images.

[0084] In some applications, when using a microscopic imaging system (especially a system that can perform image self-correction according to imaging magnification and optical lens distortion characteristics) to collect images of different regions of a sample (that is, to form microscopic images of different regions to be stitched), the microscopic imaging system will automatically focus according to the clarity of the organisms in the image region, thereby capturing as clear an image as possible. However, the aforementioned automatic focus causes different degrees of geometric distortion and positioning offset of different microscopic images to be stitched, and in this case, direct stitching of two adjacent microscopic images will result in misalignment.

[0085] To avoid the aforementioned problem, before executing the aforementioned S120, the following S130-S150 may also be executed.

[0086] S130: Determine the number of interval pixels between two stitching references in the microscopic image to be stitched.

[0087] Here Figure 4 For example, determining the number of interval pixels corresponding to the target reference in the splicing reference is to determine the number of interval pixels between the aforementioned reference point 1 and reference point 2.

[0088] S140: Determine a zoom factor according to the number of interval pixels and the corresponding number of standard pixels.

[0089] S150: scaling the corresponding microscopic images to be stitched according to the scaling factor to determine a scaled microscopic image.

[0090] The standard number of pixels wherein the standard number of pixels is determined based on microscopic imaging of the two stitched references at a standard magnification, or based on microscopic imaging of similar references at the same distance from the two stitched references.

[0091] It can be imagined that if the number of standard pixels is smaller than the aforementioned number of interval pixels, the shooting Figure 4 The image magnification is greater than the standard magnification. If the standard pixel number is greater than the aforementioned interval pixel number, the image will be magnified more than the standard pixel number. Figure 4 The image magnification is smaller than the standard magnification.

[0092] Specifically, the zoom factor is determined according to the number of intervals and the corresponding number of standard pixels, and a divisor is obtained by dividing the number of standard pixels and the number of interval pixels, and the divisor is used as the zoom factor.

[0093] After obtaining the aforementioned zoom factor, the zoom factor can be used to zoom the microscopic images to be stitched to determine the zoomed microscopic images. It can be imagined that the zoomed microscopic images of the microscopic images to be stitched use the same image magnification, and the number of imaging pixels of the same length on each zoomed microscopic image is the same.

[0094] In the case of executing the aforementioned S130 - S150 , the aforementioned S120 performs stitching processing on adjacent microscopic images to be stitched, and performs stitching and fusion processing on the zoomed microscopic images.

[0095] In some other applications, due to the optical imaging system, the microscopic images to be stitched formed by microscopic imaging have obvious distortion in specific areas (especially edge areas). Directly using distorted images for image stitching may cause the edges of aquatic organisms to not transition smoothly after stitching, making it difficult to automatically identify the types of aquatic organisms later. To solve this problem, in some embodiments of the present disclosure, before executing the aforementioned S120, the following S160-S190 may also be executed.

[0096] S160: Determine the number of pixels between two adjacent stitching references in the microscopic image to be stitched.

[0097] S170: Determine a zoom factor according to the number of interval pixels and the corresponding number of standard pixels.

[0098] The implementation process of S160-S170 is the same as that of S130-S140 above, and will not be repeated here. In a specific implementation, the two critical stitching references determined are preferably stitching references on the same first identification line or second identification line. In addition, in a specific implementation, the number of first identification lines and second identification lines included in the microscopic image to be stitched is more than two, so that different areas of the microscopic image to be stitched obtain different zoom factors, reflecting the image distortion characteristics caused by the optical imaging system.

[0099] S180: Scaling the pixel area of ​​the microscopic image to be stitched including the two stitching references according to the zoom factor to obtain a partially scaled image.

[0100] After obtaining the zoom factors corresponding to different regions of different microscopic images to be stitched, the zoom factors can be used to perform zoom processing on the region corresponding to the image to be stitched, specifically, the local pixel region including the two stitching references can be zoomed to obtain a partial zoom region. The local zoom region obtained by the above zoom processing can partially correct the image distortion caused by the imaging of the optical system.

[0101] S190: stitching the partial zoomed images to obtain a zoomed microscopic image.

[0102] After obtaining the aforementioned partial zoomed images corresponding to different local pixel areas according to S170, the partial zoomed images can be stitched together to form a zoomed microscopic image. It should be noted here that when performing image stitching, the image stitching can be performed based on the aforementioned stitching reference. And in the process of image stitching, the zoomed microscopic image corresponding to the central area of ​​the microscopic image to be stitched can be used as a basis to stitch other adjacent zoomed microscopic images in sequence. In the process of stitching, for some blank areas (areas without filled pixels), their pixel grayscale can also be determined by pixel resampling to avoid blank pixel areas in the obtained zoomed microscopic image.

[0103] It should be noted that the premise for executing the aforementioned S160 - S190 is that during the microscopic imaging process, each microscopic image to be stitched is imaged with the same magnification.

[0104] When the aforementioned S130 - S150 or S160 - S190 are executed, S120 specifically performs stitching and fusion processing on the zoomed microscopic images corresponding to the adjacent images to be stitched.

[0105] In actual applications, when the glass slide is moved relative to the imaging lens of the microscope imaging system according to the preset step length, the glass slide may rotate slightly. In this case, image stitching cannot be achieved by translation fusion stitching, and rotation correction processing is required. Therefore, in some embodiments, before executing the aforementioned S120, the following S200 may also be executed.

[0106] S200: performing rotation correction on the corresponding zoomed microscopic image based on the position coordinates of the stitching reference in each image to be stitched.

[0107] Still Figure 4 For example, the coordinates of reference point 1 are (x 11 ,y 11 ), the coordinates of reference point 2 are (x 12 ,y 12 ), then the rotation angle of the image to be stitched relative to the horizontal direction is determined to be θ1 = tan -1 y 12 -y 11 Zoomed microscopy images of stitched images

[0108] x 12 -x 11

[0109] The rotation correction may be performed by rotating the image according to θ1 so that the straight line where the reference point 1 and the reference point 2 are located is a horizontal straight line.

[0110] By using the above method, each microscopic image to be stitched can be corrected to the reference angle, so as to adapt to the subsequent correct stitching. In the case of executing the above S200, the above S150 or S190 specifically stitches and fuses the zoomed microscopic images that have been subjected to rotation correction processing.

[0111] The above solution is to first scale the microscopic image to be stitched and then rotate it. In other embodiments, the microscopic image to be stitched may be rotationally corrected, and then the rotationally corrected microscopic image to be stitched may be scaled according to the scaling factor to determine the scaled microscopic image.

[0112] In the foregoing, it is mentioned that adjacent microscopic images to be stitched are stitched and fused to obtain a stitched microscopic image. Here, how to perform stitching and fusion to obtain a stitched microscopic image is analyzed. In some embodiments, the stitching and fusion of the microscopic images to be stitched may include the following S121-S123.

[0113] S121: Based on the overlap of pixels in adjacent images to be stitched, the pixels to be fused and the pixels not to be fused in the adjacent images to be stitched are determined.

[0114] The pixels to be fused are pixels representing the same imaging object in the adjacent images to be stitched, that is, pixels corresponding to the overlapping imaging area. The corresponding pixels not to be fused are pixels formed without overlapping imaging area.

[0115] S122: Calculate the fusion grayscale of the pixel to be fused as the pixel grayscale of the fused pixel.

[0116] S123: splicing the pixels that do not need to be fused and the pixels that need to be fused according to the positional relationship between the pixels that do not need to be fused and the pixels that need to be fused to obtain the spliced ​​microscopic image.

[0117] For the pixels to be fused, the mean grayscale of the corresponding pixels in the two images to be stitched can be calculated, and the mean grayscale is used as the pixel grayscale of the fused pixel.

[0118] After determining the pixel grayscale of the fused pixel, the pixel before fusion corresponding to the fused pixel and the pixel that does not need to be fused can be spliced ​​to obtain a spliced ​​microscopic image. In a specific implementation, the pixel position of the splicing reference in the two microscopic images to be spliced ​​(after rotation correction and scaling processing) can be used to determine the number of pixels that need to be translated in the horizontal and vertical directions for one of the microscopic images to be spliced, and then the pixel coordinates of each point after splicing can be determined according to the aforementioned number of pixels translated, and then it can be determined which pixels are to be fused and which are not to be fused.

[0119] In a specific implementation, due to the aforementioned scaling and rotation processing, some pixels may have null values. In this case, the grayscale value of the pixel may be set to the maximum brightness.

[0120] According to the previous analysis, the pixel grayscale of each point in the stitched microscopic image can be expressed by the following formula.

[0121]

[0122] Among them, P1(i,j) is the pixel grayscale in the first microscopic image to be stitched, and P2(i,j) is the pixel grayscale in the second microscopic image to be stitched. In the case of , the pixel is characterized as a pixel that needs to be fused, and in other cases, it is a pixel that does not need to be fused.

[0123] In a specific implementation, a larger image canvas may be established, adjacent images to be stitched may be determined in sequence according to the aforementioned method, and each image to be stitched may be stitched in sequence according to the stitching order until an image of the entire microscopic imaging area 102 of the slide 100 is obtained.

[0124] In some embodiments, the microscopic images to be stitched obtained by using the microscopic imaging system are color images. In order to achieve fast stitching of the images to be stitched, the color images can be converted into grayscale images, specifically, I gray x,y=0.299×R(x,y)+0.587×G(x,y)+0.114×B(x,y) to get the grayscale value of the pixel (x,y).

[0125] In addition to providing the aforementioned method for stitching microscopic scanning images of aquatic organisms, the embodiment of the present disclosure also provides a microscopic image stitching device.

[0126] Figure 7 Schematic diagram of a microscopic image stitching device provided in an embodiment of the present disclosure. Figure 7 As shown, the microscopic image stitching device 700 includes a stitching reference identification unit 701 and a stitching unit 702 .

[0127] The stitching reference identification unit 701 is used to identify the first identification line and the second identification line in each microscopic image to be stitched, and determine the position coordinates of the stitching reference in the microscopic image to be stitched based on the position characteristics of the identified first identification line and the second identification line, and determine the two-dimensional position identification of each stitching reference according to the pixel grayscale of the first identification line and the second identification line; wherein the microscopic image to be stitched is obtained by photographing the sample based on the glass slide prepared as above by the microscopic imaging system.

[0128] The stitching unit 702 is used to perform stitching and fusion processing on adjacent microscopic images to be stitched, based on aligning stitching reference pixels with the same two-dimensional position identifier in adjacent microscopic images to be stitched, to obtain a stitched microscopic image.

[0129] In some embodiments, the microscopic image stitching device 700 further includes an image correction unit, which is used to determine the number of interval pixels between two stitching references in the microscopic image to be stitched; determine the zoom factor according to the number of interval pixels and the corresponding number of standard pixels; wherein the number of standard pixels is determined by performing microscopic imaging of the two stitching references at a standard magnification, or performing microscopic imaging of similar references at the same distance from the two stitching references; and scaling the corresponding microscopic image to be stitched according to the zoom factor to determine the zoomed microscopic image; or, scaling the local pixel area of ​​the microscopic image to be stitched including the two stitching references according to the zoom factor to obtain a local zoomed image; and stitching the partial zoomed images to obtain a zoomed microscopic image. Correspondingly, the stitching unit 702 performs stitching and fusion processing on the zoomed microscopic images corresponding to adjacent images to be stitched.

[0130] In some embodiments, the image correction unit performs rotation correction on the corresponding zoomed microscopic image based on the position coordinates of the stitching reference in each image to be stitched; the stitching unit 702 performs stitching and fusion processing on the zoomed microscopic image that has undergone rotation correction processing.

[0131] Alternatively, the image correction unit performs rotation correction on each image to be stitched based on the position coordinates of the stitching reference in each image to be stitched; then, the corresponding microscopic image to be stitched that has undergone rotation correction is scaled according to the scaling factor to determine a scaled microscopic image.

[0132] During the stitching process, the stitching unit 702 uses the overlap of pixels in adjacent images to be stitched as a reference to determine the pixels to be fused and the pixels not to be fused in the adjacent images to be stitched, where the pixels to be fused are pixels in the adjacent images to be stitched that represent the same imaging object; calculates the fusion grayscale of the pixels to be fused as the pixel grayscale of the fused pixels; then stitches the pixels not to be fused and the pixels to be fused according to their positional relationship to obtain a stitched microscopic image.

[0133] In some embodiments, the stitching reference is the intersection of the first identification line and the second identification line; the stitching reference identification unit 701 determines the position coordinates of the line intersection point based on the pixel coordinates of the identified first identification line and the second identification line or the coordinates of the straight lines on which they are located; determines the first dimensional position identifier according to the pixel grayscale of the first identification line forming the line intersection point, and determines the second dimensional position identifier according to the pixel grayscale of the second identification line forming the line intersection point; combines the first dimensional position identifier and the second dimensional position identifier to obtain a two-dimensional position identifier of the line intersection point.

[0134] The embodiment of the present disclosure also provides a computing device for implementing the aforementioned method. Figure 8 Schematic diagram of the structure of the computing device provided by the embodiment of the present disclosure. Figure 8 , which shows a schematic diagram of the structure of a computing device 800 suitable for implementing the embodiments of the present disclosure. Figure 8 The computing device shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0135] like Figure 8 As shown, the computing device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory ROM 802 or a program loaded from a storage device 808 to a random access memory RAM 803. Various programs and data required for the operation of the computing device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output I / O interface 805 is also connected to the bus 804.

[0136] Typically, the following devices may be connected to the I / O interface 805: input devices 805 including, for example, a touch screen, a touch pad, a camera, a microphone, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication devices 809 may allow the computing device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The computing device 800 is shown with various devices, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0137] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0138] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable storage medium, a computer-readable signal medium, or any combination of the above two.

[0139] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0140] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer readable program codes. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0141] In some embodiments, the client and computing device can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), internets (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

[0142] The computer-readable medium may be included in the computing device, or may exist independently without being incorporated into the computing device.

[0143] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the tester computer, partially on the tester computer, as a separate software package, partially on the tester computer and partially on a remote computer, or entirely on a remote computer or computing device. In cases involving a remote computer, the remote computer may be connected to the tester computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0144] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system based on hardware that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The name of the unit does not, in some cases, constitute a limitation on the unit itself. The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0146] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A slide glass, characterized in that: include: A slide body, and a reticulated identification line arranged at a microscopic observation area of ​​the slide body; The reticulated identification lines include a plurality of first identification lines running through the microscopic observation area along a first direction, and second identification lines running through the microscopic observation area along a second direction, wherein the first direction is different from the second direction; At least adjacent first identification lines are set to have different light processing characteristics, so that the grayscales of the lines after microscopic imaging under the same lighting conditions are different; at least adjacent second identification lines are set to have different light processing characteristics, so that the grayscales of the lines after microscopic imaging under the same lighting conditions are also different; The line spacing between adjacent first identification lines and the line spacing between adjacent second identification lines are both less than a threshold distance; the threshold distance is a distance to ensure that the microscopic images to be stitched formed after microscopic imaging have at least two first identification lines and at least one second identification line.

2. The slide glass according to claim 1, characterized in that: Each first identification line has different light processing characteristics, and the line grayscale of each first identification line changes sequentially after microscopic imaging; and / or, Each of the second identification lines has different light processing characteristics, and the line grayscale of each of the second identification lines changes sequentially after microscopic imaging.

3. The slide glass according to claim 1 or 2, characterized in that: The first identification line and / or the second identification line are etching lines formed by etching the microscopic observation area; at least the etching depths and / or etching widths of adjacent first identification lines are different; and / or, at least the etching depths and / or etching widths of adjacent second identification lines are different; or, The first marking line and / or the second marking line are deposited lines formed by deposition on the microscopic observation area; at least adjacent first marking lines have different deposition thicknesses and / or different light absorption characteristics; and / or at least adjacent second marking lines have different deposition thicknesses and / or different light absorption characteristics.

4. The slide glass according to claim 1 or 2, characterized in that: The first marking lines are arranged on the slide body at equal intervals; And / or, each of the second identification lines is arranged on the slide body at equal intervals.

5. A method for stitching microscopic scanning images of aquatic organisms, characterized in that: include: Identify the first identification line and the second identification line in each microscopic image to be stitched, and determine the position coordinates of the stitching reference in the microscopic image to be stitched based on the positions of the identified first identification line and the second identification line, and determine the two-dimensional position identification of each stitching reference according to the pixel grayscale of the first identification line and the second identification line; wherein the microscopic image to be stitched is obtained by photographing a sample prepared based on the glass slide according to any one of claims 1 to 4 by a microscopic imaging system; Based on the overlap of stitching reference pixels with the same two-dimensional position mark in adjacent microscopic images to be stitched, stitching and fusion processing is performed on adjacent microscopic images to obtain a stitched microscopic image.

6. The splicing method according to claim 5, characterized in that: Before performing the stitching and fusing process on the adjacent microscopic images to be stitched, the method further comprises: determining the number of interval pixels between two stitching references in the microscopic images to be stitched; Determining a zoom factor based on the number of spaced pixels and a corresponding number of standard pixels; wherein the number of standard pixels is determined based on microscopic imaging of the two stitched references at a standard magnification, or microscopic imaging of a similar reference at the same distance from the two stitched references; Performing scaling processing on the corresponding microscopic images to be stitched according to the scaling factor to determine a scaled microscopic image; or performing scaling processing on the local pixel area of ​​the microscopic image to be stitched including the two stitching references according to the scaling factor to obtain a local scaled image; and stitching the partial scaled images to obtain a scaled microscopic image; The stitching and fusing process of the adjacent microscopic images to be stitched includes: stitching and fusing the zoomed microscopic images corresponding to the adjacent images to be stitched.

7. The method for stitching microscopic scanning images of aquatic organisms according to claim 6, characterized in that: Before performing splicing and fusing processing on the zoomed microscopic images corresponding to the adjacent images to be spliced, the method further includes: Based on the position coordinates of the stitching reference in each image to be stitched, the corresponding zoomed microscopic image is rotationally corrected; The stitching and fusing process of the adjacent zoomed images to be stitched includes: stitching and fusing the zoomed microscopic images that have been subjected to rotation correction; Alternatively, rotation correction is performed on each image to be stitched based on the position coordinates of the stitching reference in each image to be stitched; The step of scaling the corresponding microscopic images to be spliced ​​according to the scaling factor to determine the scaled microscopic image includes: scaling the corresponding microscopic images to be spliced ​​that have undergone rotation correction according to the scaling factor to determine the scaled microscopic image.

8. The method for stitching microscopic scanning images of aquatic organisms according to claim 5, characterized in that: Based on the overlap of stitching reference pixels with the same two-dimensional position mark in adjacent microscopic images to be stitched, the adjacent microscopic images to be stitched are stitched and fused to obtain a stitched microscopic image, including: Based on the overlap of pixels in adjacent images to be stitched, the pixels to be fused and the pixels not to be fused in the adjacent images to be stitched are determined, wherein the pixels to be fused are the pixels in the adjacent images to be stitched that represent the same imaging object; Calculate the fused grayscale of the pixel to be fused as the pixel grayscale of the fused pixel; The pixels that do not need to be fused and the pixels that need to be fused are spliced ​​according to the positional relationship between the pixels that do not need to be fused and the pixels that need to be fused to obtain the spliced ​​microscopic image.

9. The method for stitching microscopic scanning images of aquatic organisms according to any one of claims 5 to 8, characterized in that: The splicing reference is the intersection point of the first identification line and the second identification line; The determining the position coordinates of the stitching reference in the to-be-stitched microscopic image based on the positions of the identified first identification line and the second identification line comprises: determining the position coordinates of the intersection point of the lines based on the position coordinates of the identified first identification line and the second identification line or the coordinates of the straight lines on which they are located; The step of determining the two-dimensional position marks of each splicing reference according to the pixel grayscales of the first identification line and the second identification line includes: Determine a first-dimensional position marker according to the pixel grayscale of a first marking line forming the line intersection point, and determine a second-dimensional position marker according to the pixel grayscale of a second marking line forming the line intersection point; The first dimensional position identifier and the second dimensional position identifier are combined to obtain a two-dimensional position identifier of the line intersection point.

10. A computing device, characterized in that: It comprises a processor and a memory, wherein the memory is used to store a computer program; when the computer program is loaded by the processor, the processor executes the method for stitching microscopic scanning images of aquatic organisms as described in any one of claims 5 to 9.

Images