Microscopic image processing method and apparatus

Abstract

The application provides a kind of microscopic image processing method and device, which can be applied to image processing technical field.The method comprises: collecting the sub-image sequence of sample, and obtaining the overall low-resolution image of sample, the sub-image sequence is optical microscopic sub-image sequence and / or backscattered electron sub-image sequence;According to the imaging type, perform differential preprocessing, obtain the preprocessed sub-image sequence;According to the aspect ratio of overall low-resolution image of sample and the total number of sub-image, determine the grid size of panorama stitching;Call external stitching tool to run in the working mode of only generating coordinate file without outputting panoramic large image, obtain the stitching coordinate information of each sub-image;According to the grid size, create a blank canvas, and according to the coordinate information, map each sub-image to the corresponding position of blank canvas, generate panoramic stitched image;According to the imaging type, perform differential post-processing on panoramic stitched image, generate standardized result map.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a microscopic image processing method and apparatus. Background Technology

[0002] Image stitching is an important aspect of image processing, aiming to combine multiple images with overlapping areas into a single high-resolution panoramic image. This technology is widely used in fields such as microscopic image analysis, digital video, medical image processing, virtual reality, and remote sensing image processing. In microscopic imaging, due to the limited field of view of high-magnification objectives, a single frame image cannot cover the entire sample area. Therefore, it is necessary to acquire and stitch together a sequence of sub-images with overlapping areas to obtain a panoramic image that combines a large field of view with high resolution.

[0003] In planetary science sample research, microscopic imaging of extraterrestrial return samples is a crucial tool for analyzing their mineral composition, microstructure, and geological history. Optical microscopes can obtain color morphological images of the sample surface, while field emission scanning electron microscopy (FEM) with backscattered electron (BSE) imaging mode can produce grayscale images reflecting the atomic number contrast of the sample. To comprehensively understand the macroscopic morphology, statistical grain characteristics, and precisely locate the study area, numerous sub-images with overlapping regions need to be stitched together to form a high-resolution panoramic image covering the entire sample. Such panoramic images can serve research needs such as remote characterization of sample mineralogical and petrological features, searching for rare mineral phases, and repeated observations of valuable samples.

[0004] Currently, the stitching of microscopic images mainly relies on general image stitching tools based on feature point matching (such as the Microscopy Image Stitching Tool, MIST). The MIST tool extracts feature points from the overlapping regions of adjacent sub-images, calculates candidate translation amounts, and then generates a stitched image after translation optimization. However, the aforementioned existing tools and methods have the following significant limitations when processing microscopic images of planetary science samples:

[0005] First, the size of the stitched image is limited. Existing stitching tools have a hard limit on the total number of pixels in the output image. Taking MIST as an example, it is limited by the size of Java arrays and cannot directly generate ultra-large panoramic images. Panoramic images of planetary science samples (such as meteorite light sheets or returned samples) typically contain tens of thousands of sub-images, and the final panoramic image can have a pixel scale of 10 to 150 gigapixels, with a storage capacity of tens of gigabytes. When the total number of sub-images exceeds the tool's processing limit, the stitching process will terminate abnormally due to memory overflow, resulting in the inability to produce a complete panoramic image.

[0006] Second, the processing workflow is simplistic and lacks customized pre- and post-processing procedures tailored to the specific characteristics of the samples. On the one hand, optical microscopic images are generally affected by factors such as uneven light source and inconsistent response between the lens center and edges during acquisition, resulting in "flat-field" noise due to uneven illumination distribution. This manifests as a bright center and dark edges in the image. If this noise is not effectively corrected before stitching, it will directly affect the stitching accuracy and panoramic image quality. On the other hand, after backscattered electron images are stitched, researchers usually need to overlay a coordinate grid on the panoramic image to accurately locate and identify sample particle regions. However, existing stitching tools do not have such dedicated post-processing functions for planetary science research.

[0007] Third, the level of automation is low. For samples of different shapes and sizes, the grid layout (i.e., the number of rows and columns of sub-images) required for stitching must be manually set by operators based on experience. Since planetary science samples vary in shape (e.g., rock fragments are circular, powder samples are dispersed particles), manually setting the grid size is not only inefficient but also difficult to standardize and process in batches, especially when dealing with a large number of samples. Furthermore, the MIST tool requires samples to be acquired in a regular grid pattern, and the overlap error between adjacent sub-images must be less than a preset threshold, which further limits its tolerance for irregular samples or acquisition deviations.

[0008] Fourth, there is a lack of scientific metadata. The stitched images generated by existing stitching tools are usually just simple combinations of the original images, lacking standardized metadata specifically required for scientific data management. Specifically, backscattered electron panoramas lack coordinate grids for quickly locating sample particle regions and scale bars for indicating spatial scale, which is not conducive to data archiving, sharing, and area identification; optical panoramas also lack standardized display views that match the actual observation shape of the sample (such as a circular light sheet base).

[0009] In summary, existing microscopic image stitching methods suffer from limitations in size, simplistic workflows, low automation, and lack of scientific annotation when processing images of planetary science samples. Therefore, there is an urgent need in this field for a method that can automatically, efficiently, and with high precision process microscopic images of recoverable astronomical samples and generate panoramic images that conform to scientific research standards. Summary of the Invention

[0010] (a) Technical problems to be solved

[0011] To address at least one of the aforementioned problems in the prior art, this invention provides a microscopic image processing method and apparatus. By constructing a complete technical path covering differentiated preprocessing, adaptive grid size inference, hybrid stitching, and standardized post-processing, it achieves fully automated generation of microscopic images of planetary science samples from raw acquisition to standardized result images.

[0012] (II) Technical Solution

[0013] To address the aforementioned technical problems, embodiments of the present invention provide a microscopic image processing method and apparatus.

[0014] According to a first aspect of the present invention, a microscopic image processing method is provided, comprising: acquiring a sub-image sequence of a sample and obtaining a low-resolution image of the entire sample, wherein the sub-image sequence is an optical microscopic sub-image sequence and / or a backscattered electron sub-image sequence; performing differential preprocessing according to the imaging type of the sub-image sequence to obtain a preprocessed sub-image sequence; determining the grid size for panoramic stitching according to the aspect ratio of the overall low-resolution image of the sample and the total number of sub-images in the sub-image sequence; calling an external stitching tool to run in a working mode that only generates coordinate files without outputting a panoramic image; if the external stitching tool runs normally, obtaining the coordinate file it generates; if the external stitching tool terminates abnormally due to exceeding the total number of output pixels, reading the generated historical coordinate file in response to the captured abnormal information; creating a blank canvas according to the grid size, and mapping each preprocessed sub-image to the corresponding position on the blank canvas according to the coordinate information in the coordinate file to generate a panoramic stitched image; and performing differential postprocessing on the panoramic stitched image according to the imaging type to generate a standardized result image.

[0015] In some exemplary embodiments, differential preprocessing is performed according to the imaging type of the sub-image sequence, including: when the sub-image sequence is an optical microscopy sub-image sequence, constructing a flat-field image and performing flat-field correction on the optical microscopy sub-image sequence to obtain a flat-field corrected optical sub-image sequence; when the sub-image sequence is a backscattered electron sub-image sequence, directly using the backscattered electron sub-image sequence as the preprocessed sub-image sequence.

[0016] In some exemplary embodiments, constructing a flat-field image and performing flat-field correction on an optical microscopy sub-image sequence includes: selecting blank sub-images from the optical microscopy sub-image sequence that do not contain the sample coverage area; calculating the pixel-level mean of the gray values ​​of multiple blank sub-images to generate a flat-field image; and dividing the difference between each original optical microscopy sub-image and the dark current background image by the flat-field image that has been normalized by the gray-level mean to obtain the flat-field corrected optical sub-image.

[0017] In some exemplary embodiments, when the sample is in powder form and the number of blank sub-images is less than a preset threshold, the grayscale mean is calculated after performing low-frequency filtering or morphological opening processing on all optical microscopy sub-images to construct a flat field image.

[0018] In some exemplary embodiments, determining the grid size for panoramic stitching specifically includes: using the product of the number of grid columns attempted and the fixed width pixel count of a single sub-image as the estimated total width pixel count; using the product of the number of grid rows attempted and the fixed height pixel count of a single sub-image as the estimated total height pixel count; using the ratio of the estimated total width pixel count to the estimated total height pixel count as the theoretical aspect ratio of the stitched image; calculating the theoretical aspect ratio corresponding to each combination of grid column and row counts, and selecting the combination with the smallest difference between the theoretical aspect ratio and the aspect ratio of the overall low-resolution image of the sample as the grid size.

[0019] In some exemplary embodiments, mapping the preprocessed sub-images to corresponding positions on a blank canvas to generate a panoramic stitched image includes: using the difference between the maximum and minimum horizontal values ​​of each coordinate value in the coordinate file as the canvas width, and the difference between the maximum and minimum vertical values ​​of each coordinate value in the coordinate file as the canvas height, and creating a blank canvas according to the canvas width and canvas height; sequentially reading each preprocessed sub-image and its corresponding coordinate information, and placing each sub-image in the corresponding coordinate area of ​​the blank canvas according to the position parameters recorded in the coordinate information; and filling the empty areas in the grid size that are not covered by the sub-images with the background color.

[0020] In some exemplary embodiments, differentiated post-processing is performed on the panoramic stitched image according to the imaging type, including: when the imaging type is an optical microscopic image, background color matching and circular base synthesis are performed on the panoramic stitched image; when the imaging type is a backscattered electron image, mask extraction, circular background synthesis with coordinate grids and scale bar addition are performed on the panoramic stitched image.

[0021] In some exemplary embodiments, performing background color matching and circular base synthesis on a panoramic stitched image includes: calculating the average pixel color value of the edge region of the panoramic stitched image; generating a global background color that coordinates with the edge color of the panoramic stitched image based on the average pixel color value; generating a base image with a circular field of view, the size of which matches the size of the panoramic stitched image; and superimposing the panoramic stitched image within the circular field of view of the base image, so that the synthesized image presents a circular observation field of view consistent with the actual base shape of the sample.

[0022] In some exemplary embodiments, the circular background composition with coordinate grid includes: generating a coordinate grid background plate with a preset number of columns and a preset number of rows, wherein the horizontal axis of the coordinate grid background plate is marked with a numerical sequence number and the vertical axis is marked with a letter sequence number; cropping the coordinate grid background plate into a circular background plate; constructing a binary mask for extracting sample particle regions based on the grayscale difference between the sample region and the background region in the panoramic stitched image; and superimposing the sample particle regions within the coverage area of ​​the binary mask onto the grid layer of the circular background plate, so that the grid lines remain visible in the gap regions of the sample particles and the edge regions of the circular background plate.

[0023] According to a second aspect of the present invention, a microscopic image processing apparatus is provided, comprising: an image acquisition module for acquiring a sub-image sequence of a sample and obtaining a low-resolution image of the entire sample, wherein the sub-image sequence is an optical microscopic sub-image sequence and / or a backscattered electron sub-image sequence; a preprocessing module for performing differential preprocessing according to the imaging type of the sub-image sequence to obtain a preprocessed sub-image sequence; a grid determination module for determining the grid size for panoramic stitching according to the aspect ratio of the overall low-resolution image of the sample and the total number of sub-images in the sub-image sequence; a coordinate generation module for calling an external stitching tool to operate in a working mode that generates only coordinate files without outputting a panoramic image; if the external stitching tool operates normally, acquiring the coordinate file it generates; if the external stitching tool terminates abnormally due to exceeding the total number of output pixels, reading the generated historical coordinate file in response to the captured abnormal information; an initial generation module for creating a blank canvas according to the grid size and mapping each preprocessed sub-image to the corresponding position on the blank canvas according to the coordinate information in the coordinate file to generate a panoramic stitched image; and a target generation module for performing differential post-processing on the panoramic stitched image according to the imaging type to generate a standardized result image.

[0024] (III) Beneficial Effects

[0025] As can be seen from the above technical solutions, the microscopic image processing method and apparatus provided by the embodiments of the present invention have at least the following beneficial effects:

[0026] (1) Improving image stitching quality: Differentiated preprocessing strategies were established for optical microscopic images and backscattered electron images. After flat-field correction, the background noise introduced by uneven illumination and lens response differences in optical microscopic images was eliminated, ensuring the consistency of image quality in the input stitching process. Backscattered electron images were directly entered into the stitching process, avoiding unnecessary computational overhead and artificial artifacts introduced by improper preprocessing. The above-mentioned differentiated preprocessing strategies significantly improved the stitching accuracy and panoramic image quality of different types of microscopic images.

[0027] (2) Automated adaptation of the splicing grid size is achieved. By obtaining the aspect ratio and the total number of sub-images of the overall low-resolution image of the sample, and using the degree of matching between the theoretical splicing aspect ratio and the actual aspect ratio of the sample as the criterion, all possible grid row and column combinations are automatically traversed and the optimal solution is selected. This avoids the uncertainty caused by the need for manual setting of grid size based on experience in traditional methods. This invention achieves automated adaptation of samples of different shapes and sizes, and can determine the optimal splicing layout without manual intervention, significantly improving processing efficiency and batch processing capabilities.

[0028] (3) Breaking through the upper limit constraint of output pixels, realizing the generation of ultra-large panoramic images. By calling the external stitching tool to run in the working mode of generating only coordinate files without outputting panoramic images, and when the stitching tool terminates abnormally due to the total number of output pixels exceeding the limit, the generated historical coordinate files are read, and each sub-image is mapped to a blank canvas based on the coordinate information to complete the stitching. This breaks through the hard limit of the external stitching tool on the total number of output image pixels, so that the generation of high-resolution panoramic images of ultra-large samples is no longer limited by the memory or array capacity constraints of the tool.

[0029] (4) To achieve the scientific and standardized output of panoramic results, differentiated standardized post-processing is performed according to the imaging type. For optical microscopic images, background color matching and circular base synthesis are performed to generate a standardized display view consistent with the shape of the actual sample base. For backscattered electron images, mask extraction is performed to separate the sample particle area, and coordinate grid and scale are superimposed so that the final result image has both rapid particle area positioning and spatial scale measurement functions, which facilitates the archiving, sharing and cross-institutional area identification of scientific data. Attached Figure Description

[0030] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0031] Figure 1 A flowchart illustrating a microscopic image processing method according to an embodiment of the present invention is shown schematically.

[0032] Figure 2 This diagram illustrates the imaging sequence during optical microscopic image acquisition according to an embodiment of the present invention.

[0033] Figure 3 The diagram illustrates the original morphology of an optical micrograph according to an embodiment of the present invention.

[0034] Figure 4 This illustration schematically shows the effect of a flat field image according to an embodiment of the present invention;

[0035] Figure 5This illustration schematically shows the effect of an optical micrograph after flat-field correction according to an embodiment of the present invention;

[0036] Figure 6 This illustration schematically shows the effect of a panoramic stitched image after the optical microscopic image stitching is completed according to an embodiment of the present invention;

[0037] Figure 7 This schematic diagram illustrates the effect of adding a circular base to an optical micrograph according to an embodiment of the present invention.

[0038] Figure 8 This illustration schematically shows the effect of a panoramic stitched image after the powder sample stitching is completed according to an embodiment of the present invention;

[0039] Figure 9 A schematic diagram illustrating the original form of a backscattered electron image according to an embodiment of the present invention is shown.

[0040] Figure 10 This illustration schematically shows the effect of a panoramic stitched image after backscattered electron image stitching is completed according to an embodiment of the present invention;

[0041] Figure 11 This illustration schematically shows the effect of backscattered electron normalization results after superimposing coordinate grids according to an embodiment of the present invention;

[0042] Figure 12 A schematic diagram illustrating the structure of a microscopic image processing method apparatus according to an embodiment of the present invention is shown.

[0043] Figure 13 A block diagram of an electronic device for a microscopic image processing method according to an embodiment of the present invention is shown schematically. Detailed Implementation

[0044] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0045] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0046] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0047] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0048] Figure 1 A flowchart illustrating a microscopic image processing method according to an embodiment of the present invention is shown.

[0049] like Figure 1 As shown, the microscopic image processing method according to an embodiment of the present invention includes steps S110 to S160.

[0050] In step S110, a sub-image sequence of the sample is acquired, and a low-resolution image of the entire sample is obtained. The sub-image sequence is an optical microscopy sub-image sequence and / or a backscattered electron sub-image sequence.

[0051] In this step, the target image is acquired first. Optical microscopic images and backscattered electron (BSE) images are captured using corresponding imaging methods. Optical microscopic images are obtained using light-sheet imaging for observing light-sheet samples; backscattered electron images are obtained using field emission scanning electron microscopy for analyzing microstructures.

[0052] In some embodiments of the present invention, each sub-image in the sub-image sequence has a fixed pixel size. For example, the pixel size of a single sub-image is fixed at 2500 × 1900 pixels. A preset percentage of overlap is maintained between adjacent sub-images; for example, the overlap is approximately 10%. The sub-images are captured in a grid pattern according to a preset imaging order.

[0053] Figure 2 A schematic diagram illustrating the imaging sequence during optical microscopic image acquisition according to an embodiment of the present invention is shown. Figure 2As shown, during acquisition, each sub-image is captured sequentially along a serpentine (S-shaped) path from left to right and top to bottom, where the arrow direction indicates the acquisition path of the sub-image sequence. Each sub-image retains a preset overlap area (e.g., approximately 10%) with adjacent sub-images in both the horizontal and vertical directions to ensure sufficient feature point matching basis for subsequent stitching. Figure 2 The numbers 1, 2, 3... indicate the acquisition order of the sub-images, which sequentially cover the entire target area of ​​the sample.

[0054] Simultaneously, a low-resolution image of the entire sample is acquired. This low-resolution image can be obtained in a single shot using a low-power objective lens or a stereomicroscope, or generated by the wide-field navigation function of a scanning electron microscope, and is used to reflect the overall macroscopic morphology of the sample. This low-resolution image is used for automatic inference of the stitching grid size in subsequent steps.

[0055] In step S120, differential preprocessing is performed according to the imaging type of the sub-image sequence to obtain the preprocessed sub-image sequence.

[0056] Because optical microscopic images and backscattered electron images have different imaging mechanisms, they employ different processing strategies in the preprocessing stage.

[0057] When the imaging type of the sub-image sequence is optical microscopy, a flat-field image is constructed from the optical microscopy sub-image sequence and flat-field correction is performed to obtain the flat-field corrected optical sub-image sequence.

[0058] Optical microscopic images are commonly affected by uneven light distribution and inconsistent lens center and edge responses during acquisition, resulting in flat-field noise that manifests as a bright center and dark edges. To eliminate this noise, a flat-field image needs to be constructed and flat-field correction performed.

[0059] Specifically, constructing and flattening a flat-field image involves the following steps: First, blank sub-images that do not contain the sample coverage area are selected from the optical microscopy sub-image sequence; then, the pixel-level mean of the grayscale values ​​of multiple blank sub-images is calculated to generate a flat-field image. The flat-field image is calculated according to the following formula:

[0060]

[0061] in, This represents the generated flat-field image, where N represents the number of blank sub-images selected that do not contain the sample coverage area. Let represent the pixel grayscale value matrix of the i-th blank sub-image.

[0062] Figure 3 A schematic diagram illustrating the original morphology of an optical micrograph according to an embodiment of the present invention is shown. Figure 3As shown, the original optical microscopic image is affected by uneven illumination, resulting in a vignetting phenomenon where the brightness is higher in the central area and gradually decreases at the edges and corners. If this phenomenon is not effectively corrected before stitching, it will directly affect the accuracy of feature point matching and the visual consistency of the panoramic image.

[0063] Figure 4 The illustration schematically shows a diagram illustrating the effect of a flat-field image according to an embodiment of the present invention. Figure 4 As shown, the flat-field image is generated from the gray-level average of multiple blank sub-images that do not contain the sample coverage area, and is used to characterize the illumination response distribution of the optical imaging system across the entire field of view. The flat-field image records the brightness attenuation pattern from the image center to the edge, providing a pixel-by-pixel normalized reference for subsequent correction.

[0064] After the flat-field image is constructed, flat-field correction is performed. First, the flat-field image is divided pixel by pixel by the gray-scale mean of all pixels in the flat-field image to achieve standardization and obtain the responsivity of each pixel. Then, the difference between each original optical microscopy sub-image and the dark current background image is divided by the flat-field image after gray-scale mean standardization to obtain the flat-field corrected optical sub-image.

[0065] Figure 5 The illustration schematically shows a planar correction effect of an optical micrograph according to an embodiment of the present invention. Figure 5 As shown, after flat field correction, the uneven lighting distribution phenomenon of bright center and dark periphery in the image is effectively eliminated, the overall brightness of the image tends to be uniform, and the grayscale consistency of the background area is significantly improved, providing a consistent input image for the subsequent stitching process.

[0066] When the sample is in powder form and the number of blank sub-images is less than a preset threshold, since the number of images captured in a single shot is large and the captured targets are small and scattered, all optical microscopy sub-images can be directly used as blank images to create flat-field images. That is, low-frequency filtering or morphological opening operations are performed on all optical microscopy sub-images and the gray-scale mean is calculated to construct flat-field images.

[0067] When the sub-image sequence is a backscattered electron image (BSE), since the background of a BSE image is relatively uniform and mainly reflects atomic number contrast, no flat-field correction is required. The BSE sub-image sequence can be directly used as the preprocessed sub-image sequence. This differentiated design not only reduces unnecessary computation time for BSE images, but more importantly, it avoids artificial artifacts caused by introducing flat-field correction into BSE images (such as over-smoothing that destroys fine mineral boundaries), ensuring the authenticity of the original data for subsequent component analysis.

[0068] In step S130, the grid size for panoramic stitching is determined based on the aspect ratio of the overall low-resolution image of the sample and the total number of sub-images in the sub-image sequence.

[0069] To accommodate the number of rows and columns of the image grid corresponding to samples of different sizes, it is necessary to automatically determine the grid size for sub-image stitching, namely the grid width w and height h.

[0070] Specifically, determining the grid size for panoramic stitching includes the following steps:

[0071] First, the product of the number of grid columns tried and the fixed width pixel count of a single sub-image is used as the estimated total width pixel count; the product of the number of grid rows tried and the fixed height pixel count of a single sub-image is used as the estimated total height pixel count; and the ratio of the estimated total width pixel count to the estimated total height pixel count is used as the theoretical aspect ratio of the stitched image.

[0072] Then, the theoretical aspect ratio corresponding to each combination of the number of columns and rows of the grid is calculated, and the combination with the smallest difference between the theoretical aspect ratio and the aspect ratio of the overall low-resolution image of the sample is selected as the grid size.

[0073] More specifically, first, a low-resolution image of the entire sample is acquired, and its aspect ratio W / H is calculated, where W represents the width in pixels and H represents the height in pixels. Then, based on the total number N of sub-images used for stitching, all possible grid columns are iterated. With grid row number The combination is given by , where i is the combination number. The matching degree of each width-height combination is then calculated using the following formula. :

[0074]

[0075] Where 2500 and 1900 are the width and height in pixels of a single sub-image (fixed values), respectively. This represents the number of grid columns for the i-th combination. W represents the number of grid rows for the i-th combination, and W / H represents the aspect ratio of the overall low-resolution image of the sample. The smaller the value, the closer the theoretical aspect ratio of the spliced ​​image determined by the combination of the number of grid columns and rows is to the actual aspect ratio of the sample. The smallest combination is used as the optimal mesh size. This mechanism eliminates the uncertainty caused by manual settings and enables adaptive adaptation to samples of different shapes and sizes.

[0076] In step S140, the external stitching tool is invoked to run in a working mode that generates only coordinate files without outputting a panoramic image; if the external stitching tool runs normally, the generated coordinate files are obtained; if the external stitching tool terminates abnormally due to exceeding the total number of output pixels, the generated historical coordinate files are read in response to the captured abnormal information.

[0077] In this step, a hybrid stitching strategy is employed. The preprocessed sub-image sequence is imported into an external stitching tool. For example, the external stitching tool is a microscopic image stitching tool based on feature point matching.

[0078] Existing stitching tools have a hard limit on the total number of pixels in the output image. When the total number of pixels in the sub-images exceeds the tool's processing limit, the stitching process will terminate abnormally due to memory overflow. To solve this problem, this step prioritizes using the "generate coordinate files only, do not output panoramic images" working mode when calling external stitching tools. In this mode, the external stitching tool only performs feature point matching and generates stitching coordinate information for each sub-image, without rendering the panoramic image, thus avoiding abnormal termination due to exceeding the output pixel limit.

[0079] If the external stitching tool runs normally, the system acquires the generated coordinate file, which records the reference pixel coordinates of each sub-image in the panoramic canvas. If the external stitching tool terminates abnormally due to exceeding the total output pixel limit, the system automatically captures the exception information and, in response, reads the generated historical coordinate file. This strategy ensures that even if the total pixel count of the sub-images exceeds the processing limit of the external tool, the coordinate information of each sub-image with completed feature point matching can still be obtained.

[0080] In step S150, a blank canvas is created according to the grid size, and the preprocessed sub-images are mapped to the corresponding positions on the blank canvas according to the coordinate information in the coordinate file to generate a panoramic stitched image.

[0081] After obtaining the stitching coordinate information of each sub-image, mapping stitching needs to be performed based on the coordinate information. In this step, the size of the blank canvas is directly determined according to the coordinate range of each sub-image in the coordinate file, and the grid size is used to determine the position of the empty area and the background filling range.

[0082] Specifically, step S150 includes the following sub-steps:

[0083] First, use the difference between the maximum and minimum horizontal values ​​of each coordinate value in the coordinate file as the canvas width, and the difference between the maximum and minimum vertical values ​​of each coordinate value in the coordinate file as the canvas height, and create a blank canvas according to the canvas width and canvas height.

[0084] Next, each preprocessed sub-image and its corresponding coordinate information are read sequentially, and each sub-image is placed in the corresponding coordinate area of ​​the blank canvas according to the position parameters recorded in the coordinate information.

[0085] Finally, for any empty areas in the grid that are not covered by sub-images, fill them with the background color.

[0086] Figure 6 This illustration schematically shows the effect of a panoramic stitched image after optical microscopic image stitching is completed according to an embodiment of the present invention. Figure 6 As shown, after the above stitching process, all optical sub-images have been accurately mapped to the corresponding positions on the blank canvas according to the coordinate information, forming a complete panoramic image covering the entire target area of ​​the sample. The overlapping areas between the sub-images have been aligned according to the coordinate information, with no obvious misalignment or distortion.

[0087] Figure 9 A schematic diagram illustrating the original form of a backscattered electron image according to an embodiment of the present invention is shown. Figure 9 As shown, the backscattered electron image is a grayscale image. Its image contrast mainly reflects the difference in atomic number in different regions of the sample surface. The image background is relatively uniform, and it can be directly entered into the stitching process without the need for flat field correction preprocessing.

[0088] Figure 10 This illustration schematically shows the effect of a panoramic stitched image after backscattered electron image stitching is completed according to an embodiment of the present invention. Figure 10 As shown, the stitched backscattered electron panoramic image covers the entire scanned area of ​​the sample, fully presenting the morphology and distribution of each particle on the sample surface, providing basic data for subsequent mineral phase identification and particle statistics.

[0089] In step S160, differential post-processing is performed on the panoramic stitched image according to the imaging type to generate a standardized result image.

[0090] Specialized post-processing is performed on the stitched panoramic images to generate standard output images that can be directly used for scientific observation and analysis.

[0091] Specifically, when the imaging type is an optical microscopic image, background color matching and circular base synthesis are performed on the panoramic stitched image; when the imaging type is a backscattered electron image, mask extraction, circular background synthesis with coordinate grids, and scale bar addition are performed on the panoramic stitched image.

[0092] In some exemplary embodiments, performing background color matching and circular base composition on the panoramic stitched image includes the following sub-steps:

[0093] First, the average pixel color value of the edge region of the panoramic stitched image is calculated. Then, a global background color that coordinates with the edge color of the panoramic stitched image is generated based on the average pixel color value. Next, a base image with a circular field of view is generated, the size of which matches the size of the panoramic stitched image. Finally, the panoramic stitched image is superimposed on the circular field of view of the base image, so that the synthesized image presents a circular observation field of view consistent with the actual shape of the sample base.

[0094] Figure 7 The illustration schematically shows the effect of adding a circular base to an optical micrograph according to an embodiment of the present invention. Figure 7 As shown, the panoramic stitched image is placed within the circular field of view of the circular base image. The synthesized image presents a circular observation field of view that is completely consistent with the shape of the actual light sheet sample base, which facilitates researchers to conduct macroscopic observation and comparison of the sample area according to laboratory habits.

[0095] For the assembly results of powder samples, they usually already include a circular light plate base, so no additional synthesis base is required.

[0096] Figure 8 This illustration schematically shows the effect of a panoramic stitched image after the powder sample stitching is completed according to an embodiment of the present invention. Figure 8 As shown, the powder sample is dispersed on a circular base. The stitched panoramic image naturally retains the original circular base boundary, and the sample particles are clearly visible within the base area.

[0097] In some exemplary embodiments, the synthesis of a circular background with a coordinate grid from a backscattered electron image includes the following sub-steps:

[0098] First, a coordinate grid background with a preset number of columns and rows is generated. The horizontal axis of the coordinate grid background is marked with a numerical sequence number, and the vertical axis is marked with a letter sequence number. For example, the coordinate grid background has a specification of 24 columns × 26 rows, with the horizontal axis marked from 01 to 24 and the vertical axis marked from A to Z.

[0099] Secondly, the coordinate grid background board is cropped into a circular background board.

[0100] Then, based on the grayscale difference between the sample coverage area and the background area in the panoramic stitched image, a binary mask for extracting the sample particle area is constructed. Specifically, a blank mask with the same size as the panoramic stitched image is first created, and then the target area is initially extracted using a color thresholding method with the average grayscale value of the image edge area as a reference threshold, thus constructing the binary mask.

[0101] Finally, the sample particle area within the coverage area of ​​the binary mask is superimposed on the grid layer of the circular background plate, so that the grid lines remain visible in the gap area of ​​the sample particles and the edge area of ​​the circular background plate.

[0102] Figure 11 The diagram illustrates the effect of backscattered electron normalization after overlaying coordinate grids according to an embodiment of the present invention. Figure 11 As shown, the horizontal axis of the resulting graph is labeled with numerical serial numbers (01 to 24), and the vertical axis is labeled with alphabetical serial numbers (A to Z), forming a complete 24-column × 26-row coordinate grid system. Sample particle regions are precisely extracted using masking technology and superimposed on the coordinate grid background, with grid lines remaining visible in the inter-particle areas and at the edges of the circular background. This resulting graph allows researchers to quickly and accurately locate particle regions of interest using coordinates (e.g., "Area A-12"), greatly facilitating data archiving, inter-institutional collaboration, and region identification.

[0103] In some exemplary embodiments, the scale bar addition includes the following sub-steps: calculating the number of pixels corresponding to a unit physical length based on the pixel resolution parameters calibrated during the acquisition of the original image (e.g., the number of pixels per micrometer), and generating a scale bar graphic with the corresponding physical length; and compositing the scale bar graphic into a specified corner area of ​​the panoramic stitched image according to a preset corner anchoring position (e.g., the lower left corner or the lower right corner).

[0104] The final output map has both coordinate positioning and spatial measurement functions, which facilitates data archiving, sharing and cross-institutional area identification.

[0105] In combination with the above Figures 1 to 11 Method and process Figure 12 From the perspective of functional modular design, the logical architecture of the system of the present invention during actual deployment is further demonstrated.

[0106] Figure 12 A schematic block diagram of a microscopic image processing apparatus according to an embodiment of the present invention is shown.

[0107] like Figure 12 As shown, the microscopic image processing apparatus 1200 of this embodiment includes an image acquisition module 1210, a preprocessing module 1220, a mesh determination module 1230, a coordinate generation module 1240, an initial generation module 1250, and a target generation module 1260.

[0108] Image acquisition module 1210 is used to acquire sub-image sequences of the sample and obtain a low-resolution image of the entire sample. The sub-image sequences are optical microscopic sub-image sequences and / or backscattered electron sub-image sequences. In one embodiment, image acquisition module 1210 can be used to perform the operation S110 described above, which will not be repeated here.

[0109] The preprocessing module 1220 is used to perform differential preprocessing according to the imaging type of the sub-image sequence to obtain a preprocessed sub-image sequence. In one embodiment, the preprocessing module 1220 can be used to perform the operation S120 described above, which will not be repeated here.

[0110] The grid determination module 1230 is used to determine the grid size for panoramic stitching based on the aspect ratio of the overall low-resolution image of the sample and the total number of sub-images in the sub-image sequence. In one embodiment, the grid determination module 1230 can be used to perform the operation S130 described above, which will not be repeated here.

[0111] The coordinate generation module 1240 is used to call an external stitching tool to operate in a mode that generates only coordinate files without outputting a panoramic image. If the external stitching tool operates normally, the generated coordinate file is obtained. If the external stitching tool terminates abnormally due to exceeding the total number of output pixels, the previously generated historical coordinate files are read in response to the captured abnormal information. In one embodiment, the coordinate generation module 1240 can be used to perform the operation S140 described above, which will not be repeated here.

[0112] The initial generation module 1250 is used to create a blank canvas according to the grid size, and map the preprocessed sub-images to the corresponding positions on the blank canvas according to the coordinate information in the coordinate file, thereby generating a panoramic stitched image. In one embodiment, the initial generation module 1250 can be used to perform the operation S150 described above, which will not be repeated here.

[0113] The target generation module 1260 is used to perform differentiated post-processing on the panoramic stitched image according to the imaging type to generate a standardized result image. In one embodiment, the data acquisition module 510 and the target generation module 1260 can be used to perform the operation S160 described above, which will not be repeated here.

[0114] According to embodiments of the present invention, any plurality of modules among the image acquisition module 1210, preprocessing module 1220, mesh determination module 1230, coordinate generation module 1240, initial generation module 1250, and target generation module 1260 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules can be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the image acquisition module 1210, preprocessing module 1220, mesh determination module 1230, coordinate generation module 1240, initial generation module 1250, and target generation module 1260 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the image acquisition module 1210, preprocessing module 1220, mesh determination module 1230, coordinate generation module 1240, initial generation module 1250, and target generation module 1260 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0115] Figure 13 A block diagram of an electronic device for a microscopic image processing method according to an embodiment of the present invention is shown schematically.

[0116] like Figure 13 As shown, an electronic device 1300 according to an embodiment of the present invention includes a processor 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage portion 1308 into a random access memory (RAM) 1303. The processor 1301 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1301 may also include onboard memory for caching purposes. The processor 1301 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0117] RAM 1303 stores various programs and data required for the operation of electronic device 1300. Processor 1301, ROM 1302, and RAM 1303 are interconnected via bus 1304. Processor 1301 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 1302 and / or RAM 1303. It should be noted that programs may also be stored in one or more memories other than ROM 1302 and RAM 1303. Processor 1301 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0118] According to an embodiment of the present invention, the electronic device 1300 may further include an input / output (I / O) interface 1305, which is also connected to the bus 1304. The electronic device 1300 may also include one or more of the following components connected to the input / output (I / O) interface 1305: an input section 1306 including a keyboard, mouse, etc.; an output section 1307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the input / output (I / O) interface 1305 as needed. A removable medium 1311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1310 as needed so that computer programs read from it can be installed into the storage section 1308 as needed.

[0119] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0120] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 1302 and / or RAM 1303 and / or one or more memories other than ROM 1302 and RAM 1303 described above.

[0121] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

Claims

1. A method for processing microscopic images, characterized in that, include: Acquire sub-image sequences of the sample and obtain a low-resolution image of the entire sample. The sub-image sequences are optical microscopy sub-image sequences and / or backscattered electron sub-image sequences. Differential preprocessing is performed based on the imaging type of the sub-image sequence to obtain the preprocessed sub-image sequence; The grid size for panoramic stitching is determined based on the aspect ratio of the overall low-resolution image of the sample and the total number of sub-images in the sub-image sequence. The external stitching tool is invoked to run in a working mode that generates only coordinate files without outputting a panoramic image; if the external stitching tool runs normally, the generated coordinate files are obtained; if the external stitching tool terminates abnormally due to exceeding the total number of output pixels, the previously generated historical coordinate files are read in response to the captured abnormal information. A blank canvas is created according to the grid size, and the preprocessed sub-images are mapped to the corresponding positions on the blank canvas according to the coordinate information in the coordinate file to generate a panoramic stitched image. Differentiated post-processing is performed on the panoramic stitched image according to the imaging type to generate a standardized result image.

2. The method according to claim 1, characterized in that, The differential preprocessing based on the imaging type of the sub-image sequence includes: When the sub-image sequence is an optical microscopy sub-image sequence, a flat-field image is constructed from the optical microscopy sub-image sequence and flat-field correction is performed to obtain a flat-field corrected optical sub-image sequence. When the sub-image sequence is a backscattered electron sub-image sequence, the backscattered electron sub-image sequence is directly used as the preprocessed sub-image sequence.

3. The method according to claim 2, characterized in that, The step of constructing a flat-field image from the optical microscopy image sequence and performing flat-field correction includes: Select blank sub-images that do not contain the sample coverage area from the optical microscopy sub-image sequence; Calculate the pixel-level mean of the gray values ​​of multiple blank sub-images to generate a flat field image; The original optical microscopy images are subtracted from the dark current background image and then divided by the flat-field image that has been normalized by grayscale mean to obtain the flat-field corrected optical sub-images.

4. The method according to claim 3, characterized in that, When the sample is in powder form and the number of blank sub-images is less than a preset threshold, the grayscale mean is calculated after performing low-frequency filtering or morphological opening processing on all optical microscopy sub-images to construct a flat field image.

5. The method according to claim 1, characterized in that, The determination of the grid size for panoramic stitching specifically includes: The estimated total width in pixels is obtained by multiplying the number of grid columns attempted by the fixed width of a single sub-image. The estimated total height in pixels is obtained by multiplying the number of grid rows attempted by the fixed height in pixels of a single sub-image. The ratio of the estimated total width pixels to the estimated total height pixels is used as the theoretical aspect ratio of the stitched image frame; Calculate the theoretical aspect ratio corresponding to each combination of the number of columns and rows of the grid, and select the combination with the smallest difference between the theoretical aspect ratio and the aspect ratio of the overall low-resolution image of the sample as the grid size.

6. The method according to claim 1, characterized in that, The step of mapping the preprocessed sub-images to their corresponding positions on the blank canvas to generate a panoramic stitched image includes: The canvas width is determined by the difference between the maximum and minimum horizontal values ​​of each coordinate value in the coordinate file, and the canvas height is determined by the difference between the maximum and minimum vertical values ​​of each coordinate value in the coordinate file. A blank canvas is then created based on the canvas width and the canvas height. Read each preprocessed sub-image and its corresponding coordinate information sequentially, and place each sub-image in the corresponding coordinate area of ​​the blank canvas according to the position parameters recorded in the coordinate information; For any empty areas in the grid size that are not covered by the sub-image, fill them with the background color.

7. The method according to claim 1, characterized in that, The step of performing differential post-processing on the panoramic stitched image according to the imaging type includes: When the imaging type is an optical microscopic image, background color matching and circular base synthesis are performed on the panoramic stitched image; When the imaging type is a backscattered electron image, mask extraction, circular background synthesis with coordinate grid, and scale bar addition are performed on the panoramic stitched image.

8. The method according to claim 7, characterized in that, The step of performing background color matching and circular base synthesis on the panoramic stitched image includes: Calculate the average pixel color value of the edge region of the panoramic stitched image; A global background color that coordinates with the edge color of the panoramic stitched image is generated based on the average pixel color value; Generate a base image with a circular field of view, the size of which matches the size of the panoramic stitched image; The panoramic stitched image is superimposed on the circular field of view of the base image, so that the synthesized image presents a circular observation field of view consistent with the actual shape of the sample base.

9. The method according to claim 7, characterized in that, The circular background composition with coordinate grid includes: Generate a coordinate grid background with a preset number of columns and a preset number of rows. The horizontal axis of the coordinate grid background is marked with a numerical sequence number, and the vertical axis is marked with a letter sequence number. The coordinate grid background board is cut into a circular background board; Based on the grayscale difference between the sample area and the background area in the panoramic stitched image, a binary mask is constructed to extract the sample particle area. The sample particle area within the coverage area of ​​the binary mask is superimposed on the grid layer of the circular background plate, so that the grid lines remain visible in the gap area of ​​the sample particles and the edge area of ​​the circular background plate.

10. A microscopic image processing apparatus, characterized in that, include: An image acquisition module is used to acquire sub-image sequences of a sample and obtain a low-resolution image of the entire sample. The sub-image sequences are optical microscopic sub-image sequences and / or backscattered electron sub-image sequences. The preprocessing module is used to perform differential preprocessing based on the imaging type of the sub-image sequence to obtain the preprocessed sub-image sequence; The grid determination module is used to determine the grid size for panoramic stitching based on the aspect ratio of the overall low-resolution image of the sample and the total number of sub-images in the sub-image sequence. The coordinate generation module is used to call an external stitching tool to run in a working mode that generates only coordinate files without outputting a panoramic image; if the external stitching tool runs normally, the generated coordinate file is obtained; if the external stitching tool terminates abnormally due to exceeding the total number of output pixels, the previously generated historical coordinate files are read in response to the captured abnormal information. The initial generation module is used to create a blank canvas according to the grid size, and to map the preprocessed sub-images to the corresponding positions on the blank canvas according to the coordinate information in the coordinate file, thereby generating a panoramic stitched image. The target generation module is used to perform differentiated post-processing on the panoramic stitched image according to the imaging type to generate a standardized result image.

Images