Microscope equipment, slice sample scanning method, device, equipment and medium

Abstract

The embodiment of the invention discloses microscope equipment, a slice sample scanning method, a slice sample scanning device, slice sample scanning equipment and a medium. When the microscope equipment is used for scanning a slice sample, the polarizer is electrically controlled to perform polarization at a preset polarization angle through the stepping motor, so that full-vision-field scanning of the slice sample is realized; and an electric Z-axis focusing system and a high-sensitivity industrial camera are adopted to scan the slice sample, so that the acquisition precision of the slice sample image is improved. Intelligent merging and splicing of multi-angle and full-vision-field images are carried out based on an improved scale invariant feature transformation algorithm, so that an image acquisition function of multi-angle polarization and full-vision-field splicing is realized under the condition that an objective table is not moved; meanwhile, a random sampling consensus algorithm is adopted to carry out multiple optimization iterations when feature points are matched, mismatching points are removed, purified and improved, the splicing time is shortened, the image splicing precision is improved, and data saved after cutting can meet the requirements of a video segmentation data set.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical fields of thin-section identification and microscope equipment, and in particular to a microscope equipment and a thin-section sample scanning method, device, equipment and medium. Background Art

[0002] Thin section identification in the field of oil exploration and development mainly relies on visual observation by geologists using polarizing microscopes. Currently, there are two main problems when using polarizing microscopes for thin section identification:

[0003] On the one hand, the stage of a traditional microscope is usually rotating. When observing samples, experts need to rotate the microscope stage to observe the sample information at multiple angles for comprehensive analysis and then give the results. Since the horizontal direction of the stage cannot achieve precise movement of each field of view, the solution of rotating the sample is not conducive to the subsequent analysis and repositioning of the sample. In the case of observing full-field images, due to the limitations of the expert's field of view, it is necessary to piece together the images when collecting them, so the stage that can stitch the images cannot be rotated. In this case, traditional microscopes cannot meet the corresponding equipment requirements, and the scanned images are not conducive to the data requirements of the deep learning algorithm modeling process, making it difficult to build an intelligent model. Therefore, the limitations of traditional microscopes have brought huge challenges to the image sample collection process, and they are in urgent need of certain upgrades and modifications.

[0004] On the other hand, when observing and capturing large-pixel images, traditional microscopes require stitching together multiple captured microscopic images due to pixel-level constraints. However, due to limitations in software algorithms, these microscopes often produce poor image stitching results. Furthermore, the data stored using traditional microscope acquisition methods is unsuitable for use in video segmentation networks. Summary of the Invention

[0005] The embodiments of the present invention provide a microscope device and a thin-sheet sample scanning method, device, electronic device and storage medium, so as to realize the acquisition and splicing of thin-sheet samples under multi-angle polarization of full-field images using an upgraded microscope device, and meet the data requirements of thin-sheet samples in the field of video segmentation.

[0006] In a first aspect, an embodiment of the present invention provides a microscope device, comprising: an industrial camera, a Z-axis focusing system, a polarizer and an analyzer, a sample holder, and a hardware controller and a control handle; wherein,

[0007] The industrial camera is configured to use a fast and highly sensitive camera to acquire images of thin slice samples observed in a microscope device;

[0008] The Z-axis focusing system is configured to use electric Z-axis auxiliary focusing;

[0009] The polarizer and the analyzer are configured such that a stepper motor is used to electrically control the polarization of the polarizer at a preset polarization angle, and the polarizer and the analyzer are synchronously controlled;

[0010] The hardware controller and the control handle are configured to communicate with the communication device through the COM port and determine the focal plane of the thin sample. The hardware controller integrates an integrated controller and a highly integrated five-axis stepper motor system including X, Y, Z, polarization and analysis.

[0011] In a second aspect, an embodiment of the present invention provides a microscope device and a thin-section sample scanning method, including:

[0012] Using a microscope to obtain original images and real-time images of the thin slice sample at various polarization angles, and extracting feature points from the original images and real-time images;

[0013] Describing the feature points of the original image and the real-time image after feature point extraction based on the neighborhood point information of the feature points of the original image and the real-time image;

[0014] Match feature points of the original image and the real-time image based on the feature point description information, and determine the optimal solution matrix for feature point matching;

[0015] transforming the real-time image according to the optimal solution matrix, and superimposing the transformed real-time image with the original image to obtain a target thin slice sample image;

[0016] The target thin slice sample images at each polarization angle are determined and stitched together to obtain a panoramic image of the thin slice sample.

[0017] In a third aspect, an embodiment of the present invention further provides a microscope device and a thin-section sample scanning device, comprising:

[0018] A feature point extraction module is used to obtain original images and real-time images of the thin-film sample at various polarization angles using a microscope device, and to extract feature points from the original images and real-time images;

[0019] A feature point description module is used to describe the feature points of the original image and the real-time image after the feature points are extracted based on the neighborhood point information of the feature points of the original image and the real-time image;

[0020] The feature point matching module is used to match the feature points of the original image and the real-time image based on the feature point description information, and determine the optimal solution matrix for feature point matching;

[0021] An image fusion module is used to transform the real-time image according to the optimal solution matrix, and superimpose the transformed real-time image with the original image to obtain a target thin slice sample image;

[0022] The thin-sheet sample scanning module is used to determine the target thin-sheet sample images at various deflection angles and stitch them together to obtain a panoramic image of the thin-sheet sample.

[0023] In a fourth aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0024] one or more processors;

[0025] a storage device for storing one or more programs;

[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the microscope device and thin-section sample scanning method according to any embodiment of the present invention.

[0027] In a fifth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the microscope device and thin-section sample scanning method described in any embodiment of the present invention.

[0028] Embodiments of the present invention provide a microscope device and a thin-film sample scanning method, device, electronic device, and storage medium. When scanning a thin-film sample, the microscope device electrically controls the polarizer polarization at a preset polarization angle using a stepper motor, thereby achieving full-field scanning of the thin-film sample. An electric Z-axis focusing system and a high-sensitivity industrial camera are used to scan the thin-film sample, thereby improving the accuracy of image acquisition of the thin-film sample. An improved scale-invariant feature transformation algorithm is used to intelligently merge and splice multi-angle, full-field images, thereby achieving multi-angle polarization and full-field stitching image acquisition capabilities without moving the stage. Furthermore, a random sampling consensus algorithm is used to perform multiple optimization iterations during feature point matching, eliminating and purifying mismatched points, thereby shortening the stitching time and improving the image stitching accuracy. Furthermore, the cropped and saved data can meet the requirements of a video segmentation dataset. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Other features, objects, and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered as limiting the present invention. Like reference characters are used throughout the drawings to denote like parts. In the drawings:

[0030] Figure 1 is a schematic structural diagram of a microscope device provided in an embodiment of the present invention;

[0031] Figure 2 is a schematic structural diagram of a traditional microscope provided in an embodiment of the present invention;

[0032] Figure 3 1 is a schematic structural diagram of an upgraded integrated microscope device provided in an embodiment of the present invention;

[0033] Figure 4 is a schematic flow chart of a thin-sheet sample scanning method provided in an embodiment of the present invention;

[0034] Figure 5 is a schematic structural diagram of a feature point neighborhood point provided in an embodiment of the present invention;

[0035] Figure 6 This is a schematic diagram of a panoramic image obtained by stitching single images of a thin-section sample using an upgraded microscope device provided in an embodiment of the present invention;

[0036] Figure 7 1 is a schematic diagram of a panoramic image obtained by stitching multiple images of a thin-section sample using a microscope device, provided in an embodiment of the present invention;

[0037] Figure 8 1 is a schematic diagram of an image when stereo segmentation of a panoramic image is performed, provided in an embodiment of the present invention;

[0038] Figure 9 3D grid image diagram of a single viewing area after segmenting a panoramic image provided in an embodiment of the present invention;

[0039] Figure 10 is a structural schematic diagram of a thin-sheet sample scanning device provided in an embodiment of the present invention;

[0040] Figure 11 It is a structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0042] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as sequential processes, many of the operations (or steps) therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0043] Among them, the acquisition, storage, use and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0044] Figure 1 1 is a schematic structural diagram of a microscope device provided in an embodiment of the present invention. This embodiment is applicable to the case where a thin-sheet sample is scanned using an upgraded microscope device. The microscope device 100 includes: an industrial camera 110, a Z-axis focusing system 120, a polarizer and analyzer 130, a sample holder 140, and a hardware controller and a control handle 150; wherein,

[0045] The industrial camera 110 is configured to use a fast and highly sensitive camera to acquire an image of a thin slice sample observed in a microscope device;

[0046] The Z-axis focusing system 120 is configured to use electric Z-axis auxiliary focusing;

[0047] The polarizer and analyzer 130 are configured to electrically control the polarizer polarization at a preset polarization angle using a stepper motor, and to synchronously control the polarizer and analyzer;

[0048] The hardware controller and the control handle 150 are configured to communicate with the communication device through the COM port and determine the focal plane of the thin sample. The hardware controller integrates an integrated controller and a highly integrated five-axis stepper motor system including X, Y, Z, polarization and analysis.

[0049] Thin-section identification in the field of oil exploration and development primarily relies on visual observation by geologists using polarizing microscopes. Thin-section identification primarily focuses on mineral composition, and mineral observation requires a comprehensive consideration of both single polarization and multiple angles of orthogonal polarization. Minerals are crystalline and exhibit extinction properties. When exposed to orthogonal polarization at different angles, the brightness of the minerals changes. This property allows the identification of the minerals present in the sample. When observing minerals under a microscope, transmitted light is passed through the sample from below. Below the sample is a fixed polarizing filter, called a polarizer; above the sample is another polarizing filter, orthogonal to the polarizer, called an analyzer.

[0050] See also Figure 2 When using a traditional microscope to identify thin-section samples, the microscope stage is typically rotated to observe sample information from multiple angles. However, because the microscope stage rotates horizontally, field-by-field identification is not possible. Therefore, embodiments of the present invention provide a microscope device that upgrades a traditional microscope to enable field-by-field identification of thin-section samples and to position thin-section samples.

[0051] The present invention provides a microscope device capable of performing multi-angle polarization and full-field stitching on thin-section samples without moving the stage. Figure 3 The microscope is equipped with a fast, high-sensitivity industrial camera capable of capturing images at a maximum resolution of 2448×2048 pixels and an exposure time of 0.13ms-15s. At full resolution, it can achieve rapid imaging at a rate of 35 frames per second, enabling fast and accurate image capture of thin-section samples. The microscope's Z-axis focusing system utilizes a motorized Z-axis, improving Z-axis accuracy and providing auxiliary focusing, ensuring clear image quality on the display and facilitating accurate identification of thin-section samples. The polarizer utilizes a motorized mechanism, with polarization angle control electronically controlled. The angle can be set to switch every 15°. The polarizer and analyzer are controlled synchronously, ensuring orthogonality between the analyzer and the polarizer. Furthermore, a stepper motor controls the polarization angle, providing increased accuracy and speed, enabling precise movement of the field of view and improved stitching of captured thin-section sample images. The microscope incorporates an integrated quartz matte filter to ensure excellent transmitted single-polarized light imaging without any image anomalies. A custom three-hole thin-section sample holder is designed for the microscope, measuring 76mm x 26mm, and matching the stage frame, which measures 116mm x 160mm. The microscope also features an integrated control handle that communicates with the display device via a COM port to ensure proper operation, enabling rapid sample movement and finding the focal plane. The integrated hardware controller features a highly integrated five-axis stepper motor system for X, Y, Z, polarization, and analysis.

[0052] An embodiment of the present invention provides a microscope device, comprising: an industrial camera, a Z-axis focusing system, a polarizer and an analyzer, a sample holder, and a hardware controller and a control handle; wherein the industrial camera is configured to use a fast and highly sensitive camera to acquire an image of a thin sample observed in the microscope device; the Z-axis focusing system is configured to use an electric Z-axis to assist in focusing; the polarizer and analyzer are configured to use a stepper motor to electrically control polarization of the polarizer at a preset polarization angle, and to synchronously control the polarizer and analyzer; the hardware controller and the control handle are configured to communicate with a communication device via a COM port and determine the focal plane of the thin sample; the hardware controller is integrated into an all-in-one controller, and highly integrated into a five-axis stepper motor system for X, Y, Z, polarization, and analysis. When scanning a thin sample using the microscope device of the embodiment of the present invention, the polarizer is electrically controlled to polarize at a preset polarization angle through a stepper motor, thereby achieving full-field scanning of the thin sample. The thin sample is scanned using an electric Z-axis focusing system and a high-sensitivity industrial camera, thereby improving the accuracy of acquiring the image of the thin sample. The modified microscope device is different from the rotating stage of a traditional polarizing microscope, and the entire optical path meets the Köhler illumination system, thereby providing a powerful technical means for collecting relevant microscopic images.

[0053] Figure 4 This is a flow chart of a thin-section sample scanning method provided in an embodiment of the present invention. This embodiment is applicable to scanning thin-section samples using upgraded microscope equipment. The method of this embodiment can be performed by a thin-section sample scanning device, which can be implemented using hardware and / or software. The device can be deployed in a thin-section sample scanning server. The method specifically includes the following steps:

[0054] S410 , using a microscope to obtain original images and real-time images of the thin slice sample at various polarization angles, and extracting feature points from the original images and real-time images.

[0055] Among them, the embodiment of the present invention uses an improved scale-invariant feature transform algorithm (SIFT) to intelligently stitch the thin-film sample images collected under multiple channels, that is, every two images of the same channel are taken out for correlation stitching, and finally all the images collected under the channel are merged and stitched into a panoramic image of the full field of view. Repeat this step until all the image samples of all channels are merged and stitched. Among them, the thin-film sample images under the multiple channels can refer to thin-film sample images obtained at different polarization angles. Specifically, feature points are first extracted from the input image, and the feature description of the feature points is completed. Then, the feature points are matched in combination with the RANSAC algorithm. Finally, the sample images are fused to obtain the final panoramic image.

[0056] As an optional but non-limiting implementation, the method of using a microscope to obtain the original image and real-time image of the thin film sample at each polarization angle and extracting feature points from the original image and real-time image includes but is not limited to steps A1-A3:

[0057] Step A1: Using a microscope to obtain an original image and a real-time image of a thin slice sample at various deflection angles, determining the Gaussian scale space of the original image and the real-time image to obtain a Gaussian blurred image; wherein the scale space of the real-time image is obtained by convolving the scale space of the original image with a Gaussian function.

[0058] Step A2: performing Gaussian smoothing and downsampling processing on the Gaussian blurred image to obtain a processed thin slice sample image; wherein the Gaussian blurred image includes a Gaussian blurred original image and a Gaussian blurred real-time image.

[0059] Step A3: Use the Gaussian difference function to extract feature points from the processed slice sample image.

[0060] In this case, a stepper motor is used to control the polarization of the microscope device at a preset polarization angle to obtain thin-film sample images at different polarization angles. Two thin-film sample images acquired at the same polarization angle are obtained, defined as the original image (i.e., Figure a) and the real-time image (i.e., Figure b). The scale space of the original image a is denoted as I(x, y), and the scale space of the image b is denoted as L(x, y, σ), where L(x, y, σ) is the convolution of a scale-varying Gaussian function G(x, y, σ) with I(x, y), and the expression is as follows:

[0061] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0062]

[0063] Here, * denotes the convolution calculation, and (x, y) represents the pixel position of the input thin sample image. σ is the scale space factor, and smaller σ values ​​indicate less smoothing of the sample image, corresponding to smaller scales. Large scales correspond to the contour features of the sample image, while small scales correspond to the details of the sample image. The scale space of the original image is transformed to construct the Gaussian scale space of the input image, generating Gaussian blurred images of different scales. After a series of scale space transformations and two-fold downsampling, the Gaussian pyramid is ultimately obtained.

[0064] The thin slice sample image is simultaneously subjected to Gaussian smoothing (Gaussian blurring) and downsampling. After dimensionality reduction sampling, a series of thin slice images with decreasing sizes are obtained, and finally a multi-scale spatial sequence of sample images is obtained. The Gaussian difference function is used to extract feature points from the processed thin slice sample image. For example, the DOG (Gaussian difference) function is used to compare adjacent pixels to find the stable feature key points of the input sample image. DOG only requires the subtraction of adjacent Gaussian smoothed images. The Gaussian difference image can be used to show the changes in pixel values ​​on the sample image. The DOG expression is as follows:

[0065] D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)

[0066] =L(x,y,kσ)-L(x,y,σ)

[0067] where kσ and σ are the smoothing scales of the two consecutive images, and the resulting difference image is in the Difference of Gaussian pyramid.

[0068] The feature points of the thin sample image are composed of local extreme points in the DOG space. In order to find the extreme points of the DOG function, each pixel is compared with all its neighboring points. For example, see Figure 5 , the middle detection point ( Figure 5 The point represented by “×” in the figure is compared with its 8 adjacent points at the same scale and 9×2 points corresponding to the upper and lower adjacent scales, a total of 26 pixels, to ensure that the extreme points can be detected in both scale space and two-dimensional image space.

[0069] S420 , describing the feature points of the original image and the real-time image after feature point extraction based on the neighborhood point information of the feature points of the original image and the real-time image.

[0070] After extracting feature points from the original image and the real-time image, a descriptor is used to describe each feature point in the original image and the real-time image. The description information includes but is not limited to the feature point coordinates, scale, and direction.

[0071] As an optional but non-limiting implementation, the feature point description of the original image and the real-time image after feature point extraction based on the neighborhood point information of the feature points of the original image and the real-time image includes but is not limited to steps B1-B2:

[0072] Step B1: Determine neighborhood point information of feature points of the original image and the real-time image; the neighborhood point information includes the coordinates, scale, and direction of the neighborhood point.

[0073] Step B2: Describing the coordinates, scale and direction of each feature point in the original image and the real-time image after feature point extraction based on the neighborhood point information.

[0074] The descriptor describes the coordinates, scale, and orientation of each feature point in the original and live images. To mitigate the effects of lighting and viewing angle on the thin-film sample image, the feature descriptor also includes, but is not limited to, information about the neighborhood of the image feature point.

[0075] S430 , performing feature point matching on the original image and the real-time image according to the feature point description information, and determining an optimal solution matrix for feature point matching.

[0076] During the feature point matching step, if the traditional SIFT algorithm is used for stitching, a large number of useless matching points will appear when determining the registration points. Therefore, the present invention incorporates the Random Sampling Consensus (RANSAC) algorithm into this step. This algorithm performs multiple optimization iterations on the SIFT algorithm during feature point matching, eliminating and purifying incorrect matching points to achieve accurate matching of the images to be registered, thereby eliminating incorrectly spliced ​​feature points and shortening the stitching time.

[0077] As an optional but non-limiting implementation, the feature point matching of the original image and the real-time image based on the feature point description information and determining the optimal solution matrix for the feature point matching include but are not limited to steps C1-C3:

[0078] Step C1: Match the feature points of the original image and the real-time image according to the feature point description information and perform iterative optimization to eliminate the incorrectly spliced ​​feature points and obtain the feature data after feature point matching.

[0079] Step C2: obtaining at least two sets of matching feature data pairs from the feature data after feature point matching, and performing matching data calculation on the at least two sets of matching feature data pairs to determine a homography matrix.

[0080] Step C3: Calculating the feature data after all feature points are matched according to the homography matrix, determining the target feature points and the cost function that conform to the homography matrix, and determining the optimal solution matrix according to the cost function.

[0081] Among them, a random sampling consensus algorithm is used to match the feature points of the original image and the real-time image and iteratively optimize them to eliminate the mismatched points; the calculation formula of the random sampling consensus algorithm is expressed as:

[0082]

[0083] Where s is the scale parameter, (x′, y′) represents the corner position of the real-time image, and (x, y) represents the corner position of the original image.

[0084] The RANSAC algorithm is used to randomly extract several matching pairs from the matched data set, and at the same time, the extracted matching pairs are required to be non-collinear. Then, a homography matrix is ​​obtained by calculating the matching data. Next, the obtained model is used to test all data, and all data points that meet the model characteristics and their cost functions are calculated. If the model is optimal, the cost function value is minimized. The embodiment of the present invention uses the cost function to represent the optimal solution matrix. If the model is the optimal solution, the corresponding cost function is minimized, that is:

[0085]

[0086] Among them, (x i ,y i ) can refer to the feature points of the real-time image, (x′ i , y′ i ) can refer to the feature points in the original image that match the real-time image, n refers to the maximum number of feature points i, J refers to the cost function, h ij Refers to the value of the matrix H.

[0087] S440 , transforming the real-time image according to the optimal solution matrix, and superimposing the transformed real-time image with the original image to obtain a target thin slice sample image.

[0088] Among them, after the feature points of the original image and the real-time image are extracted, described and matched, the real-time image is transformed into a new image through the transformation matrix H, and then image a is added to the new image to complete the fusion of the original image and the real-time image, and obtain the target thin film sample image at the current deflection angle.

[0089] S450: Determine the target thin-sheet sample images at each polarization angle and stitch them together to obtain a panoramic image of the thin-sheet sample.

[0090] Here, steps S410 to S440 in the embodiment of the present invention are repeated to obtain target thin slice sample images at various polarization angles, and the target thin slice sample images at various polarization angles are spliced ​​to obtain a panoramic image of the thin slice sample.

[0091] As an optional but non-limiting implementation, after obtaining the panoramic image of the thin-section sample, the method further includes but is not limited to steps D1-D2:

[0092] Step D1: performing stereoscopic custom segmentation on the panoramic image of the thin slice sample to obtain stereoscopic grid data under a single viewing area.

[0093] Step D2: saving the three-dimensional grid data under the single view and naming the three-dimensional grid data according to the position coordinates of the three-dimensional grid data under the single view in the panoramic image of the thin slice sample.

[0094] The stitched panoramic image is segmented into customizable 3D images, facilitating intelligent identification and analysis. Finally, the resulting cropped 3D grid data (in a proprietary ".CZI" format) is saved for the single-view field. The saved thin-section sample data remains uncompressed, maintaining resolution. The segmented thin-section images are named according to their coordinates within the panoramic image, enabling rapid location of specific minerals, thus satisfying the input data requirements of the video segmentation network.

[0095] Optionally, the embodiment of the present invention takes the upgrade and transformation of the Zeiss microscope AxioScope5 as an example. The camera model of the upgraded microscope equipment is a customized fast and high-sensitivity camera ProCam405color, which captures thin-section sample images with an image resolution of 2448*2048. Figure 6 It is a panoramic image obtained by stitching together single images of thin-section samples using upgraded microscope equipment. Figure 7 It is a panoramic image obtained by stitching together multiple images of thin-section samples using a microscope. Figure 8 is the image when stereo segmentation is performed on the panoramic image. Figure 9 It is a stereo grid image in a single field of view after the panoramic image is segmented. It can be seen that the microscope equipment upgrading and transformation technology proposed in the present invention can meet the input data requirements of the video segmentation network.

[0096] An embodiment of the present invention provides a thin-sheet sample scanning method. The method uses a microscope to obtain original images and real-time images of the thin-sheet sample at various polarization angles, extracts feature points from the original and real-time images, describes feature points of the extracted original and real-time images based on neighborhood information of the feature points in the original and real-time images, matches feature points of the original and real-time images based on the feature point description information, and determines an optimal solution matrix for feature point matching. The real-time image is transformed based on the optimal solution matrix, and the transformed real-time image is superimposed with the original image to obtain a target thin-sheet sample image. The target thin-sheet sample images at various polarization angles are determined and stitched together to obtain a panoramic image of the thin-sheet sample. The technical solution of the embodiment of the present invention utilizes an improved SIFT algorithm to intelligently merge and stitch multi-angle, full-view images, thereby achieving multi-angle polarization, full-view stitching image acquisition without a stationary stage. Furthermore, the resolution and accuracy of image acquisition are improved, as is the accuracy of image stitching. Furthermore, the cropped and saved data can meet the requirements of a video segmentation dataset.

[0097] Figure 10This is a schematic diagram of the structure of a thin-section sample scanning device provided in an embodiment of the present invention. The technical solution of this embodiment can be applied to the case where an upgraded microscope device is used to scan thin-section samples. The device can be implemented by software and / or hardware and is generally integrated into any electronic device with network communication function, including but not limited to: servers, computers, personal digital assistants and other devices. Figure 10 As shown, the sheet sample scanning device provided in this embodiment may include: a feature point extraction module 1010, a feature point description module 1020, a feature point matching module 1030, an image fusion module 1040 and a sheet sample scanning module 1050; wherein,

[0098] A feature point extraction module 1010 is configured to obtain an original image and a real-time image of the thin-film sample at each polarization angle using a microscope, and extract feature points from the original image and the real-time image;

[0099] A feature point description module 1020 is used to describe the feature points of the original image and the real-time image after the feature points are extracted based on the neighborhood point information of the feature points of the original image and the real-time image;

[0100] The feature point matching module 1030 is used to perform feature point matching on the original image and the real-time image based on the feature point description information, and determine the optimal solution matrix for feature point matching;

[0101] An image fusion module 1040 is configured to transform the real-time image according to the optimal solution matrix, and superimpose the transformed real-time image with the original image to obtain a target thin slice sample image;

[0102] The thin slice sample scanning module 1050 is used to determine the target thin slice sample images at various polarization angles and stitch them together to obtain a panoramic image of the thin slice sample.

[0103] Based on the above embodiment, optionally, the feature point extraction module is specifically configured to:

[0104] Using a microscope to obtain an original image and a real-time image of the thin-film sample at various deflection angles, determining the Gaussian scale space of the original image and the real-time image to obtain a Gaussian blurred image; wherein the scale space of the real-time image is obtained by convolving the scale space of the original image with a Gaussian function;

[0105] Performing Gaussian smoothing and downsampling processing on the Gaussian blurred image to obtain a processed thin slice sample image; wherein the Gaussian blurred image includes a Gaussian blurred original image and a Gaussian blurred real-time image;

[0106] The Gaussian difference function is used to extract feature points from the processed thin slice sample image.

[0107] Based on the above embodiment, optionally, the feature point description module is specifically configured to:

[0108] Determine neighborhood point information of feature points of the original image and the real-time image; the neighborhood point information includes coordinates, scale and direction of the neighborhood point;

[0109] The coordinates, scale and direction of each feature point in the original image and the real-time image after feature point extraction are described based on the neighborhood point information.

[0110] Based on the above embodiment, optionally, the feature point matching module is specifically configured to:

[0111] According to the feature point description information, the original image and the real-time image are matched and iteratively optimized to eliminate the mis-joined feature points and obtain the feature data after feature point matching;

[0112] Acquire at least two sets of matching feature data pairs from the feature data after feature point matching, and perform matching data calculation on the at least two sets of matching feature data pairs to determine a homography matrix;

[0113] The feature data after all feature points are matched are calculated according to the homography matrix, the target feature points and the cost function that meet the homography matrix are determined, and the optimal solution matrix is ​​determined according to the cost function.

[0114] Based on the above embodiment, optionally, the feature point matching module is further configured to:

[0115] A random sampling consensus algorithm is used to match feature points of the original image and the real-time image and perform iterative optimization to eliminate mismatched points. The calculation formula of the random sampling consensus algorithm is expressed as:

[0116]

[0117] Where s is the scale parameter, (x′, y′) represents the corner position of the real-time image, and (x, y) represents the corner position of the original image.

[0118] Based on the above embodiment, optionally, the device further includes an image segmentation module for

[0119] Performing stereoscopic custom segmentation on the panoramic image of the thin slice sample to obtain stereoscopic grid data under a single viewing area;

[0120] The three-dimensional grid data under the single field of view is saved and named according to the position coordinates of the three-dimensional grid data under the single field of view in the panoramic image of the thin slice sample.

[0121] The sheet sample scanning device provided in the embodiment of the present invention can execute the sheet sample scanning method provided in any of the above embodiments of the present invention, and has the corresponding functions and beneficial effects of executing the sheet sample scanning method. For detailed processes, please refer to the relevant operations of the sheet sample scanning method in the above embodiments.

[0122] Figure 11 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0123] like Figure 11 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0124] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0125] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as the microscope device and thin-section sample scanning method.

[0126] In some embodiments, the microscope apparatus and thin-section specimen scanning method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the microscope apparatus and thin-section specimen scanning method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the microscope apparatus and thin-section specimen scanning method in any other suitable manner (e.g., via firmware).

[0127] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0128] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0129] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0130] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0131] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0132] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0133] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0134] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A microscope device, characterized in that The microscope equipment includes: an industrial camera, a Z-axis focusing system, a polarizer and an analyzer, a sample holder, a hardware controller and a control handle; wherein, The industrial camera is configured to use a fast and highly sensitive camera to acquire images of thin slice samples observed in a microscope device; The Z-axis focusing system is configured to use electric Z-axis auxiliary focusing; The polarizer and the analyzer are configured such that a stepper motor is used to electrically control the polarization of the polarizer at a preset polarization angle, and the polarizer and the analyzer are synchronously controlled; The hardware controller and the control handle are configured to communicate with the communication device through the COM port and determine the focal plane of the thin sample. The hardware controller integrates an integrated controller and a highly integrated five-axis stepper motor system including X, Y, Z, polarization and analysis.

2. A thin sample scanning method, characterized in that: The method comprises: Using a microscope to obtain original images and real-time images of the thin slice sample at various polarization angles, and extracting feature points from the original images and real-time images; Describing the feature points of the original image and the real-time image after feature point extraction based on the neighborhood point information of the feature points of the original image and the real-time image; Match feature points of the original image and the real-time image based on the feature point description information, and determine the optimal solution matrix for feature point matching; transforming the real-time image according to the optimal solution matrix, and superimposing the transformed real-time image with the original image to obtain a target thin slice sample image; The target thin slice sample images at each polarization angle are determined and stitched together to obtain a panoramic image of the thin slice sample.

3. The method according to claim 2, characterized in that The method of using a microscope to obtain an original image and a real-time image of the thin-film sample at each polarization angle, and extracting feature points from the original image and the real-time image, includes: Using a microscope to obtain an original image and a real-time image of the thin-film sample at various deflection angles, determining the Gaussian scale space of the original image and the real-time image to obtain a Gaussian blurred image; wherein the scale space of the real-time image is obtained by convolving the scale space of the original image with a Gaussian function; Performing Gaussian smoothing and downsampling processing on the Gaussian blurred image to obtain a processed thin slice sample image; wherein the Gaussian blurred image includes a Gaussian blurred original image and a Gaussian blurred real-time image; The Gaussian difference function is used to extract feature points from the processed thin slice sample image.

4. The method according to claim 2, characterized in that The feature point description of the original image and the real-time image after feature point extraction based on the neighborhood point information of the feature points of the original image and the real-time image includes: Determine neighborhood point information of feature points of the original image and the real-time image; the neighborhood point information includes coordinates, scale and direction of the neighborhood point; The coordinates, scale and direction of each feature point in the original image and the real-time image after feature point extraction are described based on the neighborhood point information.

5. The method according to claim 2, characterized in that The matching of feature points of the original image and the real-time image based on the feature point description information and determining the optimal solution matrix for the feature point matching includes: According to the feature point description information, the original image and the real-time image are matched and iteratively optimized to eliminate the mis-joined feature points and obtain the feature data after feature point matching; Acquire at least two sets of matching feature data pairs from the feature data after feature point matching, and perform matching data calculation on the at least two sets of matching feature data pairs to determine a homography matrix; The feature data after all feature points are matched are calculated according to the homography matrix, the target feature points and the cost function that meet the homography matrix are determined, and the optimal solution matrix is ​​determined according to the cost function.

6. The method according to claim 5, characterized in that The feature point matching and iterative optimization of the original image and the real-time image based on the feature point description information, eliminating the mis-joined feature points, and obtaining feature data after feature point matching include: A random sampling consensus algorithm is used to match feature points of the original image and the real-time image and perform iterative optimization to eliminate mismatched points. The calculation formula of the random sampling consensus algorithm is expressed as: Among them, s is the scale parameter, (x ′ ,y ′ ) represents the corner position of the real-time image, and (x, y) represents the corner position of the original image.

7. The method according to claim 2, characterized in that After obtaining the panoramic image of the thin-section sample, the method further includes: Performing stereoscopic custom segmentation on the panoramic image of the thin slice sample to obtain stereoscopic grid data under a single viewing area; The three-dimensional grid data under the single field of view is saved and named according to the position coordinates of the three-dimensional grid data under the single field of view in the panoramic image of the thin slice sample.

8. A microscope device and a thin-section sample scanning device, characterized in that: The device comprises: A feature point extraction module is used to obtain original images and real-time images of the thin-film sample at various polarization angles using a microscope device, and to extract feature points from the original images and real-time images; A feature point description module is used to describe the feature points of the original image and the real-time image after the feature points are extracted based on the neighborhood point information of the feature points of the original image and the real-time image; The feature point matching module is used to match the feature points of the original image and the real-time image based on the feature point description information, and determine the optimal solution matrix for feature point matching; An image fusion module is used to transform the real-time image according to the optimal solution matrix, and superimpose the transformed real-time image with the original image to obtain a target thin slice sample image; The thin-sheet sample scanning module is used to determine the target thin-sheet sample images at various deflection angles and stitch them together to obtain a panoramic image of the thin-sheet sample.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the microscope device according to claim 1 and the thin-section sample scanning method according to any one of claims 2 to 7.

10. A storage medium containing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the microscope apparatus according to claim 1 and the thin-section sample scanning method according to any one of claims 2 to 7.

Images