Crystal grain-based mineralogical classification system and method

Abstract

A method of enhancing resolution of an EDS image of a sample, comprising generating an EDS image of the sample, generating a non-EDS image of the sample, and using a machine learning algorithm to generate an enhanced resolution EDS image of the sample based on the generated feature map and based on the first EDS image, wherein the enhanced resolution EDS image has a higher resolution than the first EDS image.

Description

TECHNICAL FIELD

[0001] Embodiments relate to automated analysis of images, and in particular to grain-based mineralogical partitioning systems and methods. BACKGROUND

[0002] Optical petrography and mineralogy can be performed on subsurface oil and gas reservoir samples and mineral samples to provide information such as mineralogy, porosity, permeability, grain size distribution, and mineral associations, for example, for samples taken from a well or borehole. The information provided by optical petrography and mineralogy is important to mining operations as it can reveal the morphological characteristics, particle size, and particle count of the material as it is processed by the mineral. This data can provide supporting evidence for decisions made at the well or mine site that can potentially increase the yield of the mineral of interest.

[0003] However, optical petrography and mineralogy has generally proven difficult to automate and instead relies on a skilled petrologist to "point count" images of material samples from a well or borehole, which requires manual processing in which random points in the image are selected and manually defined by the petrologist. This manual processing is slow, laborious, and costly, thereby reducing the value of the analysis as scaling over the entire length of a core (extracted from the subsurface) is more difficult. Other analysis tools can be used to perform such mineralogical analysis for analyzing the structure and chemical composition of geological samples, such as quantitative mineralogical mapping based on energy dispersive X-ray spectroscopy (EDS) acquired from an electron microscope, or X-ray fluorescence (XRF) analysis, or generating backscattered electron (BSE) images or secondary electron (SE) images of the sample, but these tools are generally expensive and difficult to scale to the entire length of a core.

[0004] Particle separation (also referred to as particle partitioning), which identifies different material particles in a sample, is an important step in mineral analysis as it can reveal the morphological profile of the particles being analyzed in addition to the total particle count in the sample. However, for relatively complex, convoluted, and / or diverse samples, traditional histogram-based partitioning methods, such as those used in mineralogical mining, can not provide reliable results.

[0005] Currently, for relatively simple samples with good particle separation in the image, histogram thresholding-based particle analysis can be a suitable solution. However, for relatively complex samples in which particles are in contact with each other at multiple points or particles are clustered in clusters, histogram thresholding-based particle analysis can not be suitable. SUMMARY

[0006] In general aspects, an apparatus, system, non-transitory computer- readable medium (having computer-executable program code stored thereon that is executable on a computer system) and / or method can perform processes including generating an optical microscope image of a sample, using a first clustering algorithm to identify individual grains in the optical microscope image on a per-pixel basis; using a second clustering algorithm to classify the individual grains identified in the optical microscope image; and mapping the grain classification to known mineral content.

[0007] Implementations can include one or more of the following features, alone or in any combination. For example, the processes can further include identifying pore space in the sample on a per-pixel basis based on output of a third clustering algorithm that analyzes the optical microscope image, and identifying individual grains in the image using the first clustering algorithm can be based at least in part on the identified pore space.

[0008] The pore space can be filled with a polymer material.

[0009] The optical microscope image can include a plurality of registered optical microscope images.

[0010] Classifying the identified individual grains by the second clustering algorithm can include classifying each grain based on aggregate statistical information for a plurality of pixels corresponding to the grain.

[0011] In another general aspect, a method of enhancing resolution of an EDS image of a sample includes generating a first EDS image of the sample, generating a non-EDS image of the sample, generating an enhanced resolution EDS image of the sample based on the generated feature map and based on the first EDS image using a machine learning algorithm, wherein the enhanced resolution EDS image has a higher resolution than the first EDS image.

[0012] Implementations can include one or more of the following features, alone or in any combination.

[0013] For example, the non-EDS image can include a BSE image.

[0014] The non-EDS image can include an XRM image.

[0015] The non-EDS image can include an EM image.

[0016] The non-EDS image can include an SE image.

[0017] The processing can further include normalizing the generated non-EDS image using feature scaling, and generating the explicit feature map for the sample from the non-EDS image can include generating the explicit feature map from the normalized non-EDS image.

[0018] Using the machine learning algorithm can include using a trained regressor to generate the enhanced resolution EDS image for the sample, wherein the regressor is trained using features from a feature map generated from a non-EDS image of an object that is the target of the sample and information from an EDS image of the object that is the target. BRIEF DESCRIPTION OF DRAWINGS

[0019] Example embodiments will become better understood from the detailed description and accompanying drawings, wherein like reference numerals identify like elements, the specific embodiments are given by way of illustration only, and thus are not limiting of the example embodiments, wherein:

[0020] Figure 1 is an example image of a granular rock sample including an optical microscope image of the sample;

[0021] Figure 2 shows how different grains in the image of Figure 1 are classified, wherein three grains of the black outlined grains in the right magnified image are classified according to their material and / or properties;

[0022] Figure 3 shows a schematic diagram illustrating an embodiment of a mineral analyzer;

[0023] Figure 4 is a flowchart of an example process for enhancing the resolution of an EDS image of a sample based on a BSE image of the same sample; and

[0024] Figure 5 shows an example of a process of obtaining a relatively high resolution BSE image and a relatively low resolution image of a sample, and generating a feature map approximation from the BSE image, such that a high resolution EDS image can be generated based on the mapping.

[0025] Figure 6A shows a layer in a convolutional neural network (CNN) without sparsity constraints.

[0026] Figure 6B shows a layer in a convolutional neural network (CNN) with sparsity constraints.

[0027] Figure 7 shows a block diagram of a machine learning model according to an example embodiment.

[0028] Figure 8 shows an example of a computing system that can implement the techniques described herein. DETAILED DESCRIPTION

[0029] To address the deficiencies of the prior art, the improved technology described herein uses machine learning based filters and cascading classifiers to automatically separate and distinguish touching objects (particles) in images. With these techniques, touching particles can be divided and labeled as individual objects, making the analysis more reliable for those cases where histogram based thresholding methods are insufficient.

[0030] The application of machine learning techniques allows for more complex image division and classification, and allows for object recognition, separation, and classification algorithms that are much more powerful than previous possibilities.

[0031] Further, when imaging a mineralogy sample using both BSE and EDS, the resolution of the image obtained by BSE is typically higher than the image obtained by EDS when using the same acquisition time for both images. The result of the lower resolution in the EDS image results in lower sensitivity of mineral identification / classification and higher rate of mineral misclassification.

[0032] To address the issue of low resolution of images obtained by EDS imaging techniques, machine learning techniques can be used to model sample images obtained by other techniques (e.g., BSE imaging, SE imaging, X-ray Microscope (XRM), Focused Ion Beam Electron Microscope (FIB-SEM), and optical microscope (EM)), and then use the modeled images obtained by other techniques to enhance the resolution of the EDS images. In this way, relatively high resolution EDS images can be obtained using relatively short acquisition times and used for mineral identification / classification.

[0033] Automatic partitioning and classification of mineralogy images

[0034] According to the techniques described herein, division and classification of images of a mineralogy sample can be obtained from optical microscope images of the sample by using machine learning techniques. These images can be obtained by various optical techniques, including obtaining single and multi-modal images from a rock sample. The rock sample can be a thin section or a bulk sample, with or without voids with epoxy infiltration in the rock sample.

[0035] Figure 1 is an example image 100 of a particulate rock sample 102 that includes optical microscope images of the sample. The rock sample can include voids 104 between different grains 106, 108 of different materials in the sample, and in some implementations, the sample can be prepared prior to imaging by filling the voids with a fluid material (e.g., a polymeric material, an epoxy material, etc.) that hardens and solidifies in the voids. For example, in Figure 1In this case, the epoxy-filled pores of the sample 102 are represented by a first color (e.g., blue), while the different isolated grains 106, 108 are located between the epoxy material. As shown, the density of the isolated grains is high, and many of the grains are in contact with other grains at their boundaries. Figure 1

[0036] In some embodiments, the image 100 of the sample can include not only an optical microscope image of the sample, but also two or more registered images, where the different registered images can be obtained by different modalities (e.g., optical microscopy, EDS, BSE, XRF, etc.). In some embodiments, an image can be generated based on multiple images of the sample, where the multiple images are acquired from a single imaging modality using multiple illumination techniques. The different images can include, for example, multiple fluorescence images (e.g., obtained from different fluorescence wavelengths), or multiple polarization orientations, etc.

[0037] The image 100, including the optical microscope image of the rock sample and other such images, can be used as input to an unsupervised clustering algorithm or a supervised clustering algorithm to automatically determine the different grains in the image. In determining the different grains in the image, the clustering algorithm can use information from the pore space between the grains in addition to using information in the image corresponding to the grains themselves.

[0038] For example, due to the presence of the strong signal from the blue epoxy in the image, the pore space of the rock can be uniquely identified on a pixel-by-pixel basis. Because the optical signal from the epoxy in the pore space of the sample is relatively uniform compared to the signals from the various minerals in the sample, a clustering algorithm can be used to identify the signal from the epoxy, and thus to automatically identify the pore space in the sample.

[0039] The pattern of pore space identified in this way can be treated as a “mask” that defines the presence of mineral grains in the openings of the mask. From this mask, each individual grain in the image 100 can be separated into isolated objects. For example, by using a second round of machine learning segmentation (e.g., to identify grain boundary regions on a pixel-by-pixel basis), or by applying a separation algorithm, and marking connected objects, each individual grain can be automatically separated into multiple isolated objects.

[0040] ​In some embodiments, the applied separation algorithm can be based on ray-based object separation techniques or morphological watershed-based techniques, but other grain separation algorithms can also be used. For example, the separation algorithm can include so-called "instance segmentation" machine learning techniques such as mask RCNN, or object recognition techniques such as YOLO (You Only Look Once), or can combine machine learning techniques with traditional image processing. By automatically separating each individual grain into isolated objects in this way, each individual grain in the sample image can be identified even in the case of grain contact, so that relevant area object statistics (e.g., size, shape, and spatial associations of objects, which object classes are next to which other object classes) related to the sample can be determined, and also the internal structure of the objects (e.g., pixel class distribution within the objects, etc.).

[0041] Next, a clustering algorithm can be used to classify populations of separated grains in the image 100 into different types. Figure 2 An example of how different grains 202, 204, 206 in the image 100 can be classified is shown, in which the three grains in the magnified image are classified according to their material and / or properties by the black outline. The clustering algorithm can have a stronger discrimination power and be less susceptible to noise than traditional pixel-by-pixel classification techniques, since the clustering algorithm can utilize aggregated statistics of all pixels of the entire grain, rather than local information available for individual pixels. These statistics can include examining the pixel value distribution within the separated grain area, the pixel value distribution of a filtered version of the area (e.g., extracting local and non-local pixels or gray levels, local and non-local gradients, or local and non-local texture information), or size / shape metrics of the pixel area (e.g., area, perimeter, major axis length, minor axis, inscribed radius, Feret diameter, or any number of other morphological measures). In addition, the clustering algorithm can improve the ability to correctly classify different grains based on geometric properties (e.g., size, shape, Feret diameter, fitted ellipsoid, sphericity, etc.), gray level information (e.g., maximum, minimum, mean, and median gray level values, histograms, and texture filter information, etc.), thereby improving the ability to correctly classify different grains.

[0042] The different types of grains output by the classification step performed by the clustering algorithm can then be assigned to specific mineral types or classes. In one embodiment, the different types of grains can be manually mapped to specific mineral classes by manual training by an expert (e.g., a trained petrologist or mineralogist). In some embodiments, the different types of grains can be mapped to specific mineral classes by comparison to a standard database of mineral grain images. In some embodiments, the different types of grains can be mapped to specific mineral classes by identification or quantitative comparison to absolute mineral quantification techniques, such as, for example, EDS-based mineral analysis, either locally (by correlation data) or by training at a library.

[0043] The mineral classification of the grains in the image can then be analyzed for properties of interest to subsurface geologists, including grain size, size distribution, shape, exposed surface area, mineral associations, etc., which can then be correlated to critical functional properties such as reservoir permeability, diffusivity, or capillary pressure.

[0044] Figure 3 A schematic diagram illustrating an embodiment of a mineral analyzer 300 is shown. The mineral analyzer 300 analyzes image data, such as the image data shown in Figure 1 and Figure 2 In this example, the mineral analyzer 300 includes an image data receiver 302, a grain boundary determiner 306, and a grain classifier 304.

[0045] The image data receiver 302 receives image data. The image data can correspond to a sample, e.g., can include an image of a sample. In some embodiments, the image data receiver 302 receives the image data from a local data store. In some embodiments, the image data receiver 302 receives the image data from another device via a network.

[0046] The grain boundary determiner 306 determines boundaries between grains. For example, in some embodiments, the grain boundary determiner 306 can analyze individual pixels of the received image data to obtain color and brightness indicative of boundaries between grains. For example, when the pores of a sample forming boundaries between grains are injected with an epoxy material having a characteristic optical signature, analysis of the image data by the grain boundary determiner 306 can identify pixels emitting the characteristic optical signature, thereby determining the grain boundaries in the image data.

[0047] In some embodiments, the grain boundary determiner 306 can use image processing techniques (e.g., such as morphological techniques and / or ray casting techniques) to determine individual grains to analyze the content of the image. In some embodiments, the grain boundary determiner 306 can automatically separate individual grains into isolated objects, for example, by using a machine learning segmentation algorithm (e.g., to identify grain boundary regions on a pixel-by-pixel basis), or by applying a separation algorithm and labeling connected objects. In some embodiments, the separation algorithm can be based on ray-based object separation techniques or morphological watershed techniques, but other grain separation algorithms can also be used. For example, the separation algorithm can include a so-called “instance segmentation” machine learning technique such as mask RCNN, or an object recognition technique (e.g., YOLO), or a coupling of machine learning with traditional image processing. By automatically separating individual grains into isolated objects in this way, individual grains in the sample image can be identified so that relevant area object statistics (e.g., size, shape, spatial associations, which object classes are next to which other object classes) related to the sample can be determined, and also the internal structure of the objects (e.g., pixel class distribution in the objects, etc.).

[0048] The grain classifier 304 classifies the separated grains in the image data and includes a machine learning classifier 318. The machine learning classifier 318 classifies the image data from the sample. For example, the machine learning classifier 318 can classify the image data based on labeled training data (such as training data labeled by a human expert). Training data (classified grains) can also be provided by other analysis tools, such as EDS or electron backscatter diffraction (EBSD) based mineral / grain classification (on an electron microscope). Some embodiments can classify the image data using a classifier trained on labeled training data. The training data can include training samples corresponding to labeled grains. In some embodiments, specific sites are extracted from the training samples prior to training the classifier, and then only these specific sites are used to train the classifier.

[0049] The machine learning classifier 318 can be trained by a machine learning module (not shown) of the mineralogy analyzer 300 and / or can be trained and retrieved by another computing device via a network. The machine learning classifier 318 can have one of various configurations. For example, the machine learning classifier 318 can include a Naive Bayes model, a Random Tree model, a Random Forest model, a neural network model (e.g., a R-CNN model, a Mask CNN model, a UNET model), a logistic regression model, or a support vector machine model. The support vector machine model can be trained using sequential minimal optimization, for example. The instance segmentation model can operate as a single classification model or can be linked to subsequent image analysis steps, for example, a UNET model can be used to identify grain regions and boundary regions, and then morphological tools, ray casting, or simple labeling can be used to identify individual grains. Statistics of the identified grains can be used to assign the grains to particular classes.

[0050] In some implementations, the machine learning classifier 318 can classify samples for new samples (e.g., select labels for different grains in a sample image) based on similarity of the sample to characteristics of training samples. In other words, the machine learning classifier 318 can classify inputs based on similarity to learned characteristics of each class (label). In some implementations, the machine learning classifier 318 can provide a probability for each class, e.g., a number indicating a likelihood that the sample should be classified in that class.

[0051] EDS resolution enhancement based on multi-modal imaging

[0052] While generation of EDS images of samples generally has lower resolution and / or is more time consuming than generation of sample images using other techniques (e.g., such as BSE imaging, SE imaging, XRF imaging, FIB-SEM imaging, and EM imaging), the information obtained by the different modes is different and can provide different value to a user. While higher resolution EDS images can be obtained using longer acquisition times, the cost of increasing acquisition times can make such higher resolution EDS less desirable in high-demand production environments where high turn-around and short lead times are important.

[0053] Accordingly, many systems are designed to collect data from different modes (e.g., EDS and BSE) at different resolutions to meet time acquisition requirements, but with the caveat of speed at the expense of data quality. Generally, limitations of low resolution EDS in such systems include reduced mineral detection sensitivity and higher mineral misclassification rates. The former is due to the small contribution of the signal of a given mineral in a particular scan window where other minerals are more abundant, and the latter is due to mixed signal effects between multiple minerals, e.g., at boundaries between two minerals.

[0054] To address these issues, described herein are techniques that can enhance EDS image resolution based on machine learning and data fusion techniques, where details of an EDS image of a sample are modeled based on sample images obtained from the sample using other imaging techniques, e.g., such as BSE, SE, XRM, FIB-SEM, EM. This can improve the resolution of the final EDS image without significantly increasing the acquisition time of the EDS image.

[0055] For example, Figure 4 is a flowchart of an exemplary process 400 for enhancing the resolution of an EDS image of a sample based on a BSE image of the same sample. In the process 400, an EDS image of a sample can be acquired (402), and a non-EDS image of the sample (e.g., a BSE image, a SE image, an XRM image, a FIB-SEM image, an EM image) can be acquired (404). The non-EDS image can be normalized using feature scaling (406). An explicit feature map approximation can be generated from the normalized non-EDS image (408). For example, an Nth order feature map can be generated. In some implementations, N can be greater than or equal to 100. In some implementations, the feature map can be generated by radial basis function kernel features of a Monte Carlo approximation of a Fourier transform of the normalized non-EDS image (410). A regressor can be trained using explicit kernel features generated from the non-EDS image of the object of interest and information in the EDS image of the object of interest on a per-element basis (412). In implementations, the regressor can use a nearest neighbor regression where the data labels are continuous variables rather than discrete variables. In implementations, the label assigned to a query point can be computed based on the average of the labels of its nearest neighbors. The trained regressor can then be applied to enhance the resolution of a low resolution EDS image, thereby generating an enhanced resolution image from the low resolution EDS image. In some implementations, the enhanced resolution EDS image of the sample can be analyzed to determine the chemical composition of the imaged sample.

[0056] Figure 5 An example of the process 400 is illustrated, where a relatively high resolution BSE image 502 and a relatively low resolution EDS image 504 of a sample are obtained, and a feature map approximation is generated from the BSE image, such that a high resolution EDS image 506 can be generated based on the mapping.

[0057] Machine learning processes involving convolutional neural networks (CNNs) can be used to enhance the resolution of an EDS image of a sample based on a BSE image of the same sample. Figure 6A Layers in a convolutional neural network (CNN) without sparsity constraints are shown. Figure 6BLayers in a convolutional neural network (CNN) with sparse constraints are shown. Referring to Figure 6A and Figure 6B Different configurations of neural networks for at least one example implementation will be described. An example hierarchical neural network is shown in Figure 6A . The hierarchical neural network includes three layers 610, 620, 630. Each layer 610, 620, 630 can be formed of a plurality of neurons 605. In this implementation, no sparse constraints are applied. Thus, all neurons 605 in each layer 610, 620, 630 are networked with all neurons 605 in any adjacent layer 610, 620, 630.

[0058] Due to the small number of neurons 605 and layers, Figure 6A the example neural network shown is not computationally complex. However, due to the density of connections (e.g., connections between neurons / layers), Figure 6A the arrangement of the neural network shown can not scale to larger sized networks. In other words, the computational complexity can be too great as the network size scales and scales in a non-linear manner. Thus, if the neural network needs to scale to work on inputs with a large number of dimensions, networking all neurons 605 in each layer 610, 620, 630 with all neurons 605 in one or more adjacent layers 610, 620, 630 can be computationally too complex.

[0059] The computational complexity of a neural network can be reduced using initial sparse conditions. For example, if the neural network acts as an optimization process, the neural network method can handle high dimensional data by limiting the number of connections between neurons and / or layers. An example of a neural network with sparse constraints is shown in Figure 6B . Figure 6B The neural network shown is arranged such that each neuron 605 is connected to only a small number of neurons 605 in adjacent layers 640, 650, 660. This can form an incompletely connected neural network and can scale to function with higher dimensional data. The small number of connections allows the number of connections between neurons to scale in a substantially linear manner as compared to a fully networked neural network.

[0060] In some implementations, a convolutional neural network that is not fully connected and has lower complexity than a fully connected neural network can be used. The convolutional neural network can also use pooling or max pooling to reduce the dimensionality of data flowing through the neural network (and thus reduce complexity). Other methods can be used to reduce the computational complexity of the convolutional neural network.

[0061] Figure 7A block diagram of a machine learning model is shown in accordance with example embodiments. The ML model 700 can be configured to implement pixel level image resolution enhancement. The ML model 700 can include, for example, a regression type, split type convolutional neural network (CNN). The pixel level image enhancement can generate an enhanced EDS image 706 from at least a low resolution EDS image 702 and a BSE image 704 having complementary characteristics and information corresponding to the same sample.

[0062] A convolutional layer or convolution can be configured to extract features from an image. The features can be based on color, frequency domain, edge detector, etc. The convolution can have a filter (sometimes referred to as a kernel) and a stride. For example, the filter can be a 1x1 filter (or 1x1xn for converting to n output channels, the 1x1 filter is sometimes referred to as a pointwise convolution) with a stride of 1, which results in an output of a cell based on a combination (e.g., addition, subtraction, multiplication, etc.) of the features of the cell for each channel at an MxM grid location. In other words, a feature map having more than one depth or channel is combined into a feature map having a single depth or channel. The filter can be a 3x3 filter with a stride of 1, which results in an output of fewer cells for each channel of the MxM grid or feature map.

[0063] The output can have the same depth or number of channels (e.g., 3x3xn filter, where n = depth or number of channels, sometimes referred to as a depthwise separable filter) or a reduced depth or number of channels (e.g., 3x3xk filter, where k < depth or number of channels). Each channel, depth, or feature map can have an associated filter. Each associated filter can be configured to emphasize a different aspect of the channel. In other words, different features can be extracted from each channel based on the filter (sometimes referred to as a depthwise separable filter). Other filters are within the scope of the present disclosure.

[0064] Another type of convolution can include a combination of more than two convolutions. For example, the convolution can include a depthwise and pointwise separable convolution. This can include, for example, a two-step convolution. The first step can include a depthwise convolution (e.g., 3x3 convolution). The second step can include a pointwise convolution (e.g., 1x1 convolution). The depthwise and pointwise convolution can include a separable convolution because different filters (e.g., filters to extract different features) can be used for each channel or each depth of the feature map. In example implementations, the pointwise convolution can convert the feature map to include c channels based on the filter. For example, an 8x8x3 feature map (or image) can be converted to an 8x8x256 feature map (or image) based on the filter. In some implementations, more than one filter can be used to convert the feature map (or image) to an MxMxc feature map (or image).

[0065] The convolutions can be linear. Linear convolutions describe an output as a linear time-invariant (LTI) function of an input. The convolutions can also include a rectified linear unit (ReLU). The ReLU is an activation function that rectifies an LTI output of a convolution and limits the rectified output to a maximum value. The ReLU can be used to speed up convergence (e.g., more efficient computation).

[0066] In example implementations, the first type of convolution can include a 1x1 convolution, and the second type of convolution can include a depthwise and pointwise separable convolution. Each of the plurality of convolution layers 720, 735, 740, 745, 750, 755, 760 can have a plurality of units and each unit at least one bounding box. The convolution layers 715, 720, 725 and the addition layer 730 can be used to convert the images 702, 704 to feature maps of a size equivalent to the feature maps of the Conv_3 layer of the VGG-16 standard. In other words, the convolution layers 715, 720, 725 and the addition layer 730 can convert the images 702, 704 to 38x38x512 feature maps.

[0067] In example implementations, the ML model 700 CNN (e.g., a regression type CNN) can include a plurality of convolution layers 705, 710, 715, 720, 725, 730, 735, 740, 745, 750, 755, 760, 765, and 770. The plurality of convolution layers 705, 710, 715, 720, 725, 730, 735, 740, 745, 750, 755, 760, 765, and 770 can correspond to at least one type of convolution layer, respectively. As shown, the convolution layer 705 can include a first type of convolution, the convolution layer 710 can include a second type of convolution, the convolution layers 715, 720, 725, 730 can include a second type of convolution and a third type of convolution, the convolution layer 735 can include a third type of convolution, the convolution layer 740 can include a fourth type of convolution, the convolution layers 745, 755, 765 can include a fifth type of convolution, the convolution layers 750, 760 can include a sixth type of convolution, and the convolution layer 770 can include a seventh type of convolution. Figure 7

[0068] ​Each convolutional layer is capable of generating many alternating convolutions, so the weight matrix is a x * y * n tensor, where x * y is the size of the sliding window (typically x = y), and n is the number of convolutions. The low resolution EDS image 702 and the BSE image 704 can be input to the CNN. In the first convolutional layer, the image 702 and the image 704 can be transformed using a 224 * 224 * 3 weight matrix. The convolutional layer 710 can transform the resulting feature map using a 224 * 224 * 64 weight matrix. The convolutional layer 715 can transform the resulting feature map using a 112 * 112 * 128 weight matrix. The convolutional layer 720 can transform the resulting feature map using a 56 * 56 * 256 weight matrix. The convolutional layer 725 can transform the resulting feature map using a 28 * 28 * 512 weight matrix. The convolutional layer 730 can transform the resulting feature map using a 14 * 14 * 512 weight matrix, and the convolutional layer 735 can transform the resulting feature map using a 7 * 7 * 4096 weight matrix.

[0069] The next portion of the ML model 700 can be configured to transform the feature map output from the convolutional layer 735 into an image of a size equivalent to the input image (e.g., the image 702 or the image 704). The convolutional layer 740 receives the feature map from the convolutional layer 735 and transforms the feature map using a 7 * 7 * 4096 weight matrix. The convolutional layer 745 can transform the resulting feature map using a 7 * 7 * class weight matrix (where class is the number in the feature map). The convolutional layer 750 can transform the resulting feature map using a 14 * 14 * class weight matrix, the convolutional layer 755 can transform the resulting feature map using a 14 * 14 class weight matrix along with the feature map output from the convolutional layer 725 (convolutional layer 775), and the convolutional layer 760 can transform the resulting feature map using a 28 * 28 * class weight matrix along with the feature map output from the convolutional layer 720 (convolutional layer 780). Figure 1 The convolutional layer 765 can transform the resulting feature map using a 28 * 28 * class weight matrix along with the feature map output from the convolutional layer 720 (convolutional layer 780). The convolutional layer 770 can transform the resulting feature map using a 224 * 224 * class weight matrix. The resulting feature map can include the output image (e.g., the fused image 706). Figure 1 The convolutional layer 765 can transform the resulting feature map using a 28 * 28 * class weight matrix along with the feature map output from the convolutional layer 720 (convolutional layer 780). The convolutional layer 770 can transform the resulting feature map using a 224 * 224 * class weight matrix. The resulting feature map can include the output image (e.g., the fused image 706).

[0070] Once a model (e.g., the ML model 700) architecture is designed (and / or in operation), the model should be trained (sometimes referred to as developing the model). The model can be trained using a plurality of sample images obtained through BSE and EDS imaging.

[0071] Image enhancement using a machine learning (ML) model can include two stages. In the first stage, a supervised learning can be used to train an enhancement algorithm. In the second stage, the enhancement algorithm can be employed. As noted above, example implementations can use a convolutional neural network (CNN) based enhancement algorithm. In the first stage, a neural network can be trained, e.g., two images with first and second characteristics can be input to the network. The output of the network can be compared to a ground truth image with the most ideal characteristics that we want the network to reproduce. An evaluation metric can be used to quantify the difference between the output and the ground truth image. This difference is used to update the network parameters that identify the training process. This process is iteratively repeated using multiple image examples until the difference between the output and the ground truth image falls within a desired magnitude and the training process ends.

[0072] Figure 8 An example of a computing system that can be used to implement the techniques described herein is shown. The system 800 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, mobile devices, and other appropriate computers. Portions of the system 800 can be implemented using various other hardware.

[0073] The components and arrangements of the system 800 can change. The system 800 includes components, such as a processor (e.g., a CPU) 805, a memory 810, input / output (I / O) device(s) 825, and a nonvolatile storage device 820. The system 800 can be implemented in various ways. For example, an integrated platform, such as a workstation, personal computer, laptop, etc., can include the processor 805, the memory 810, the nonvolatile storage device 820, and the I / O device(s) 825. In such a configuration, the components 805, 810, 820, and 825 can be connected by a local bus interface and access a database via an external connection. The connection can be implemented by a direct communication link, a local area network (LAN), a wide area network (WAN), and / or other suitable connections. The system 800 can be a standalone system or a part of a subsystem, which in turn can be a part of a larger system.

[0074] The processor 805 can be one or more processing devices, such as a microprocessor. The memory 810 can be one or more memory devices configured to store information used by the processor 805 to perform certain functions in accordance with embodiments of the present application. The storage 820 can be a volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other type of storage device or computer readable medium. In one embodiment, the memory 810 includes one or more executable instructions 815 loaded from the storage 820 or elsewhere that, when executed by the processor 805, perform various processes, operations, or functions consistent with the processes described herein. For example, the memory 810 can include programs corresponding to components of the image analyzer 300, such as the image data receiver 302, the grain boundary determiner 306, and the grain classifier 304, and execute instructions to perform one or more of the processes described herein. The memory 810 can also include other programs that perform other functions and processes, such as programs that provide communication support, Internet access, and the like.

[0075] The methods, systems, and articles of manufacture described herein are not limited to separate programs or computers configured to perform particular tasks. For example, the memory 810 can be configured with executable instructions 815 that, when executed by the processor 805, perform several functions. Additionally, the processor 805 can execute one or more programs located remotely from the system 800. For example, the system 800 can access one or more programs that, when executed, perform functions related to embodiments described herein.

[0076] The memory 810 can also be configured with an operating system (not shown) that, when executed by the processor 805, performs several functions. The choice of operating system and even its use is not important.

[0077] The I / O device(s) 825 can include one or more input / output devices allowing the system 800 to receive and / or transmit data. For example, the I / O device(s) 825 can include one or more input devices such as a keyboard, touchscreen, mouse, etc. that enable data to be entered by a user, e.g., sorting and analysis requests, adjustments of thresholds and pollution conditions, etc. Further, the I / O device(s) 825 can include one or more output devices such as a display screen, CRT monitor, LCD monitor, plasma display, printer, speaker device, etc. that enable data to be output or presented to the user. The I / O device(s) 825 can also include one or more digital and / or analog communication input / output devices that allow the computing system 800 to communicate with other machines and devices such as other continuous remote servers processing sample profile queries. The system 800 can input data from and output data to external machines and devices via the I / O device(s) 825. The configuration and number of input and / or output devices incorporated in the I / O device(s) 825 is not critical.

[0078] The systems and techniques described herein can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of these back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0079] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0080] A number of implementations have been described. Nevertheless, it will be understood that various modifications can be made without departing from the spirit and scope of the disclosure.

[0081] In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps can be provided, or steps can be eliminated, from the described flows, and other components can be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.

Claims

1. A method of automatically identifying mineral inclusions of a sample, the method comprising: generating an optical microscope image of the sample; identifying individual grains in the optical microscope image using a first clustering algorithm to identify grain boundaries on a per-pixel basis; classifying the identified individual grains in the optical microscope image using a second clustering algorithm, including classifying each grain based on aggregate statistical information for a plurality of pixels corresponding to the grain; and mapping the classified individual grains to known mineral inclusions.

2. The method of claim 1, further comprising: identifying pore space in the sample based on an output of a third clustering algorithm that analyzes the optical microscope image on a per-pixel basis, and wherein identifying the individual grains in the image using the first clustering algorithm is performed at least in part based on the identified pore space.

3. The method of claim 2, wherein, The pore space is filled with a polymeric material.

4. The method of claim 1, wherein, The optical microscope image comprises a plurality of registered optical microscope images.

5. A system for automatically identifying mineral inclusions of a sample, the system comprising: a processor; and a memory comprising executable instructions that, when executed by the processor, cause the system to: generate an optical microscope image of the sample; identify individual grains in the optical microscope image using a first clustering algorithm to identify grain boundaries on a per-pixel basis; classify the identified individual grains in the optical microscope image using a second clustering algorithm, including classifying each grain based on aggregate statistical information for a plurality of pixels corresponding to the grain; and map the classified individual grains to known mineral inclusions. The executable instructions, when executed by the processor, further cause the system to:

6. The system of claim 5, wherein, identify pore space in the sample based on an output of a third clustering algorithm that analyzes the optical microscope image on a per-pixel basis, and wherein identifying the individual grains in the image using the first clustering algorithm is performed at least in part based on the identified pore space. The pore space is filled with a polymeric material.

7. The system of claim 6, wherein, The optical microscope image comprises a plurality of registered optical microscope images.

8. The system of claim 5, wherein, ​

Images