Method for extracting color and texture information from rock particle example images

Abstract

Systems and methods are provided to extract features (e.g., color, texture features) from images of arbitrary shape and size by implementing global average pooling (GAP) and partial convolution in an autoencoder (AE) to analyze the images. A global average pooling (GAP) layer can be used at the last layer of the encoder of the AE to make the features rotationally and translationally invariant and scale equivariant. In addition, partial convolution can be used in the encoder to logically ignore invalid pixels (e.g., background, image errors, other objects) or any pixels (e.g., in any region) in the image.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 562,275, filed March 7, 2024, entitled “METHOD TO EXTRACT COLOR AND TEXTURE INFORMATION FROM ROCK PARTICLE INSTANCE IMAGES,” and to U.S. Non-Provisional Application No. 18 / 888,739, filed September 18, 2024, entitled “METHOD TO EXTRACT COLOR AND TEXTURE INFORMATION FROM ROCK PARTICLE INSTANCE IMAGES,” both of which are incorporated herein by reference in their entirety for all purposes. Background Technology

[0002] This disclosure generally relates to systems and methods for extracting color and texture information from images of rock grain instances, and more specifically, to integrating global average pooling (GAP) and partial convolution into an autoencoder (AE) to analyze images of rock grains.

[0003] This section is intended to introduce the reader to various aspects of the art that are relevant to the present invention, which are described and / or claimed below. It is believed that this discussion will help provide the reader with background information to better understand the various aspects of this disclosure. Therefore, it should be understood that these statements should be read in this context and should not be construed as an admission of any kind.

[0004] Rock particles are often the basic unit of domain-specific post-processing analysis, which is widely used in industrial and scientific applications, including but not limited to space exploration, mining, civil engineering, geothermal energy, and oil and gas. Image data of rock particles typically comes from imaging systems that produce digital images or three-dimensional (3D) images from laser scanners. Once rock particles are detected and segmented, they can be used to calculate rock particle properties, such as size, shape, texture, and other morphological, rock-related, and rock-physical properties, to answer domain-specific questions.

[0005] In oil and gas, geothermal, and scientific exploration applications, rock particles are generated during drilling activities. These rock particles are called rock cuttings (or debris, depending on their size). Rock cuttings are commonly used to identify lithological types for subsurface characterization and are one of the most available and lowest-cost data sources for understanding and characterizing subsurface rock properties. Therefore, there is an urgent industry need to automate the analysis of rock cuttings to extract rock particle properties (e.g., features), thereby reducing labor costs, improving the accuracy and efficiency of the analysis process, and shortening interpretation turnaround time. Specifically, existing feature extraction techniques may be limited in extracting texture features from rock cuttings instance images due to the nature of texture (such as spatial information captured in features, feature size depending on input size, shape similarity affected by the presence of background, and blurred reconstructed images due to pixel-based reconstruction losses). For example, rock cuttings instances may have arbitrary shapes and sizes, which may lead to the introduction of background pixels into the rock cuttings instance image during image acquisition, and these background pixels may be excluded from the feature extraction process. Furthermore, some portions of rock fragment instance images may include pixels that do not represent any features of rock particles (e.g., imaging errors, other objects), and these pixels may be excluded from the feature extraction process. These pixels do not represent any features of rock particles and are considered invalid pixels when extracting features from rock fragment instance images. Therefore, it is desirable to automatically analyze rock fragments to extract color and texture information from rock particle instance images of arbitrary shapes and sizes that contain invalid pixels. It should be noted that in some embodiments, the systems and methods described herein can be used to exclude any pixels (e.g., in any region of the input image) from the feature extraction process for image processing purposes, such as image inpainting. Summary of the Invention

[0006] The following provides an overview of the specific embodiments described herein. It should be understood that these aspects are presented only to provide the reader with a brief overview of these specific embodiments, and are not intended to limit the scope of this disclosure.

[0007] This paper describes systems and methods for extracting features (e.g., color, texture features) from images of arbitrary shape and size by implementing global average pooling (GAP) and partial convolution in an autoencoder (AE). It should be noted that the feature extraction techniques described herein can be applied to images from other images (e.g., core images, borehole images) and / or images from any domain (such as remote sensing and medical imaging). In some implementations, an autoencoder (AE) that can be data-driven and / or unsupervised can be used to extract texture features from the image. In some implementations, a variational autoencoder (VAE) can be used to control the distribution of the latent space. In some implementations, a plain AE can be used instead of a VAE. A global average pooling (GAP) layer can be used at the last layer of the encoder to make the features rotation- and translation-invariant and scale-equivariant. Furthermore, partial convolution can be used in the encoder to logically ignore invalid pixels in the image (e.g., background, image errors, other objects). It should be noted that in some implementations, the systems and methods described herein can be used to exclude any pixel (e.g., in any region of an input image) from the feature extraction process for image processing purposes, such as image inpainting.

[0008] Some embodiments of this disclosure include a method comprising: receiving an image via an encoder; generating a feature map via the encoder by performing a partial convolution operation using the image and a mask image of the image; generating an updated mask image via the encoder based on the dimensions of the mask image and the feature map; performing a global average pooling (GAP) operation on the feature map via the encoder using the updated mask image to generate a global feature map; and generating an output feature image of a rock cutting instance image via the encoder based on the global feature map.

[0009] Some embodiments of this disclosure also include a method for training an autoencoder (AE), the method comprising: generating a training image set from a plurality of images; generating a corresponding mask image for the training image set; and inputting the training image set and the corresponding mask image into an encoder of the AE. Additionally, the encoder of the AE is configured to: generate a corresponding feature map for the training image set by performing a partial convolution operation using the training images and the corresponding mask image; and generate a corresponding global feature map by performing a global average pooling (GAP) operation on the corresponding feature map. The method for training the AE further comprises: generating a corresponding output image for the training image set based on the corresponding global feature map via a decoder of the AE; and training the AE based on a loss function using the corresponding output image and the training image.

[0010] Some embodiments of this disclosure also include an autoencoder (AE), which includes an encoder. The encoder includes convolutional layers configured to generate feature maps by performing partial convolution operations using an image and a mask image of the image. The encoder also includes a global average pooling (GAP) layer configured to generate a global feature map by performing a GAP operation on the feature map using an updated mask image generated based on the mask image. The AE also includes a decoder configured to generate an output image based on the global feature map.

[0011] Various modifications may be made to the above features with respect to various aspects of this disclosure. Additional features may also be incorporated into these aspects. These modifications and additional features may exist individually or in any combination. For example, the various features discussed below with respect to one or more of the illustrated embodiments may be incorporated individually or in any combination into any aspect of the foregoing description of this disclosure. The brief overview presented above is intended to familiarize the reader with certain aspects and context of embodiments of this disclosure, without limiting the claimed subject matter. Attached Figure Description

[0012] A better understanding of the various aspects of this disclosure can be achieved by reading the following detailed description and referring to the accompanying drawings, in which: Figure 1 A drilling system according to an embodiment of this disclosure is illustrated; Figure 2 An example is illustrated of a mud shaker for removing drill cuttings from drilling fluid according to an embodiment of this disclosure; Figure 3 An example of a drill bit that generates cuttings according to an embodiment of this disclosure is shown; Figure 4 An example of an analysis and control system according to an embodiment of this disclosure for monitoring and control is illustrated. Figure 1 The system for drilling operations; Figure 5 Examples of embodiments according to this disclosure are provided. Figure 4 The analysis and control system executes the cuttings collection and image acquisition workflow; Figure 6 An example workflow for lithological characterization of rock cuttings based on image analysis, according to an embodiment of the present disclosure, is illustrated. Figure 7 Images of rock samples and corresponding segmented individual rock fragment instances are illustrated according to embodiments of this disclosure; Figure 8 Examples of embodiments according to this disclosure are provided. Figure 7Other implementations of segmented images of individual rock debris instances; Figure 9 Examples of embodiments of this disclosure for use Figure 6 Example workflow of the property estimation process; Figure 10 An embodiment of the present disclosure illustrates a process for implementing an encoder to transform a segmented image of individual rock debris instances with invalid pixels from an image space to a latent vector space; Figure 11 An example of an image analysis process using a partially textured VAE according to an embodiment of the present disclosure is illustrated, wherein the partially textured VAE includes a mask-aware texture coding layer (TEL) in the encoder and uses partial convolution in the encoder of the partially textured VAE; Figure 12 An example of an image analysis process using a partially textured VAE according to an embodiment of the present disclosure is illustrated, wherein the partially textured VAE includes a mask-aware global average pooling (GAP) layer in the encoder and uses partial convolution in the convolutional layer of the encoder of the partially textured VAE. Figure 13 This is an example of an implementation scheme according to the present disclosure for use in... Figure 12 A flowchart illustrating the training process of a partial texture VAE; Figure 14 This is an example of the use of an embodiment according to this disclosure. Figure 13 A flowchart illustrating the feature extraction process of a trained partial-texture VAE encoder extracting features from an analyzed image; and Figure 15 A visualization of rock fragments according to an embodiment of this disclosure is shown. Detailed Implementation

[0013] One or more specific embodiments of this disclosure will be described below. These described embodiments are merely examples of the currently disclosed technology. Furthermore, to provide a brief description of these embodiments, not all characteristics of an actual implementation may be described in the specification. It should be understood that, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer's specific goals, such as adhering to system-related and business-related constraints, which may vary depending on the implementation. Moreover, it should be understood that such development work may be complex and time-consuming, but will remain a routine task in design, fabrication, and manufacturing for those skilled in the art who benefit from this disclosure.

[0014] When describing the elements of various embodiments of this disclosure, the articles “an,” “a,” and “the / described” are intended to mean the presence of one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may exist in addition to the listed elements. Furthermore, it should be understood that references to “an embodiment” or “an embodiment” of this disclosure are not intended to exclude the existence of additional embodiments that also incorporate the listed features.

[0015] As used herein, the terms “connect,” “connection,” “connected,” “in connection with,” and “connecting” are used to mean “directly connected to” or “connected to” via one or more elements; and the term “set” is used to mean “one element” or “more than one element.” Furthermore, the terms “couple,” “coupling,” “coupled,” “coupled together,” and “coupled with” are used to mean “directly coupled together” or “coupled together via one or more elements.”

[0016] Furthermore, as used herein, the terms “in real time,” “real-time,” or “substantially real-time” are used interchangeably and are intended to describe operations performed without any perceptible inter-operational interruptions (e.g., computational operations). For example, as used herein, data associated with the system described herein may be collected, transmitted, and / or used to control computations “substantially in real time,” such that data reading, data transmission, and / or data processing steps occur once per second, once every 0.1 seconds, once every 0.01 seconds, or even more frequently during system operation (e.g., while the system is operating). Additionally, as used herein, the terms “continuous,” “constantly,” or “persistently” are intended to describe operations performed without any significant interruptions. For example, as used herein, control commands may be transmitted to certain devices every five minutes, every minute, every 30 seconds, every 15 seconds, every 10 seconds, every 5 seconds, or even more frequently, allowing adjustment of the device's operating parameters without causing any significant interruption to the device's closed-loop control. Furthermore, as used herein, the terms “automatic,” “automated,” “autonomous,” etc., are intended to describe operations performed, for example, by a computing system (i.e., solely by the computing system, without human intervention). Indeed, while some operations described herein may not be explicitly described as being performed automatically and / or substantially in real time during the operation of the computing system and / or the devices controlled by the computing system, it should be understood that these operations can indeed be performed automatically and / or substantially in real time during the operation of the computing system and / or the devices controlled by the computing system to improve the functionality of the computing system (e.g., by facilitating faster operational decisions without human intervention, and by improving the accuracy of operational decisions, for example, by eliminating the possibility of human error), as described in more detail herein.

[0017] As described above, rock cuttings are generated whenever an activity involves a drilling process, and these rock cuttings are typically available at the well site. The properties of the rock cuttings can include the observed or measured geological properties associated with them. Currently, rock cuttings are generally underutilized by geologists and reservoir engineers in the oil and gas industry for subsurface characterization. When these rock cuttings are observed and interpreted by the human eye, it is highly dependent on the interpreter and relatively time-consuming and labor-intensive. Therefore, there is a need within the industry to automate the rock cuttings analysis process. To this end, the implementation described herein provides a domain-based image analysis workflow that includes multiple computational modules to automatically extract relevant geological information from rock cuttings. As used herein, the terms “image,” “digital image,” “photograph,” and “photo” are intended to be used interchangeably. A digital image can be divided into small geometric subunits called image pixels, which have pixel values ​​corresponding to the image content. Depending on the content of the image, a digital image may include different color channels, such as red channel image data indicating the target brightness of red, blue channel image data indicating the target brightness of blue, green channel image data indicating the target brightness of green, or grayscale image data indicating the target brightness of gray. Image data corresponding to the image content indicates the target visual characteristic (e.g., brightness or color) at one or more specific points (e.g., image pixels) in the image content, for example by indicating the brightness level of the color channel (e.g., grayscale level). The grayscale level is a discrete level (e.g., 0, 1...255) corresponding to the quantized brightness (e.g., the brightness of the color channel, the brightness of gray) at the image pixel. For example, the brightness level can be maximized when the grayscale value is 255 and minimized when the grayscale value is 0. For example, a digital image may have three color channels (e.g., red (R), green (G), blue (B)), and the image pixels of the digital image may have pixel values ​​of (R, G, B) (e.g., (255, 255, 255)). In some embodiments, the digital image may have more than three channels per pixel (e.g., hyperspectral images, multispectral images). In some embodiments, the digital image may be a grayscale image and have only one color channel, and the image pixels of the digital image may have pixel values ​​corresponding to grayscale levels (e.g., 0, 1…255). The relationship between the grayscale level of an image pixel and the corresponding brightness at that image pixel is associated with the imaging system used to capture the image. Furthermore, although this document describes systems and methods for analyzing images of drill cuttings, it should be understood that the embodiments described herein may be able to analyze images of other types of rock particles, such as other types of rock cuttings, debris, etc., as well as non-rock objects in the drilling mud, such as mud additives, metal filings, and foreign matter.

[0018] Figure 1A drilling system 10 according to an embodiment described herein is illustrated. As illustrated, in some embodiments, the drill string 12 may be suspended at its upper end by a kelly and traveling block 14, and terminated at its lower end by a drill bit 16 (in... Figure 3 (As shown in the diagram). The drill string 12 and drill bit 16 are rotated via a rotary table 18 on the drill rig 20, thereby drilling a borehole 22 in the formation 24, a portion of which can be fitted with casing 26. As illustrated, in some embodiments, drilling fluid or drilling “mud” 28 can be pumped by mud pump 30 through a connecting mud line 32 to the upper end of the hollow drill string 12. From there, the drilling fluid 28 can be pumped down through the drill string 12, exiting the drill string 12 through an opening in the drill bit 16, and returning to the surface through the annulus formed between the wall of the borehole 22 and the outer diameter of the drill string 12. Once to the surface, the drilling fluid 28 can be returned through a return line 34, for example via a bell sub 36. As illustrated, in some embodiments, a blowout preventer 38 can be used to prevent blowouts in the drilling system 10.

[0019] like Figure 1 As illustrated, drill cuttings formed by the crushing of rock in formation 24 by drill bit 16 can typically be removed from the returned drilling fluid 28 via a mud vibrating screen 40 in return line 34, allowing the drilling fluid 28 to be reused for injection. The mud vibrating screen 40 includes a vibrating screen pit 42 and an air trap 44. The drilling fluid 28 can then be transported to a mud pit 48 from which a mud pump 30 can extract the drilling fluid 28. Figure 2 This example illustrates drill bit cuttings 46 that have been removed from the drilling fluid 28 in the vibrating screen pit 42 of the mud vibrating screen 40. Figure 2 In this process, one or more cameras 52 can be used to capture drill cuttings 46. Furthermore, Figure 3 This illustrates how drill bit 16 can generate drill cuttings 46, which then flow back upwards in drilling fluid 28 through the annulus formed between the wall of borehole 22 and the outer diameter of drill string 12.

[0020] In addition, such as Figure 1 As illustrated, in some embodiments, the analysis and control system 50 (e.g., a mud logging unit) can be used to control the drilling system 10 and provide analysis of the drill bit cuttings 46, as described in more detail herein. Specifically, in some embodiments, the analysis and control system 50 can be configured to automatically analyze the drilling process during drilling. Figure 1 The illustrated drilling system 10 is operated by a camera (e.g., Figure 2 Images of drill cuttings 46 automatically captured by one or more cameras 52, as shown in more detail herein. Figure 2As illustrated, in some embodiments, one or more cameras 52 may be directly associated with the mud vibrating screen 40 (e.g., directly attached to the mud vibrating screen or located adjacent to or adjacent to the mud vibrating screen). However, in other embodiments, one or more cameras 52 may be other types of cameras that are not directly associated with the mud vibrating screen 40.

[0021] Figure 4 System 54 is illustrated, which includes analysis and control system 50 for monitoring and control. Figure 1 The drilling system 10 is described in more detail herein. In some embodiments, the analysis and control system 50 may include one or more analysis modules 56 (e.g., tools or instruments) configured to perform various functions of the embodiments described herein, including but not limited to using certain analysis algorithms to analyze images of drill cuttings 46 captured by one or more cameras 52, as described in more detail herein. In some embodiments, to perform these various functions, the analysis modules 56 execute instructions on one or more processors 58 of the analysis and control system 50, which may be connected to one or more storage media 60 of the analysis and control system 50. In fact, in some embodiments, one or more analysis modules 56 may be stored in one or more storage media 60. In some embodiments, one or more analysis modules 56 include analysis hardware (e.g., analysis circuitry) and / or software including processor-executable instructions and associated data.

[0022] In some embodiments, one or more processors 58 may include a microprocessor, microcontroller, processor module or subsystem, programmable integrated circuit, programmable gate array, digital signal processor (DSP), or other control or computing device. In some embodiments, one or more storage media 60 may be implemented as one or more non-transitory computer-readable or machine-readable storage media. In some embodiments, one or more storage media 60 may include one or more different forms of memory, including semiconductor memory devices such as dynamic or static random access memory (DRAM or SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory; magnetic disks such as fixed disks, floppy disks, and removable disks; other magnetic media, including magnetic tape; optical media such as optical discs (CDs) or digital video discs (DVDs); or other types of storage devices. It should be noted that the processor-executable instructions and associated data of the analysis module 56 may be located on one computer-readable or machine-readable storage medium of the storage medium 60, or alternatively, may be located on multiple computer-readable or machine-readable storage media distributed across a large system that may have multiple nodes. Such computer-readable or machine-readable storage media are considered part of an article (or article of manufacture), which can refer to any single or multiple manufactured components. In some embodiments, one or more storage media 60 may be located in a machine that executes machine-readable instructions, or may be located at a remote site from which machine-readable instructions can be downloaded via a network for execution.

[0023] In some embodiments, processor 58 may connect to network interface 62 of analysis and control system 50 to allow analysis and control system 50 to communicate with various surface sensors 64 (Internet of Things (IoT) sensors, instruments, etc.) and / or downhole sensors 66 described herein, and with actuators 68 and / or PLC 70 of surface equipment 72 and / or downhole equipment 74 to monitor and / or control the operation of drilling system 10, as described in more detail herein. Furthermore, in some embodiments, network interface 62 may also facilitate analysis and control system 50 to communicate with one or more cameras 52, as described in more detail herein. In some embodiments, network interface 62 may also facilitate analysis and control system 50 to transmit data to cloud-based service 76 (or other wired and / or wireless communication networks) for example, to archive data or to enable external computing system 78 (e.g., a cloud-based computing system in some embodiments) to access data and / or interact remotely with analysis and control system 50. For example, in some embodiments, some or all of the analysis modules 56 described in more detail herein may be deployed via cloud and edge.

[0024] In some embodiments, the analysis and control system 50 may include an electronic display 80 configured to display a graphical user interface to present the results of the analysis described herein. Furthermore, in some embodiments, the graphical user interface may present additional information to the operator of devices 72, 74. For example, the graphical user interface may include a dashboard configured to present visual information to the operator. In some embodiments, the dashboard may display live (e.g., real-time) data as well as the results of the analysis described herein. Additionally, in some embodiments, the analysis and control system 50 may include one or more input devices 82 configured to enable the operator to provide commands, for example, to devices 72, 74 described herein. Furthermore, in some embodiments, the display 80 may include a touchscreen interface configured to receive input from the operator.

[0025] It should be understood that Figure 4 The illustrated system 54 is merely exemplary, and system 54 may have more or fewer components than shown, and may be combined with Figure 4 Additional components not depicted in the implementation scheme, and / or system 54 may have Figure 4 The different configurations or arrangements of the components depicted. Furthermore, Figure 4 The various components illustrated can be implemented in hardware, software, or a combination of both (including one or more signal processing and / or application-specific integrated circuits). Furthermore, the operation of system 54 as described herein can be achieved by running one or more functional modules in an information processing device (such as a dedicated chip, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic device (PLD), a system-on-a-chip (SOC), or other suitable means). These modules, combinations of these modules, and / or their combinations with hardware are all included within the scope of the embodiments described herein.

[0026] In conventional systems, drill cuttings 46 are analyzed by a mud logger in a mud logging unit. These conventional systems typically rely on humans. The implementation described in this paper enhances the analysis of such drill cuttings 46. Figure 5 Examples are given of those that can be generated by Figure 4 The analysis and control system 50 executes the cuttings collection and image acquisition workflow 84. For example... Figure 5 As illustrated, the cuttings collection and image acquisition workflow 84 begins when the drilling system 10 produces rock particles (e.g., including drill bit cuttings 46 as described above). The rock particles are then collected, for example, from a mud vibrating screen 40, as per [the relevant documentation / reference]. Figure 2 As described (e.g., sample collection 86).

[0027] Then, in some embodiments, the rock particles can be prepared for analysis, for example, by drying them in an oven (e.g., sample preparation 88). Furthermore, in some embodiments, sample preparation 88 may include sieving the rock particles using one or more sieves 90 to select rock fragments 46 falling within a certain size range. In some embodiments, the size of the sieve 90 may range from 0.25 mm to 3.0 mm, and may be approximately 0.25 mm, approximately 0.50 mm, approximately 0.75 mm, approximately 1.0 mm, approximately 1.25 mm, approximately 1.50 mm, approximately 1.75 mm, approximately 2.0 mm, approximately 2.25 mm, approximately 2.50 mm, approximately 2.75 mm, or approximately 3.0 mm. It should be understood that the continuous sieve 90 for screening rock particles can start with a larger sieve 90, followed by sieves 90 that gradually decrease in size, such that larger rock fragments 46 are screened earlier and smaller rock fragments 46 are screened later, until the screened rock particles are so fine that they are no longer considered rock fragments 46. It should also be understood that when the rock fragments 46 are approximated as ellipsoids, the size of the rock fragment particles 46 is the smallest axis of the rock fragments 46. In some embodiments, sample preparation 88 may include placing the screened rock fragments 46 in a sample tray with a relatively bright background color (e.g., pure magenta (e.g., RGB values ​​255, 0, 255), pure blue (e.g., RGB values ​​0, 0, 255), pure green (e.g., RGB values ​​0, 255, 0), etc.). Generally, such colors do not exist in nature and therefore help the instance segmentation model avoid detecting the background of the sample tray as part of an instance. In some cases, the sample tray may be prepared by humans or an automated system, so the distribution of the rock fragments 46 is usually random. For example, rock chips 46 may touch or accumulate in some areas of the sample tray, while they may be sparsely distributed in other areas.

[0028] In some embodiments, an image 94 of the rock debris 46 may be captured by an imaging device (e.g., one or more cameras 52) (e.g., image acquisition 96). The following example is given in the case of a camera that detects the visible spectrum, but the same method can also be applied to images captured using an IR or UV camera that detects light in the ultraviolet (UV) or infrared (IR) domain. As described in more detail herein, in some embodiments, during image acquisition 96, the illuminance, color, and resolution of image 94 may be calibrated and normalized to obtain quantitative and reliable measurements of pixel values ​​between images 94. For example, in some embodiments, color / illuminance calibration may be obtained by comparing previously analyzed image 94 with the currently interested image 94 using a colorimetric algorithm, while resolution calibration may be based on lens focal length, focal distance, and sensor size / resolution to compare the currently interested image 94 with the previously analyzed image 94. All of these parameters may vary, but the final image is “calibrated,” and the same object can be digitized with reasonably close values. Thus, pixel values ​​(e.g., image 94 may be a digital image) and size are considered as effective measurements of the particles, not just representations. The implementation described herein enables the generation of such calibrated and error-assessed input images 94. Without such calibration, the final object classification might vary due to the acquisition rather than any real-world differences. In some implementations, optical images of the sample can be taken in visible light. In some implementations, optical images of the same sample can be taken in UV light, which may result in fluorescence of some minerals and oils (if the oil is trapped in rock fragments). This could be a clue to the presence of oil in certain rock types or geological formations to which the rock fragments belong. In some implementations, optical images of the same sample taken in different spectra (e.g., visible, UV, IR) can be combined for image analysis of the sample, such as... Figure 6 and Figure 8 As described.

[0029] Then, in some embodiments, the image 94 of the cuttings 46 can be compressed (e.g., a digital image) to facilitate transmission (e.g., image compression 98). Specifically, well sites are often located in relatively remote locations where available network bandwidth may be relatively slow. Therefore, compressing the image 94 of the cuttings 46 can facilitate the transmission of the image 94. It should be understood that if there is higher bandwidth at the well site (e.g., when the well site has wired internet access), compressing the image 94 of the cuttings 46 may be less beneficial. Then, in some embodiments, the image 94 of the cuttings 46 can be transmitted (e.g., image transmission 100), for example, transmitted to... Figure 4The illustrated analysis and control system 50 and / or computing system 78 are provided for analysis, as described in more detail herein. The analysis and control system 50 may be located at the well site or in a remote location. Then, in some embodiments, the analysis of the image 94 of the cuttings 46 (e.g., image analysis 102) may include extracting geologically meaningful information related to the cuttings 46 from the image 94, as described in more detail herein.

[0030] The implementation scheme described herein is based on the definition of lithological geology to determine measurements related to the lithology of the stratum 24 from which the rock cuttings 46 originated, based on image 94 of rock cuttings 46. Generally, the lithology of a rock unit is a description of its physical properties visible in outcrops, handheld or core samples, or under a low-power microscope. Such physical properties include, but are not limited to, color, texture, grain size, and composition. Lithology can refer to a detailed description of these physical properties or an overview of the overall physical properties of the rock. Examples of lithology types may include sandstone, slate, basalt, limestone, etc. Therefore, color, texture, and grain size are physical properties of lithology types, and the workflow described herein illustrates how these physical properties can be measured, extracted, and integrated to obtain automated lithology image identification.

[0031] The implementation scheme described herein is applicable to the analysis of calibrated images 94 of prepared rock cuttings 46, regardless of the purpose for which it is used to achieve [the desired outcome]. Figure 5 How do the specific methods described in sample preparation 88, image acquisition 96, image compression 98, and image transmission 100 work?

[0032] In some embodiments, the analysis and control system 50 may be configured to separate the pixels of individual rock particles from multiple particles (e.g., cuttings 46) depicted in image 94 (e.g., a calibrated image). Furthermore, in some embodiments, the analysis and control system 50 is configured to determine associated morphological (e.g., size, shape, etc.), color, and texture data from one or more individual rock particles (e.g., cuttings 46) depicted in the calibrated image 94. Additionally, in some embodiments, the analysis and control system 50 may be configured to utilize information extracted from the calibrated image 94 to perform geological / lithological classification at multiple different hierarchical levels (e.g., at the individual particle / cuttings level, at the individual sample level of multiple cuttings, at specific depth intervals within borehole 22, for a specific geological formation 24, for the entire well, for the entire well site, etc.). As described in more detail herein, integrating analysis results at multiple hierarchical levels enables the operator to analyze cuttings 46 in a more robust (and automated and more accurate) manner than conventional techniques. Furthermore, in some embodiments, the analysis and control system 50 may be configured to create a mud logging report using information derived herein from the calibrated image 94. Furthermore, in some embodiments, the analysis and control system 50 may be configured to output any relevant information from the calibrated image 94 that can be integrated with other well-related answer products. Furthermore, in some embodiments, the analysis and control system 50 may be configured to infer lithology from the calibrated image 94 from the current well using a supervised machine learning model (e.g., from another well in the same oilfield or another oilfield with a similar geological environment). Furthermore, in some embodiments, the analysis and control system 50 may be configured to automatically adjust the operating parameters of the drilling system 10 (e.g., drilling fluid composition, bit speed, etc.) using all such types of information, as described in more detail herein.

[0033] Although Figure 5 The illustrated workflow 84 for cuttings collection and image acquisition has been described with particular reference to analyzing cuttings 46 to determine geologically meaningful information associated with cuttings 46 from image 94. However, the implementation described herein can be extended to analyzing image 94 to identify features of other types of objects, not just cuttings 46.

[0034] In some implementations, object-based image analysis (OBIA) can be used to analyze image 94 (e.g., a calibrated image). OBIA typically involves grouping multiple pixels in a digital image into shapes with meaningful representations of objects based on spectral or spatial similarity or external variables (e.g., soil or geological units). OBIA can be used to solve complex categories defined by spatial and hierarchical relationships within and during a classification process. In some implementations, color and texture analysis can be used to analyze image 94. In geology, rocks are characterized by their texture and color. Therefore, a key step is to extract color and texture features from image 94 for use in various downstream image analysis tasks, such as clustering, image retrieval, classification, and manifold representation, thereby visualizing rock grain distribution to achieve OBIA. Analysis of image 94 may include image segmentation, feature extraction, and classification, such as... Figure 6 As illustrated above, the calibrated image 94 can be used for image analysis to obtain more reliable results. The implementation described herein focuses on feature extraction and classification analysis of the calibrated image 94. Some implementations achieve lithological identification and / or classification from the image 94 of rock cuttings 46 by applying neural network (NN) and machine learning (ML) techniques. It should be noted that the feature extraction techniques described herein can be applied to images from other images (e.g., core images, borehole images) and / or images from any field (such as remote sensing and medical imaging).

[0035] Figure 6 An example workflow 110 is illustrated for image analysis of image 94 (e.g., a calibrated image) by system 54. Image 94 (e.g., a digital image) can be input into the cuttings instance segmentation module 112 (e.g., a tool or instrument) of system 54 to obtain segmented individual cuttings instance images 114 (in... Figure 7 (As shown in the image). The rock debris instance segmentation module 112 can identify and separate individual objects within the calibrated image 94, for example, by detecting the boundaries of each object and assigning a unique label to each object. Figure 7 and Figure 8 exemplified. Figure 7An example of a calibrated image 94 and a corresponding segmented individual rock cuttings instance image 114 is shown. In some embodiments, the rock cuttings instance segmentation module 112 may include analysis hardware (e.g., analysis circuitry) and / or software including processor-executable instructions and associated data. In some embodiments, the rock cuttings instance segmentation module 112 may utilize Mask R-CNN (region-based convolutional neural network) to identify various objects in the calibrated image 94, which has been found to outperform conventional segmentation methods. The Mask R-CNN model is trained in a supervised manner, and it has been found that the same approach can be taken to efficiently build a system to perform instance segmentation using any type of image (e.g., digital images, distance images, etc.) and rock particle scenes (e.g., rock cuttings, mining sites, space, etc.) by training the model with relevant datasets.

[0036] The segmented individual rock fragment images 114 can be used in the rock property estimation process 116 and the rock property measurement process 118. For example, in the rock property estimation process 116, the segmented individual rock fragment images 114 can be input into the feature extraction module 120 (e.g., a tool or instrument) of the analysis and control system 50. In some embodiments, the feature extraction module 120 may include analysis hardware (e.g., analysis circuitry) and / or software including processor-executable instructions and associated data. The feature extraction module 120 can analyze the segmented individual rock fragment images 114 using reference data 122 (e.g., baseline rock fragments for reference) and output a set of parameters (features) representing the color and texture properties of the rock fragments, which may be key visual properties for identifying rock types, such as... Figure 8 As illustrated, rock debris features are therefore a set of values ​​that represent or summarize the color and texture information of an image of rock debris instances. Figure 8 An example of segmented individual rock debris instance images 114 is shown, wherein the individual rock debris instance images (e.g., rock debris A and rock debris B) are segmented using D features (e.g., feature 1, feature 2... feature D). D The corresponding parameter set indicator. For example... Figure 8 As illustrated, individual rock debris instance images (e.g., rock debris A, rock debris B) in the segmented individual rock debris instance images 114 can have arbitrary size, shape (e.g., height (H), width (W)), and color. The feature extraction module 120 can analyze the segmented individual rock debris instance images 114, and each rock debris instance image (Ii) can be extracted from the image space I. i R (Hi×Wi×3) (For example, height (Hi), width (Wi), and three color channels (e.g., red, green, and blue)) are transformed into a D-dimensional feature vector space. In some implementations, the rock cuttings instance image (Ii) can be a grayscale image and has only one color channel. The D-dimensional feature vector space R... D It can include D features (e.g., feature 1, feature 2, ... features). D ) axis, and eigenvector It can be along the eigenvector space R D The corresponding feature axes in the image space have a set of parameters. In some implementations, different types of images representing different physical properties of the same rock particle sample can be obtained without changing the position of the sample in the sample tray. In such cases, we can combine those images and generate a single multi-channel image. Therefore, the transformation is from the image space I... i R (Hi×Wi×C) (Where C is the number of channels (e.g., physical properties of rock particle samples, such as color)) to a D-dimensional feature vector space. For example, in some embodiments, the fluorescence of minerals in rock cuttings may be a clue to certain rock types, and an optical image of the same sample (e.g., rock cutting 46) under UV light (which may result in mineral fluorescence) can be combined with a calibrated image 94 (e.g., obtained using visible light) to produce a 6-channel image (e.g., 3 color channels of an image taken using UV light and 3 color channels of an image taken using visible light). In some embodiments, the calibrated image 94 may include hyperspectral images (e.g., continuous wavelength range) and / or multispectral images (e.g., discrete target wavelengths) of rock cuttings taken using imaging spectroscopy techniques. These hyperspectral and / or multispectral images may have more than 3 color channels and may include spectral features of minerals in the rock cuttings that can be used to identify the minerals. In this case, the transformation is from image space I i R (Hi×Wi×6) To D-dimensional feature vector space The feature vectors of the segmented individual rock fragment images 114 can be used by the lithology classification module 124 (e.g., a tool or instrument) of the analysis and control system 50 to determine the lithology 126 of the rock fragment 46, as described in more detail herein. In some embodiments, the lithology classification module 124 may include analysis hardware (e.g., analysis circuitry) and / or software including processor-executable instructions and associated data. For a certain type of rock (e.g., sandstone), the grain size analysis module 128 (e.g., a tool or instrument) of the analysis and control system 50 can be used to analyze each individual rock fragment 46 to identify the grain size category (e.g., fine, medium, coarse, etc.) 130 and grain size distribution 132 of each individual rock fragment 46, as well as other grain size-related properties. In some embodiments, the grain size analysis module 128 may include analysis hardware (e.g., analysis circuitry) and / or software including processor-executable instructions and associated data.

[0037] The feature vectors of segmented individual cuttings instance images 114 can be used with other properties 134 (e.g., properties not easily extracted from digital images) to estimate rock properties, as described in more detail herein. For example, other properties 134 may include guiding artificial property descriptions, associated well logging values, or other external data such as, but not limited to, formation geological sequences, logging-while-drilling records, 3D model properties, cuttings physical analysis (e.g., diffraction, carbonate determination, acid testing), or any other information that may be useful in resolving lithological classification ambiguities. In some embodiments, other properties 134 can be determined using instruments / tools (e.g., surface equipment 72, downhole equipment 74), analytical hardware (e.g., analytical circuitry), and / or software including processor-executable instructions and associated data.

[0038] like Figure 8As illustrated, individual rock fragment images (e.g., rock fragment A, rock fragment B) in the segmented individual rock fragment image 114 can have arbitrary size, shape, and color. The property measurement process 118 can use the shape measurement module 136 of the analysis and control system 50 (e.g., a tool or instrument), and the shape measurement module 136 can analyze the segmented individual rock fragment image 114 based on individual instances to determine the size 138 and shape 140 of each individual instance. The property measurement process 118 can use the texture measurement module 142 of the analysis and control system 50 (e.g., a tool or instrument), and the texture measurement module 142 can analyze the segmented individual rock fragment image 114 based on individual instances to determine the texture 144 of each individual instance, such as homogeneous, heterogeneous, layered, granular, etc. The property measurement process 118 can also use the color measurement module 146 of the analysis and control system 50 (e.g., a tool or instrument), and the color measurement module 146 can analyze the segmented individual rock fragment image 114 based on individual instances to determine the color 148 of each individual instance. In some embodiments, the shape measurement module 136, texture measurement module 142, and color measurement module 146 may include analysis hardware (e.g., analysis circuitry) and / or software including processor-executable instructions and associated data. The outputs of the property estimation process 116 (e.g., lithology 126, grain size category 130, grain size distribution 132) and the outputs of the property measurement process 118 may be combined to obtain an integrated result 150.

[0039] The property estimation process 116 may include two stages: a first stage for reference data description and indexing (e.g., for baseline rock cuttings used for reference); and a second stage for property estimation of rock cuttings instance images based on the reference data description and indexing obtained in the first stage. Figure 9 The diagram illustrates a workflow 200 for the first stage of data description and indexing using reference data 122. In some embodiments, workflow 200 may include instruments / tools (e.g., surface equipment 72, downhole equipment 74), hardware (e.g., analysis circuitry), and / or software including processor-executable instructions and associated data. Reference data 122 may include image data of a set of M images (e.g., calibrated image 94) obtained when analyzing previous data from previous wells or the current well.

[0040] Workflow 200 may include instance segmentation (process 1), image feature extraction (process 2), property association (process 3), and indexing (process 4). In process 1 of workflow 200, each image in a set of M images may be analyzed (e.g., by the cuttings instance segmentation module 112) to obtain a number (e.g., N) of individual cuttings instance images 202 (e.g., I1, I2, ... I...). NThe corresponding segmented individual rock debris instance images 202 (e.g., I1, I2, ... I...). N The rock cuttings in the image can be used as reference rock cuttings or baseline rock cuttings. In process 2 of workflow 200, segmented individual rock cutting instance images can be analyzed (e.g., by feature extraction module 120) to obtain images 202 (e.g., I1, I2, ... I...) for individual rock cutting instance images. N The number of features (e.g., feature 1, feature 2, ... features) is D. D The corresponding parameter set of ). Therefore, individual rock cuttings instance images 202 (e.g., I1, I2, ... I N It can be transformed from image space to D-dimensional feature vector space R. D The corresponding eigenvector 204 in the matrix (e.g., from the corresponding matrices y1, y2, ... y N express).

[0041] In process 3 of workflow 200, individual rock cuttings instance images 202 (e.g., I1, I2, ... I...) N Artificial descriptions can be performed by geologists (e.g., 3-1 guided artificial property descriptions), guided by rock fragment description software. Figure 12 and Figure 13 A detailed description is provided. Cuttings description software may include a description wizard with multiple entries, such as lithology, hardness, and visual porosity. Some entries in the description wizard may not be easily extracted from digital images. In process 3 of workflow 200, logging values ​​can also be used for property description (e.g., 3-2 logging value correlation). Logging records may include detailed records of geological formations (e.g., formation 24). Detailed records may include information on the geological properties of the geological formations (e.g., lithology, layer, sedimentary environment) and rock physical characteristics (e.g., water saturation, porosity, permeability, shale volume), which can be used to control the drilling plan of the drilling rig or drilling system 10. Logging records may be a function of borehole depth. For example, cuttings 46 may be collected at a depth in borehole 22, and logging values ​​at that depth may be correlated with cuttings from the same depth from the same well or different wells, such as... Figure 14 As illustrated. For example, this method can be used when the reference data in the first stage includes wells with cuttings and logging measurements, while the cuttings instance images in the second stage include another well with cuttings but no logging measurements. The corresponding individual cuttings instance images 202 (e.g., I1, I2, ... I...) N Property 206 (e.g., P1, P2, ... P) NThe lithological classification can be determined using guiding artificial property descriptions, associated logging values, or other external data, such as, but not limited to, formation geological sequences, logging-while-drilling records, 3D model properties, cuttings physical analysis (e.g., diffraction, carbonate determination, acid testing), or any other information that may be useful in resolving ambiguities in lithological classification. Property 206 may include the properties of reference cuttings in a single cuttings instance image 202, such as lithology type, hardness, visual porosity, gamma ray logging, porosity logging, etc. For example, property P1 may include a description of "lithology type: limestone; hardness: 0.6, visual porosity: 1%, gamma ray logging: 30 API; porosity logging: 3%..." P2 may include a description of "lithology type: sandstone; hardness: 0.6, visual porosity: 5%, gamma ray logging: 30 API; porosity logging: 3%..." N This may include descriptions such as "lithology type: sandstone; hardness: 0.7, visual porosity: 2%, gamma ray logging: 30 API; porosity logging: 3%...". Based on feature similarity, the Nth reference cuttings (P... N The property descriptions of the rock fragments can be propagated to other rock fragments in the same segmented individual rock fragment image, and the view 207 of the property descriptions of the rock fragments (e.g., a two-dimensional (2D) view, a three-dimensional (3D) view) can be displayed on the user interface.

[0042] In process 4 of workflow 200, rock cuttings instance images (e.g., I1, I2, ... I) are generated. N ), eigenvector 204 (for example, derived from the corresponding matrices y1, y2, ... y N The representations (i) and corresponding properties 206 can be indexed together and stored in an index dataset 208 (e.g., index A). The index dataset 208 may contain instance images of rock cuttings (e.g., I1, I2, ... I1) for a set of M images in the reference data 122. N ), eigenvector 204 (for example, derived from the corresponding matrices y1, y2, ... y N (represented) and corresponding properties 206 (e.g., P1, P2, ... P) N In process 4, clustering techniques can be applied to the index dataset 208 to reduce the index data size. Clustering techniques can be data compression techniques or redundancy reduction techniques. For example, when multiple rock debris instances are visually very similar, and the feature vector space R... DWhen corresponding feature distances (e.g., squared Euclidean (L2-squared) distances) in the images are very close to each other, debris instances can be grouped into a cluster, and only representative debris instances, which can be the centroid debris instance of the cluster, can be retained. Multiple index databases can be created using different collections of M images. For example, various index datasets (e.g., index A, index B, index C) can be created based on images of specific geographic locations or specific geological strata.

[0043] Figure 9 Workflow 210 is also illustrated for property estimation (e.g., lithology, hardness, visual porosity, gamma-ray logging, porosity logging) of cuttings instance images using properties 206 and index dataset 208 obtained in workflow 200. In some embodiments, workflow 210 may include instruments / tools (e.g., surface equipment 72, downhole equipment 74), hardware (e.g., analysis circuitry), and / or software including processor-executable instructions and associated data. Workflow 210 may include instance segmentation (process 1), image feature extraction (process 2), querying (process 3), and property propagation (process 4). In process 1 of workflow 210, images (e.g., calibrated images 94) may be analyzed (e.g., by the cuttings instance segmentation module 112) to obtain a number (e.g., J) of unknown individual cuttings instance images 212 (e.g., I1, I2, ... I...). J (e.g., unknown lithology, unknown hardness, unknown visual porosity, unknown gamma ray logging, unknown porosity logging) segmented individual cuttings instance images. In process 2 of workflow 210, the segmented individual cuttings instance images can be analyzed (e.g., by feature extraction module 120) to obtain unknown individual cuttings instance images 212 (e.g., I1, I2, ... I J Each of the D features in the ) has D features (e.g., feature 1, feature 2... feature ) D The parameter set of ). Therefore, the unknown individual rock debris instance images 212 (e.g., I1, I2, ... I J Each of the elements in the image space can be transformed from the image space to the D-dimensional feature vector space R. D The corresponding eigenvector 214 in the matrix (e.g., from the corresponding matrix x) J (This is indicated). For a newly acquired image, processes 1 and 2 of workflow 210 can be repeated.

[0044] Process 3 of workflow 210 can be used for images 212 of unknown individual rock cuttings instances (e.g., I1, I2, ... I J Each of them is obtained by transforming the eigenvector 214 (e.g., by the corresponding matrix x) into eigenvectors 214. J(represented) and the eigenvectors stored by index (e.g., by corresponding matrices y1, y2, ... y... N The method involves comparing K number of most similar (K ≥ 1) rock cuttings from an indexed store (e.g., index A, index B, index C) to retrieve K number of most similar (K ≥ 1) rock cuttings. As described above, the properties (e.g., lithology, hardness, visual porosity, gamma-ray logging, porosity logging) of the K number of most similar rock cuttings can be obtained from the property association in the first stage. The computable eigenvector space R... D In the eigenvector 214 (e.g., from the corresponding matrix x) J (represented) and the eigenvectors stored by index (e.g., by corresponding matrices y1, y2, ... y... N The distance between (e.g., squared Euclidean (L2-squared) distance) can be represented, and the nearest neighbor search method can be used to determine the K number of most similar rock fragments by finding the K number of minimum distances.216

[0045] In process 4 of workflow 210, images 212 of unknown individual rock fragment instances (e.g., I1, I2, ... I1) can be targeted using K most similar rock particles 216. J Each of these determines a corresponding property description 218 (e.g., lithology type, hardness, visual porosity, gamma-ray logging, porosity logging). For example, an indexed dataset in an indexed store might be selected for better querying and therefore more accurate property estimation. For example, in Figure 9 In the process, index dataset A can be selected from other options such as index dataset B and index dataset C based on geographical location, geological strata, well location, well depth, etc. When K equals 1, the properties of the retrieved cuttings 216 can be simply copied to the unknown individual cuttings instance image 212 being analyzed, such as... Figure 9 As illustrated, for example, when K equals 1, the most similar rock grains in the index might have a property description as "lithology type: sandstone; hardness: 0.6, visual porosity: 5%, gamma-ray logging: 30 API; porosity logging: 4%...", and the unknown individual rock cuttings instance image 212 being analyzed could be identified as having the same property description. When K is greater than 1, the properties of the retrieved rock cuttings 216 can be summarized and copied to the unknown individual rock cuttings instance image 212 being analyzed using various methods. For example, for categorical properties (such as lithology category), a probability can be calculated for each category, and the most likely category can be selected.

[0046] In some implementations, texture features can be extracted from rock fragment instance images and used to analyze rock grains. Texture generally refers to the spatial organization of a set of basic elements or primitives (e.g., texture primitives) that form the fundamental microstructure of an image and are the basic units of pre-attentional human visual perception. Due to the nature of texture, extracting texture features from rock fragment instance images may differ from extracting other features (e.g., color) from them. For example, while color may remain unchanged during image rescaling, a rescaled version of a rock fragment instance image may correspond to different texture features that can be used to distinguish different types of rocks (e.g., fine sandstone and medium sandstone). Furthermore, texture may remain unchanged during image (e.g., rock fragment instance image) rotation and translation, so a rotated or flattened version of the image may be given the same or very similar texture features as the original image.

[0047] Existing feature extraction techniques may be limited in extracting texture features from rock debris instance images due to the nature of texture (such as spatial information captured in features, feature size depending on input size, shape similarity being affected by the presence of background, and blurred reconstructed images caused by pixel-based reconstruction loss). For example, in some embodiments, a segmented individual rock debris instance image 212 may contain background pixels of the background (e.g., a sample tray) used during image acquisition 96 (when the image of rock debris 46 was captured). For example, the rock particles in rock debris 46 may have arbitrary sizes and shapes; therefore, the segmented individual rock debris instance image 212 (e.g., I...) i R (Hi×Wi×3) The images of rock particles in the image can have arbitrary sizes and shapes, such that a portion (e.g., a corner) of the segmented individual rock fragment instance image 212 is filled with background pixels, such as... Figure 10 As illustrated. Background pixels do not represent any features of rock grains, and in the segmentation of individual rock debris instances image 212 (Ii) from image space I i R (Hi×Wi×3) (For example, height (Hi), width (Wi), and three color channels (e.g., red, green, and blue)) are transformed into a D-dimensional feature vector space. Pixels that are not properly segmented are considered invalid. In some implementations, background pixels may be cropped, and features of rock grains may be extracted using only the centers of segmented individual rock debris instances. However, some segmented individual rock debris instances may include a global texture representation, and using only the centers of segmented individual rock debris instances may result in the loss of this global texture representation. In some implementations, some segmented individual rock debris instances may be resized before being input into the feature extraction module to ensure uniform input image size; however, resizing may distort the geometry and scale of the texture.

[0048] This paper describes systems and methods for extracting features (e.g., color, texture features) from images of arbitrary shapes and sizes containing invalid pixels (e.g., background, image errors, other objects). It should be noted that the feature extraction techniques described herein can be applied to images from other images (e.g., core images, borehole images) and / or images from any domain (such as remote sensing and medical imaging). In some embodiments, an autoencoder (AE), which may be data-driven and / or unsupervised (e.g., in feature extraction module 120), can be used to extract texture features from the image. In some embodiments, a variational autoencoder (VAE) (e.g., in feature extraction module 120) can be used to control the distribution of the latent space. In some embodiments, a plain AE can be used instead of a VAE. A global average pooling (GAP) layer can be used at the last layer of the encoder to make the features rotation- and translation-invariant and scale-equivariant. Furthermore, partial convolutions can be used in the encoder to logically ignore invalid pixels (e.g., background, image errors, other objects) in the image.

[0049] An autoencoder is a general framework consisting of two parts. The first part is the encoder, which takes the input image I... i R (Hi ×Wi×3) Encoded as a latent vector Z i ~encoder(I) i )=q(Z i |I i ),like Figure 10 As illustrated. Potential vector Z i Also known as features or encoding. The second part is the decoder, which takes the latent vector Z... i Decode into image Decoder (Z) i )=p(I i |Z i Both the encoder and decoder are convolutional neural networks (CNNs), and the network parameters (e.g., weights, biases) can be defined through training, such as... Figure 11As illustrated, a Convolutional Neural Network (CNN) may include an input layer, one or more hidden layers, and an output layer. The basic computational unit in a neural network is a neuron / node. Each neuron / node receives input from some other node or external source and computes an output. The input layer may include neurons / nodes to receive external input, such as input data to the CNN. Each hidden layer consists of a set of neurons / nodes with learnable weights and biases, and each neuron / node in the hidden layer receives some input and performs a dot product. The output layer may include neurons / nodes to receive input from the hidden layers and output the result of the CNN. CNNs use a feedforward artificial network of neurons to perform image recognition or identification. It uses a backfeed system to learn and produces a set of weights to calibrate the execution system. CNNs may include multiple layers in the hidden layers, such as convolutional layers, pooling layers, and activation layers. Convolutional layers extract low-level features from the input data (e.g., lines or edges within an image) (e.g., operating in a sliding window manner), and pooling layers reduce variation (e.g., pooling common features on specific regions of an image by maximizing or averaging values). The results can be passed to further convolutional and pooling layers. Activation layers can apply activation functions (e.g., Rectified Linear Unit (ReLU), sigmoid, tanh) to the received results (e.g., from pooling layers). The autoencoder can be trained in an unsupervised manner (e.g., without using labeled data). Once the model is trained, the encoder part can be used to extract features from the image. The encoder and decoder can be created based on a Residual Neural Network (ResNet) or a similar architecture. The decoder architecture can be symmetric or asymmetric with respect to the encoder architecture.

[0050] Figure 10 An example 300 illustrates a process for implementing an encoder to transform a segmented image of individual rock debris instances with invalid background pixels from image space to latent vector space. For example... Figure 10 As illustrated, segmented individual rock debris instance image 302 (e.g., I) i R (Hi×Wi×3) The images of rock particles in the image can have arbitrary size and shape, such that a portion (e.g., a corner) of the segmented individual rock fragment instance image 302 is filled with background pixels. The background pixels do not represent any feature of the rock particles and are transferred from the image space I by the encoder 304. i R (Hi×Wi×3) (For example, height (Hi), width (Wi), and 3 color channels (e.g., red, green, blue)) are transformed into an N-dimensional representation space 306 (e.g., Z). i R NPixels that are not present in the segmented individual rock debris instance image 302 are considered invalid. Encoder 304 can be a partial texture VAE encoder that logically ignores background pixels in the segmented individual rock debris instance image 302, so that the extracted features in the N-dimensional representation space 306 are unaffected by background pixels in the segmented individual rock debris instance image 302. For example... Figure 11 As illustrated, encoder 304 may include a texture coding layer (TEL) and use partial convolution in the convolutional layer to be insensitive to geometric arrangement, image size and shape, and background pixels.

[0051] Figure 11 A process 400 for image analysis using an implementation of a partially textured VAE is illustrated, wherein the partially textured VAE includes a mask-aware texture coding layer (TEL) in the encoder, and partially convolutions are used in the hidden layers (e.g., convolutional layers, pooling layers) of the encoder of the partially textured VAE. Figure 11 As illustrated, an input image 402 (e.g., 64x64x3, indicating height (H) = 64, width (W) = 64, 3 color channels (e.g., red, green, blue)) can be provided together with a corresponding mask image 402M (e.g., 64x64x1) to the input layer of the VAE's encoder. The mask image 402M is a digital image with binary values, where the masked areas have one value and the unmasked areas have another value (e.g., ...). {0, 1} H×W The values ​​in the masked regions are "0", while the values ​​in the non-masked regions are "1". Input image 402 and mask image 402M can be used as input and processed using partial convolution through some hidden layers (e.g., convolutional layers, pooling layers) of the VAE encoder to generate feature maps of input image 402. For example, in... Figure 11 In the illustrated implementation, a feature map 404 (e.g., 32x32x8) can be generated by performing partial convolution using the input image 402 and the mask image 402M, and the mask image 402M can be updated to an updated mask image 404M based on the size of the feature map 404 (e.g., 32x32x1). The feature map 404 and the updated mask image 404M can be further processed (e.g., via pooling layers) to generate a feature map 406 (e.g., 16x16x32) and a corresponding updated mask image 406M, which can be used to generate a feature map 408 (e.g., 8x8x64) and a corresponding updated mask image 408M (e.g., 8x8x1). It should be noted that although in Figure 11The illustrated implementation uses three layers to process the input image 402 and the mask image 402M, but other implementations may use other numbers of layers (e.g., 2, 4, 5). During partial convolution, only the non-masked regions in the mask image can contribute to the convolution result, as illustrated in the following equation [1]. Equation [1] describes the application of a filter function (pooling can also be viewed as convolution using a special filter (e.g., average or maximum)) to the input image or feature map F. R (H×W×C) (For example, H is the height, W is the width, and C is the number of channels) One channel is partially convolved to generate the corresponding next-level feature map Z (for example, H'×W'×C', and H', W', and C' may differ from H, W, and C depending on the filter function): in W It is a matrix that includes the weights of the convolutional filters, and W T yes W transpose, It is the bias of the convolution filter. Indicates F A sliding convolution window centered on the center, Indicates M with ( The binary mask corresponding to the current sliding convolution window centered at ) This represents element-wise multiplication, and 1 has the same properties as... Same shape, but all elements are 1. Scaling factor sum(1) / sum( Apply appropriate scaling to adjust for variations in valid (non-masked) input. After each partial convolution operation, the mask can be updated as follows: if the convolution conditioned its output with at least one valid input value, the position is marked as valid, expressed as: In Equation [1], the size HxW of the mask image M (e.g., 64x64, 32x32, 8x8) is the same as the size of the feature map or input image F in one channel, where the value is “0” in the masked region and “1” in the non-masked region. As illustrated in Equation [1], the masked region in the mask image M corresponds to Z. h’,w’,c’ No contribution, because M h,w The value in the mask region is "0". Therefore, a mask region of the mask image M can be selected to cover invalid pixels in the input image (e.g., background, image errors, other objects) so that invalid pixels are logically ignored when using a partial texture VAE, such as... Figure 14 exemplified.

[0052] The mask-aware texture coding layer (TEL) 410 in a partially textured VAE may include a learnable residual coding layer, which may include a codebook 412 (a dictionary, e.g., 64x32x1). A feature map 408 may be reshaped into a feature map 414 (e.g., 64x(8x8)x1), which can be used as input to the residual coding layer. A residual vector representation 416 (e.g., 32x64x(8x8)) can be computed using pairwise differences between the feature map 414 and the codes in the codebook 412 (e.g., 32 codes). Weights 418 (e.g., 32x(8x8)x1) may be computed based on pairwise distances between the codes in the feature map 414 and the codebook 412. The mask image 408M can be reshaped into a mask array 420M (e.g., 1x(8x8)) and used, along with assigned weights 418, to aggregate with the residual vector representation 416 to generate a feature aggregation map 422 (e.g., 32x64x(8x8)). For example, mask regions 421M in the mask array 420M can be transformed into slices 421 (e.g., 32x64x1) in the feature aggregation map 422. The feature aggregation map 422 can be used to generate a feature map 424 (e.g., 32x64x1), which can be reshaped into a fixed-length representation 426 (e.g., 64x32x1), where the resulting vectors can be normalized using L2 normalization. Therefore, TEL 410 acts as a feature aggregation module. There are several advantages to integrating this module. For example, the TEL 410 can accept images of any input size and provide output in a fixed-length representation, which can be a spatially unordered representation, thereby making the features translation-invariant, scale-isovariant, and rotation-invariant.

[0053] A fixed-length representation 426 can be fed into a fully connected (FC) layer to generate an updated image 428. The FC layer multiplies the input by a weight matrix and then adds a bias vector. Because it can introduce more connection possibilities, the FC layer can be used to update the weights, as each neuron in the FC is connected to every neuron in the previous layer. The updated image 428 can be used to generate a feature map 430 via reparameterization. The feature map 430 can then be decoded by a decoder of a partially textured VAE to generate a feature map 432 (e.g., 8x8x64), which can be used to generate a feature map 434 (e.g., 16x16x32). The feature map 434 can be used to generate a feature map 436 (e.g., 32x32x8), which can be used to generate an output image 438 (e.g., 64x64x3). The decoder can integrate upsampling blocks followed by convolutional layers instead of deconvolutional layers (e.g., to avoid checkerboard effects in the generated image).

[0054] Figure 12An example of image analysis process 500 using another implementation of a partially textured VAE is illustrated, in which the partially textured VAE uses mask-aware global average pooling (GAP) 510 instead of mask-aware TEL 410 in the encoder, and uses partial convolution in the convolutional layers of the encoder of the partially textured VAE. Figure 12 As illustrated, the last feature map 408 generated by the encoder of the partial texture VAE and the last updated mask image 408M can be fed into a mask-aware global average pooling (GAP) 510. During the average pooling operation in GAP 510, only the non-masked regions in the mask image can contribute to the result, as illustrated in the following equation [2]. Equation [2] describes the input of the last feature map F R (H×W×C) (For example, feature map 408) undergoes mask-aware global average pooling to generate a global feature tensor 526: Where i corresponds to c, and M h,w M represents the last mask image. {0, 1} H×W (For example, the mask image 408M) has values ​​at positions with coordinates (h, w) (e.g., height h = 1, 2…H, width w = 1, 2…W), and F h,w,c This represents the last feature map F. R (H×W×C) The value at the location with coordinates (h, w, c). In Equation [2], the size HxW of the mask image M (e.g., 64x64, 32x32, 8x8) is the same as the size of the feature map or input image F in one channel, where the value is "0" in the masked region and "1" in the non-masked region. As illustrated in Equation [2], the masked region in the mask image M corresponds to Z i No contribution, because M h,w The value in the mask region is "0". Therefore, the mask region of the mask image M can be selected to cover invalid pixels in the input image (e.g., background, image errors, other objects) so that invalid pixels are logically ignored when using a partial texture VAE, and the pixel values ​​of invalid pixels do not contribute to the result of the partial texture VAE.

[0055] like Figure 12As illustrated, GAP 510 is used to convert feature map 408 into a one-dimensional (1D) global feature tensor 526 based on the average response magnitude of each channel of feature map 408. Since the mask image 408M has a value of "0" in the mask region, the mask region does not contribute to tensor 526. Furthermore, GAP 510 makes the feature rotation- and translation-invariant as well as scale-isovariant. These aspects of the feature are very important from a geological perspective. Therefore, rotated or translated versions of the image can be assigned the same or very similar features to the original image, while rescaled versions of the image can be assigned different feature values ​​(e.g., to distinguish between fine sandstone and medium sandstone).

[0056] A global feature tensor 526 can be input into a fully connected (FC) layer, which acts as a classifier to generate an updated image 528. The FC layer multiplies the input by a weight matrix and then adds a bias vector. Because it can introduce more connection possibilities, the FC layer can be used to update the weights, as each neuron in the FC is connected to every neuron in the previous layer. The updated image 528 can be used to generate a feature map 530 via reparameterization. The feature map 530 can then be decoded by a decoder of a partially textured VAE. For example, feature map 530 can be decoded to generate feature map 532 (e.g., 8x8x64), which can be used to generate feature map 534 (e.g., 16x16x32). Feature map 534 can be used to generate feature map 536 (e.g., 32x32x8), which can be used to generate output image 538 (e.g., 64x64x3). The decoder can integrate upsampling blocks followed by convolutional layers instead of deconvolutional layers (e.g., to avoid checkerboard effects in the generated image).

[0057] Figure 13 This is a block diagram illustrating the training process 600 for a partially textured VAE 602, which includes GAP layers, such as... Figure 12 A portion of the textured VAE in the image. Training process 600 can be used to update network parameters based on a loss function using training data. For example, a single rock debris instance image 604 (e.g., A set of images (e.g., images 604, where images 604, 602, 602, 604 ... ) can be transformed into a cropped image 606 (where ), where the cropped image size H 裁剪 and W 裁剪It is fixed to a specific value (e.g., 56, 64, 128). In some embodiments, the cropped image 606 may be a portion of the corresponding image 604 that does not contain invalid pixels (e.g., background, image errors, other objects), and the corresponding composite mask image 608 (e.g., ) can be created randomly (e.g., by adding N randomly selected elliptical spots), such as Figure 13 exemplified.

[0058] Cropped image 606 and mask image 608 The input 610 can be used for training the partial texture VAE 602. The partial texture VAE 602 may include an encoder 612, which may include hidden layers (e.g., convolutional layers, pooling layers) using partial convolution. The encoder 612 may include a GAP layer 614, and during the average pooling operation in the GAP 612, only the non-masked regions in the masked image 608 can contribute to the result, as illustrated in the above equation [2]. Features can be extracted from the cropped image 606 by the encoder 612 and corresponding feature maps 616 can be generated, which can then be decoded by the decoder 618 of the partial texture VAE 602 to generate the corresponding output image 620. Output image 620 can be related to input 610, as described in equation [3]: To extract texture information from the latent space, perceptual loss can be used in the loss function used to train the VAE 602 to better capture texture information (e.g., using mean squared error (MSE) or binary cross-entropy (BCE) might result in a blurred version of the input image). For example... It can be a network The activation of layer j when processing image I. Layer j can be a convolutional layer, therefore... It has shape The feature map, i.e. .network It can be a well-known architecture, such as VGG, AlexNet, and SqueezeNet, and can be pre-trained using large datasets such as ImageNet. To learn from the network... The layer j used can be chosen by the design. The perceptual loss using the j-th layer is defined in equation [4] as: In equation [4], there are different ways to define the function. This function is used to compute the loss from two feature maps, such as using pairwise L2 distance, L2 distance of the Gram matrix, or combining them with weight parameters. Cosine similarity or structural similarity (SSIM) can also be incorporated. The total reconstruction loss is the sum of the reconstruction losses from all target layers, as illustrated in equation [5]: .

[0059] In addition, Kullback–Leibler (KL) divergence loss can be used during the training process (e.g., The distribution of the latent vector z is controlled by an independent Gaussian random variable (e.g., a random variable with unit variance and zero mean). The final loss can be expressed as in equation [6]: in These are weighted parameters. The value can be chosen such that and Both decrease smoothly during training.

[0060] Figure 14 This is a block diagram illustrating a feature extraction process 700 that extracts features from an analysis image 702 using an encoder 612 after training the partial texture VAE 602. For example, an input image 704 may be selected from the analysis image 702. And the corresponding mask image can be... 706 is used as input 708 to be fed into encoder 612 to extract the D-dimensional feature vector included in feature map 710. The mask image 706 may be obtained from instance segmentation calculations (e.g., via the rock cutting instance segmentation module 112) or generated by examining specific pixel values ​​in image 704 (e.g., (255, 255, 255)), which may correspond to a background (e.g., a sample tray) used during image acquisition 96 (when the image of rock cutting 46 was captured). In some embodiments, the mask image 706 (e.g., dark areas corresponding to the foreground of image 704 and bright areas corresponding to the background) may be any shape corresponding to a corresponding portion of image 704 (e.g., background, image error, other object, a certain area), and the mask image 706 may be used in encoder 612 to logically ignore corresponding portions of image 704 using partial convolution operations (e.g., for image inpainting).

[0061] exist Figure 14In the illustrated implementation, since the GAP layer 614 has been trained, the input image may not be cropped. Due to the use of partial convolution as described above, invalid pixels in the input image 704 (e.g., background, image errors, other objects) are logically ignored. Feature vector It can be expressed by equation [7]: The extracted features can be used in various ways, such as classification, clustering, information retrieval, and visualization. Features can be used as input for lithological classification and similarity-based property extraction. Classification can be parametric (requiring training) or non-parametric (not requiring training). The extracted features can also be used as input for clustering of rock debris instance images. The most similar rock debris can be retrieved based on the distance between feature vectors. For example, the smaller the distance between two feature vectors of two images, the more similar the two images are. The dimensionality of the feature map can be modified (e.g., reduced) using techniques such as PCA and t-SNE, and the feature map can be plotted as a 2D or 3D graph. In visualization, visually similar rock debris can be placed in the same area. Visual distribution helps interpret rock type grouping. Figure 15 A visualization of rock fragments is shown 800. After reducing the dimensionality from the original D-dimensional feature space using projection techniques such as t-SNE (t-distributed random neighborhood embedding) or PCA (principal component analysis), an image of rock fragments projected into 2D or 3D space can be displayed. The colors and patterns used in the display 800 can be lithological patterns based on the user company's standards or based on USGS (United States Geological Survey) standards. The property description of the rock fragments can show the distribution of lithological types (e.g., dolomite 806, limestone 808, sandstone 810, mudstone 812, uncertain category 814, unknown category 816).

[0062] The techniques and systems disclosed herein relate to analyzing rock cuttings and measuring their physical lithological characteristics. Systems and methods are provided to extract features (e.g., color, texture features) from images of arbitrary shapes and sizes containing invalid pixels (e.g., background, image errors, other objects) by implementing global average pooling (GAP) and partial convolution in an autoencoder (AE). It should be noted that the feature extraction techniques described herein can be applied to images from other images (e.g., core images, borehole images) and / or images from any field (such as remote sensing and medical imaging). In some embodiments, an autoencoder (AE) that can be data-driven and / or unsupervised can be used to extract texture features from the image. In some embodiments, a variational autoencoder (VAE) can be used to control the distribution of the latent space. In some embodiments, a plain AE can be used instead of a VAE. A global average pooling (GAP) layer can be used at the last layer of the encoder to make the features rotation- and translation-invariant and scale-isovariant. Furthermore, partial convolution can be used in the encoder to logically ignore invalid pixels (e.g., background, image errors, other objects) in the image. It should be noted that in some embodiments, the systems and methods described herein can be used to exclude any pixel (e.g., in any region of the input image) from the feature extraction process for imaging processing purposes, such as image inpainting. An image analysis workflow is provided, comprising multiple computational modules to automatically extract relevant geological information from rock cuttings. Reference data, manual descriptions, and well logging values ​​are correlated and used to determine the rock properties of the rock cuttings. Software is developed for image analysis, and the results are displayed in various views. Based on the rock properties (e.g., lithology) of the rock cuttings, the results can be used to control related equipment, such as drilling system 10 and / or the drilling schedule of drilling system 10. For example, analysis and control system 50 can automatically adjust one or more operating parameters of drilling system 10 from which the rock cuttings are obtained, at least in part, based on the determined rock properties of the rock cuttings. The techniques and methods disclosed herein allow for the acquisition of high-quality well logging profiles for real-time and / or near-real-time geological formation assessment and geosteering, thereby enabling more efficient and accurate control of the drilling process. Although the examples described above are illustrated with respect to wellbores on land, similar methods can be applied to any acquisition configuration. Furthermore, in the embodiments illustrated above, the techniques and systems are primarily described in the context of digital images with three color channels (e.g., R, G, B). However, the techniques and systems described above can be applied to other types of digital images, such as grayscale images, hyperspectral images, multispectral images, etc.

[0063] Specific embodiments described above have been illustrated by way of example, and it should be understood that various modifications and alternatives are permissible. It should also be understood that the claims are not intended to limit them to the specific forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure.

[0064] In the claims, the means-plus-function clause is intended to cover structures described herein as performing the enumerated functions, encompassing not only structural equivalents but also equivalent structures. Thus, for example, although nails and screws may not be structural equivalents because nails have a cylindrical surface for securing wooden parts together, while screws have a helical surface, in the context of fastening wooden parts, nails and screws can be equivalent structures. The applicant’s explicit intent is not to invoke paragraph 6 of 35 USC § 112 to impose any limitation on any of the claims herein, unless the claims expressly use the phrase “means for…” in conjunction with the associated function.

Claims

1. A method, the method comprising: Images are received via an encoder; Feature maps are generated by performing partial convolution operations using the image and a mask image of the image via the encoder. An updated mask image is generated based on the dimensions of the mask image and the feature map via the encoder; The encoder performs a global average pooling (GAP) operation on the feature map using the updated mask image to generate a global feature map. as well as The encoder generates the output feature vector of the image based on the global feature map.

2. The method of claim 1, wherein the encoder is part of a variational autoencoder (VAE).

3. The method of claim 1, wherein the method comprises: The mask image is generated by detecting specific pixel values ​​in the image.

4. The method of claim 1, wherein the mask image comprises a binary image.

5. The method of claim 4, wherein the binary image includes a first portion corresponding to invalid pixels of the image, the invalid pixels corresponding to pixel values ​​equal to 0.

6. The method of claim 5, wherein the binary image includes a second portion corresponding to valid pixels of the image, the valid pixels corresponding to pixel values ​​equal to 1.

7. The method of claim 1, wherein the image and the mask image have the same size.

8. The method of claim 1, wherein the global feature map comprises a one-dimensional (1D) tensor generated based on the average response amplitude of each channel of the feature map.

9. The method of claim 8, wherein the feature map comprises a plurality of channels, and each value of the 1D tensor is determined based on the updated mask image and a corresponding channel among the plurality of channels.

10. The method of claim 1, wherein the image comprises an instance image of rock fragments from a geological stratum, and wherein the output feature vector comprises the rock properties of the rock fragments.

11. The method of claim 10, wherein the rock properties include texture information.

12. The method of claim 10, wherein the rock properties include color information.

13. A method for training an autoencoder (AE), the method comprising: Generate a training image set from multiple images; Generate corresponding mask images for the training image set; The training image set and the corresponding mask image are input into the encoder of the AE, wherein the encoder is configured to: A corresponding feature map is generated for the training image set by performing a partial convolution operation using the training images and the corresponding mask images; and The corresponding global feature map is generated by performing a global average pooling (GAP) operation on the corresponding feature map. The decoder of the AE generates corresponding output images for the training image set based on the corresponding global feature maps; as well as The AE is trained using the corresponding output image and the training image, based on a loss function.

14. The method of claim 13, wherein the training image set and the corresponding mask image have the same size.

15. The method of claim 13, wherein the method comprises: The corresponding mask image is randomly generated for the training image set.

16. The method of claim 13, wherein the loss function includes a perceptual loss function for extracting texture information from the training image set.

17. The method of claim 13, wherein the plurality of images comprises images of rock fragments from geological strata.

18. An autoencoder (AE), the autoencoder (AE) comprising: The encoder includes: A convolutional layer configured to generate a feature map by performing a partial convolution operation using an image and a mask image of the image; and A global average pooling (GAP) layer, configured to generate a global feature map by performing a GAP operation on the feature map using an updated mask image generated based on the mask image; and A decoder configured to generate an output image based on the global feature map.

19. The AE of claim 18, wherein the mask image comprises a binary image.

20. The AE of claim 19, wherein the binary image includes a first portion corresponding to invalid pixels of the image, the invalid pixels corresponding to pixel values ​​equal to 0.

21. The AE of claim 20, wherein the binary image includes a second portion corresponding to valid pixels of the image, the valid pixels corresponding to pixel values ​​equal to 1.

22. The AE of claim 18, wherein the GAP layer is configured to generate the global feature map having feature rotation and translation invariance and scale equivariance.

23. The AE of claim 18, wherein the image comprises an example image of rock fragments from a geological stratum.

Images