Foundation models for artificial intelligence-based geoscience solutions

Abstract

The present disclosure provides techniques for obtaining a domain-specific model targeted for a downstream task. A method may include obtaining a plurality of unlabeled geospatially-indexed datasets from different geographic locations representing different subsurface geologic conditions; training a neural network model on the plurality of unlabeled geospatially-indexed datasets in a self-supervised manner to obtain a foundation model that develops generalizable representations capturing shared interrelationships across the different subsurface geologic conditions; obtaining a labeled geospatially-indexed training dataset specific to a designated survey area; adapting parameters of the foundation model through retraining using the labeled geospatially-indexed training dataset to develop a domain-specific model targeted for a prediction task in the designated survey area; and implementing the domain-specific model on new geospatially-indexed input data from the designated survey area to generate predictions tied to locations for the targeted prediction task.

Description

FOUNDATION MODELS FOR ARTIFICIAL INTELLIGENCE-BASED GEOSCIENCE SOLUTIONSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This Application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 606,629, filed on December 6, 2023, the entire contents of which are hereby incorporated by reference.BACKGROUNDField

[0002] Aspects of the present disclosure relate to foundation models for geoscience applications.Description of Related Art

[0003] The development of machine learning-based solutions can be difficult due to various complexities and challenges. These complexities and challenges exist in the development of machine learning-based solutions for multiple geoscience applications, including but not limited to seismic, wellbore, and modeling applications. For example, subsurface geologic phenomena are inherently complex and data used for geoscience applications tends to contain large amounts of noise. Further, diverse features across surveys and / or fields and limited availability of sufficient representative training and / or label data add to the complexities and challenges associated with the development of machine learning-based solutions.

[0004] Additionally, traditional machine learning strategies exhibit limited reusability, requiring building customized models from scratch for each individual geoscience analysis task and dataset. This fragmented approach requires extensive manual effort and compute-intensive training for each new targeted geography, data source, and application type, which can incur great time, cost, and risk. Further, geoscience experts spend significant amounts of time labeling per task training data. Data scientists then configure and tune specialized models for each new model scenario. Thus, even utilizing high-performance infrastructures, training progresses slowly. Each developed and specialized model applies narrowly to other tasks. Thus, the lack of model generalization requires that the entire development cycle repeat when expanding models to newlocations or applications. This hinders scalability and adoption, as the machine learning models is not generalizable for other tasks, other regions / locations, or other geologies.

[0005] Accordingly, there is a need for methods and computing systems that can employ more efficient and accurate methods for developing machine learning-based solutions for multiple geoscience applications.SUMMARY

[0006] Certain aspects provide methods for obtaining a domain-specific model targeted for a downstream task. The method may include obtaining a plurality of unlabeled geospatially-indexed datasets from different geographic locations representing different subsurface geologic conditions; training a neural network model on the plurality of unlabeled geospatially-indexed datasets in a self-supervised manner to obtain a foundation model that develops generalizable representations capturing shared interrelationships across the different subsurface geologic conditions; obtaining a labeled geospatially-indexed training dataset specific to a designated survey area; adapting parameters of the foundation model through retraining using the labeled geospatially-indexed training dataset to develop a domain-specific model targeted for a prediction task in the designated survey area; and implementing the domain-specific model on new geospatially-indexed input data from the designated survey area to generate predictions tied to locations for the targeted prediction task.

[0007] Other aspects provide processing systems configured to perform the aforementioned methods as well as those described herein; non-transitory, computer-readable media comprising instructions that, when executed by a processors of a processing system, cause the processing system to perform the aforementioned methods as well as those described herein; a computer program product embodied on a computer readable storage medium comprising code for performing the aforementioned methods as well as those further described herein; and a processing system comprising means for performing the aforementioned methods as well as those further described herein.

[0008] The following description and the related drawings set forth in detail certain illustrative features of one or more aspects.DESCRIPTION OF THE DRAWINGS

[0009] The appended figures depict certain aspects and are therefore not to be considered limiting of the scope of this disclosure.

[0010] FIG. 1 depicts an example foundation model, in accordance with examples of the present disclosure.

[0011] FIGS. 2A-2E depict additional details directed to one or more environments in which data can be obtained for use with a foundation model and / or a downstream application derived or based on the foundation model can be implemented, in accordance with examples of the present disclosure.

[0012] FIGS. 3A-3D depict additional details directed to one or more environments in which data can be obtained for use with a foundation model and / or a downstream application derived or based on the foundation model can be implemented, in accordance with examples of the present disclosure.

[0013] FIG. 4 depicts an exemplary flow diagram directed to building a seismic foundation model, in accordance with examples of the present disclosure.

[0014] FIG. 5 is an exemplary flow diagram directed to implementing and / or using a pretrained seismic foundation model for a specific application, in accordance with examples of the present disclosure.

[0015] FIG. 6 is an exemplary flow diagram directed to implementing and / or using a pretrained seismic foundation model for a specific application, in accordance with examples of the present disclosure.

[0016] FIGS. 7A-7C depict example facies models, in accordance with examples of the present disclosure.

[0017] FIG. 8 depicts an exemplary flow diagram directed to building a well log, or wellbore, foundation model, in accordance with examples of the present disclosure.

[0018] FIG. 9 is an exemplary flow diagram of implementing and / or using a pre-trained well log foundation model for a specific application, in accordance with examples of the present disclosure.

[0019] FIG. 10 is an exemplary flow diagram of implementing and / or using a pre-trained well log foundation model for a specific application, in accordance with examples of the present disclosure.

[0020] FIGS. 11A & 11B depict example predictions and training accuracy for an application of a pre-trained well log foundation model, in accordance with examples of the present disclosure.

[0021] FIG. 12 depicts an exemplary flow diagram of building a controlled-source electromagnetic (CSEM) or magnetotellurics (MT) foundation model, in accordance with examples of the present disclosure.

[0022] FIG. 13 is an exemplary flow diagram of implementing and / or using a pre-trained CSEM or MT foundation model for a specific application, in accordance with examples of the present disclosure.

[0023] FIGS. 14A-14B depict an exemplary flow diagram directed to implementing and / or using a first foundation model and a second foundation model for a joint downstream task, in accordance with examples of the present disclosure.

[0024] FIG. 15 depicts an exemplary flow diagram directed to building a joint seismic-well log foundation model, in accordance with examples of the present disclosure.

[0025] FIG. 16 depicts an exemplary flow diagram directed to implementing and / or using a pre-trained large joint seismic-well log foundation model for a specific application, in accordance with examples of the present disclosure.

[0026] FIG. 17 depicts an example method for developing a machine-learning model.

[0027] FIG. 18 depicts an example processing system with which aspects of the present disclosure can be performed.

[0028] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the drawings. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation.DETAILED DESCRIPTION

[0029] Aspects of the present disclosure provide apparatuses, methods, processing systems, and computer-readable mediums for building and implementing one or more foundation models.

[0030] Due to complex subsurface geology and noisy data, developing accurate machine learning (ML) solutions for geoscience is difficult and time and resource intensive. Traditional supervised learning approaches require extensive labeled data, expert guidance, and tuning for each specific task and region. For example, a seismic interpretation model trained on data from one basin may not perform well when applied to a different basin with distinct geologic features. As another example, a fault detection model trained on seismic surveys from the North Sea may fail to generalize to the structurally distinct Permian Basin. This makes developing and deploying ML solutions across tasks and regions challenging.

[0031] Foundation models provide an efficient way to develop artificial intelligence (Al) solutions for complex geoscience applications like seismic data analysis. A foundation model is a machine learning model that is trained in a way to learn rich, generalizable representations from a diverse set of unlabeled data. However, developing foundation models can require large amounts of labeled data and may require extensive amounts of compute and time dedicated to foundation model training. Thus, there is a need for more efficient methods to develop reusable foundation models for geographic applications.

[0032] Techniques disclosed herein, provide methods and system for training foundation models using diverse unlabeled geoscience data in a self-supervised fashion to obtain pre-trained foundation models. The pre-trained foundation models learn robust data representations and can generalize across regions, data types, and tasks. Fine-tuning the foundation models with small labeled datasets enables efficient deployment for downstream applications. The disclosed techniques improve machine learning model quality by learning from diverse data. The disclosed techniques also reduce the data labeling, expert guidance, compute, and time required for a downstream application of a resulting foundation model. Accordingly, a foundation model in accordance with examples of the present disclosure, enables fast and reliable deployment across regions, data types, and tasks.

[0033] In examples, supervised labeling may refer to generating labels or targets for a machine learning model by leveraging the structure in the unlabeled data itself. This allows a model to betrained in a “self-supervised” manner without needing human-labeled examples. Some examples of supervised labeling tasks for self-supervised learning include: reconstruction, context prediction, future prediction, and cross-modal prediction. In reconstruction tasks, a corrupted version of an input is created by adding noise, masking sections, applying distortions, etc. The foundation model is trained to reconstruct the original uncorrupted input from this modified version such that the target is the original input itself. In context prediction tasks, sections of the input are masked. That is, the model is trained to predict the masked sections based on the surrounding observable context. In such instances, the targets are the missing sections. In future prediction tasks, given a sequence of inputs, the foundation model is trained to predict subsequent values in the sequence. Thus, the future values are the targets. In cross-modal prediction tasks, together with multi-modal data, the foundation model learns to predict representations in one modality from other modalities; for example, an image could be used to predict associated text.

[0034] Thus, unlike supervised learning that relies on human annotations, self-supervised techniques automatically generate inputs and corresponding labels / targets from the data itself. This allows models, like neural networks, to learn meaningful representations from unlabeled datasets in a scalable way. The self-supervised tasks provide a form of automatic “self-teaching”, guiding the foundation model to understand properties of the data distributions needed for downstream tasks.Example Foundation Model

[0035] FIG. 1 depicts an example foundation model 100 in accordance with examples of the present disclosure. The foundation model 100, also referred to as a large domain model, can be built and / or developed for multiple geoscience applications. As described above, there are many complexities and challenges associated with developing machine learning-based solutions. The foundation model 100 can be a broadly applicable machine learning solution that is capable of handling the intricate details and challenges of geology and data analysis, while requiring only small amount of labeled training data in some instances. Specifically, the foundation model 100 can leverage large volumes of unlabeled, diverse training data across different geologies and data modalities to learn complex spatial and temporal relationships. This allows the foundation model 100 to build a rich conceptual understanding of subsurface structures. As a result, the foundation model 100 can accurately interpret and model geological phenomena despite complications suchas incomplete data coverage, significant amounts of noise, and high heterogeneity between geographic areas. By pre-training a foundation model 100 on diverse unlabeled data at scale, the foundation model 100 can exploit performance across different downstream prediction tasks using small application-specific labeled datasets for fine-tuning. This enables efficient adaptation to new geologic regions and data types.

[0036] The foundation model 100 can include, but is not limited to, the following components: a training database 102, a deep learning model 104, and training strategies for the deep learning model. In some examples, the training database 102 can include a diverse set of data types. For example, a training database 102 may include wireline logs from multiple geographic regions (e.g., 101). In such cases, the data in the training database 102 can be unlabeled. An example implementation of deep learning model 104 is deep learning model 104 A, which can include an encoder 108 module coupled to a decoder 110 module via a central feature extraction layer 112. The central feature extraction layer can utilize a combination of techniques such as dimensionality reduction, attention mechanisms, and / or bottleneck architectures to refine the intermediate data representation from the encoder 108. The central feature extraction layer 112 can selectively emphasize the most informative features and suppress the less relevant ones. The exact mechanism of feature selection and emphasis can be adaptive, based on backpropagation signals received during the training phase, thereby allowing the model to tailor its feature extraction strategy to the specific data and task. The encoder 108 and decoder 110 modules are constructed by stacking layers which transform representations of the input data. For instance, these stacked layers may include convolutional and fully-connected layers or attention mechanisms as used in transformer architectures. After training, the encoder 108 can be used to extract descriptive features from new data samples. These learned representations serve as inputs to task-specific downstream models. The decoder 110 supports supervised training by attempting to reconstruct the original input from the extracted features. By training the foundation model 100 to minimize the difference between the input and reconstructed data, the encoder 108 can be optimized to capture semantics useful across various downstream tasks.

[0037] The deep learning model 104 can be alternatively based on a generative adversarial networks (GAN) based paradigm where a generator and a discriminator are simultaneously trained in an adversarial manner. In some cases regarding the training strategies of the deep learning model 104, training can occur before, during, and after performance of a downstream task(s). Forexample, before performing a downstream task, a foundation model can be trained on unlabeled training data from a training database 102 in a self-supervised fashion, where such training is directed to various objectives, including but not limited to masked autoencoding, contrastive learning, and generative-adversarial learning. In another example, during the performance of a downstream task, an encoder may be kept constant as previously trained, or trainable together with the downstream machine learning model(s). In another example, after the performance of a downstream task, the entire deep learning model 104 may be updated using new data on which the downstream task is performed.

[0038] In some examples, the foundation model 100 can include standard neural networks, transformers, convolutional neural networks, or other specialized model architectures. For example, the foundation model 100 may have stacked transformer blocks that utilize attention mechanisms and self-attention to spatially and temporally process seismic signal inputs. Alternatively, convolutional filters can capture local spatial patterns in the seismic data. The architecture of the foundation model 100 may comprise encoder and decoder stages for representation learning and seismic data reconstruction tasks during pre-training. Both convolutional and transformer components provide complementary modeling capacities that can be combined in a deep neural network framework.

[0039] In some examples, the foundation model 100 can include a pre-trained deep learning model 104 that can be adapted for multiple downstream tasks. In some examples, the foundation model 100 can accelerate training for downstream tasks by enabling rapid fine-tuning of new models, instead of requiring full retraining from scratch. For instance, by leveraging representations and patterns learned from diverse training data, the foundation model 100 enables downstream models to be fine-tuned using only small task-specific datasets. This transfer learning capability also yields faster convergence during application-specific training. As a result, developing production-ready models for new geoscience analysis tasks can be performed in a much shorter amount of time, such as in hours or days rather than weeks or months.

[0040] Additionally, since less training data and compute resources are required for fine- tuning, the foundation model 100 enables deploying performant Al solutions even when access to infrastructure or expert labeling is limited. The reduced data and compute requirements provideopportunity for real-time interactivity for model development workflows as well as lightweight deployments of production models to edge devices in the field.

[0041] The building and / or development of a foundation model 100 for multiple geoscience applications can improve the quality of downstream predictions. For example, by pre-training the foundation model 100 on large, diverse unlabeled data, the learned representations capture informative patterns that facilitate generalization across tasks, data modalities, and geographical regions. In some examples, the foundation model 100 may operate on geospatially-indexed datasets. Geospatially-indexed data sets refer to data that contains measurements associated with geographic locations or spatial positioning information. Some examples of geospatially-indexed datasets include, but are not limited to: seismic surveys, well logs, electromagnetic (EM) readings, and satellite imagery. Seismic wave recordings may include tracing coordinates tying them to surface positions where reflected signals were detected. Well logs measure subsurface properties versus depth along a borehole trajectory, indexing measurements to underground locations. EM readings encode signal strength levels at surface spots where measurements were taken. Satellite imagery includes pixels that encode spectral readings spatially over a surveyed landscape area. Thus, geospatially-indexed means the data has inherent spatial information anchoring signal values to physical positions, whether on the surface or sub-surface. This differs from generic datasets that may lack innate positioning relationships. The spatial aspect provides context and enables alignment between datasets captured across different modalities like seismic, wells, and EM surveys - facilitating a joint analysis. Additionally, geospatial indexing allows interpolating between measurement locations to estimate properties at unsampled areas based on proximity relationships through geospatial modeling techniques.

[0042] In examples, the foundation model 100 enables efficient transfer learning, allowing accurate models for new analysis tasks to be rapidly created by fine-tuning a pre-trained model instead of training custom models from scratch. Fine-tuning requires less training data, compute resources, and human effort compared to traditional supervised learning approaches. This significantly accelerates downstream model development cycles and reduces the risk of overfitting on limited data.

[0043] By developing a reusable foundation model, examples of the present disclosure enable streamlined applications for new geoscience use cases. The transfer learning capability reduces thecomplexity and time associated with adapting machine learning solutions to new geographical areas or analysis tasks. Additionally, the reduced training data and infrastructure requirements enable model deployment even under connectivity, budget, and / or expert resourcing constraints. Thus, the foundation modeling approach saves significant human and computational resources while providing accessible Al-powered geoscience analysis.

[0044] As described above, advantages associated with the foundation model 100 include the concept of reusability and transferability. Specifically, the foundation model 100 can be trained only once on large, diverse unlabeled datasets to learn generally applicable data representations. The pre-trained model can then be efficiently fine-tuned to perform well on a wide range of downstream prediction problems. This reusable transfer learning capability enables developing accurate models for novel analysis applications using limited task-specific labeled data. By leveraging a pretrained foundation model 100 as a starting point, less training examples and compute are required compared to traditional supervised learning. This allows extending machine learning solutions to geoscience domains lacking sizable labeled datasets, like seismic analysis. Additionally, the foundation model’s applicability across domains enables streamlined model development workflows. Expert users can provide limited application-specific annotations to tailor a reusable model to new use cases rather than building custom solutions from scratch. This facilitates incorporating user feedback to enhance model robustness and user adoption.

[0045] In accordance with examples of the present disclosure, a foundation model 100 can be built and / or developed as a solution for multiple downstream tasks 106, such as geoscience data analysis, including seismic data conditioning, geo-feature extraction, fault / salt detection, stratigraphy delineation, property estimation and monitoring, well marker picking, log processing, log interpretation, subsurface conductivity anomaly geometry, and properties quantification (e.g., conductivity, or any other property sensitivity to CSEM measurement including fluid saturation, porosity, etc.). The foundation model 100 can include a deep learning model 104 capable of supporting differing downstream applications and downstream tasks 106 in various capacities. In some examples, the foundation model 100 can be trained using a self-supervised approach across multiple datasets that include unlabeled data and can generalize when solving multiple downstream tasks during a fine-tuning stage.

[0046] Additionally, in some examples, the foundation model 100 training process can leverage multi-task learning to improve generalizability. Specifically, the model can be trained jointly on primary geoscience analysis tasks as well as auxiliary tasks using weakly supervised or semi-supervised techniques. For example, a fault detection model can be trained concurrently to predict fault likelihoods from seismic data as well as to complete missing sequences and generate estimates for another prediction from partial inputs, such as predicting missing log values or seismic traces.

[0047] At least one goal of training the neural network model in a self-supervised manner on the variety of unlabeled geospatially-indexed datasets is to develop generalizable data representations that effectively capture meaningful patterns describing distinct subsurface conditions. Rather than just learning features specific to one geographic site or geologic environment, the foundation model extracts a rich collection of shared subsurface representations in the form of embeddings. These embeddings encompass commonalities across different structures, formation configurations, fluid signatures, and other latent attributes generalized across domains. Even with limited labeled data from a new target region, a model having generalizable representations can quickly adapt to unfamiliar subsurface patterns previously unseen during initial training.

[0048] The foundation model developed through unsupervised pre-training establishes a broad set of universal subsurface representations and relationships that lend adaptability to new locations and measurement types. Configuring a domain-specific model involves tuning the model parameters to specialize its operation towards particular prediction tasks tied to individual survey areas. This configuration is achieved by further training the foundation model using labeled examples from the target geography that adjust the internal weightings and connections to boost sensitivity to distinguishing characteristics and geo-attributes within that selected locale. For instance, fine-tuning a reservoir sweet-spot locator would incorporate labels highlighting porous formations, hydrocarbon indicators, existing production assets and other distinguishing sitespecific traits to mold enhanced attunement.

[0049] The resulting domain-specific model is then specifically contoured to its intended operating region and objectives. Thus, the configuration adjusts model focus while retaining pretrained comprehension. This enables efficiently generating accurate predictions for assignedgeoscience applications in specified areas without requiring full retraining from scratch. A domainspecific model is tuned to its niche based on the scaffolding developed through generalized pretraining.

[0050] Moreover, training on noisy or inaccurate auxiliary labels (e.g. labels picked automatically without verification) allows incorporating unlabeled data at scale. While the weak labels themselves may have limited utility, learning in tandem with primary labeled data can provide regularization that reduces overfitting and increases applicability. During a fine-tuning stage, the foundation model 100 can be adapted to solve for specific down-stream prediction tasks.

[0051] The foundation model 100 can be customized to new problems without extensive retraining using transfer learning. This transfer learning approach enables accelerating convergence compared to training custom models from scratch. Specifically, the foundation model’s representations, extracted by base encoder layers, facilitate rapid adaptation using small labeled datasets. Fine-tuning requires fewer examples, less compute, and achieves higher performance than conventional supervised learning. This reduces overfitting risks, enables unsupervised or self-supervised application scenarios, and increases accuracy. In comparison to conventional supervised techniques, the fine-tuning stage can present enhanced performance 118, such as faster convergence, higher stability, reduced risk of overfitting in small training data, increased accuracy, and reduced dependency on training data 116. Thus, a pre-training methodology increase applicability across tasks, data types, and regions. This flexibility provides that the foundation model can provide reusable and configurable building blocks tailored for diverse seismic, well, electromagnetic or other analysis applications.Example Data Collection Environments

[0052] FIGS. 2A-3D depict additional details directed to one or more environments in which data can be obtained for use with a foundation model and / or a downstream application derived or based on the foundation model. FIGS. 2A-2E illustrate simplified, schematic views of oilfield 200A-200E having subterranean formation 202 containing reservoir 204 therein in accordance with implementations of various technologies and techniques described herein.

[0053] More specifically, FIG. 2A illustrates a survey operation 200A (such as a seismic data acquisition process) being performed by a survey tool, such as seismic truck 206.1, to measureproperties of the subterranean formation and generate input data suitable for training and the application of seismic foundation models as described in FIG. 1. The survey operation is a seismic survey operation for producing sound vibrations. In FIG. 2A, one such sound vibration, e.g., sound vibration 212 generated by source 210, reflects off horizons 214 in earth formation 216. Sensors, such as geophones 218, situated on the earth’s surface, receive a set of sound vibrations. The data received 220 is provided as input data to a computer 222.1 of a seismic truck 206.1, which processes the signals into seismic data output 224 that images the subsurface.. The seismic images encode information about structures and stratigraphy that could facilitate pre-training of foundation models and subsequent fine-tuning for analysis tasks like fault detection. However, traditionally generating high-quality labeled training data requires extensive human effort. Foundation models offer a pathway to unlocking the value of abundant unlabeled surveys like this, derived from diverse surface seismic shoots, for developing reusable seismic Al tools. In examples, the seismic data output 224 may be stored, transmitted or further processed as desired, for example, by data reduction.

[0054] FIG. 2B illustrates a drilling operation being performed by drilling tools 206.2 suspended by rig 228 and advanced into subterranean formations 202 to form wellbore 236. Mud pit 230 is used to draw drilling mud into the drilling tools via flow line 232 for circulating drilling mud down through the drilling tools, then up wellbore 236 and back to the surface. The drilling mud is typically filtered and returned to the mud pit. A circulating system may be used for storing, controlling, or filtering the flowing drilling mud. The drilling tools are advanced into subterranean formations 202 to reach reservoir 204. Each well may target one or more reservoirs. The drilling tools are adapted for measuring downhole properties using logging while drilling tools. The logging while drilling tools may also be adapted for taking core sample 233 as shown.

[0055] Computer facilities may be positioned at various locations about the oilfield 200B (e.g., the surface unit 234) and / or at remote locations. Surface unit 234 may be used to communicate with the drilling tools and / or offsite operations, as well as with other surface or downhole sensors. Surface unit 234 is capable of communicating with the drilling tools to send commands to the drilling tools, and to receive data therefrom. Surface unit 234 may also collect data generated during the drilling operation and produce data output 235, which may then be stored or transmitted.

[0056] Sensors, such as gauges, may be positioned about the oilfield 200B to collect data relating to various oilfield operations as described previously. As shown in FIG. 2B, a sensor can be positioned in one or more locations in the drilling tools and / or at rig 228 to measure drilling parameters, such as weight on bit, torque on bit, pressures, temperatures, flow rates, compositions, rotary speed, and / or other parameters of the field operation. Sensors may also be positioned in one or more locations in the circulating system.

[0057] Drilling tools 206.2 may include a bottom hole assembly (BHA) (not shown), generally referenced, near the drill bit (e.g., within several drill collar lengths from the drill bit). The BHA includes capabilities for measuring, processing, and storing information, as well as communicating with surface unit 234. The BHA further includes drill collars for performing various other measurement functions.

[0058] The BHA may include a communication subassembly that communicates with surface unit 234. The communication subassembly is adapted to send signals to and receive signals from the surface using a communications channel such as mud pulse telemetry, electro-magnetic telemetry, or wired drill pipe communications. The communication subassembly may include, for example, a transmitter that generates a signal, such as an acoustic or electromagnetic signal, which is representative of the measured drilling parameters. It will be appreciated by one of skill in the art that a variety of telemetry systems may be employed, such as wired drill pipe, electromagnetic, or other known telemetry systems.

[0059] In examples, the wellbore is drilled according to a drilling plan that is established prior to drilling. The drilling plan typically sets forth equipment, pressures, trajectories and / or other parameters that define the drilling process for the wellsite. The drilling operation may then be performed according to the drilling plan. However, as information is gathered, the drilling operation may need to deviate from the drilling plan. Additionally, as drilling or other operations are performed, the subsurface conditions may change. The earth model may also need adjustment as new information is collected.

[0060] Surface unit 234 and / or other data collection sources for analysis or other processing may collect the data gathered by sensors. For example, the data gathered by sensors in FIG. 2B may be collected for foundational model training and analysis. This raw sensor data can capturesubsurface properties and operational conditions. When accumulated across diverse sites and well types, the data forms an unlabeled training corpus for developing reusable foundation models.

[0061] The collected data may be historical logs or real-time streams from active drilling. These measurements may quantify key physical attributes including temperature, pressure, resistivity, density, porosity, and composition. Along with operational metrics like drill rate and weight-on-bit, these signals characterize subsurface environments. By concatenating and standardizing heterogeneous streamed and logged data from many wells, a universal dataset can be obtained. Such a high-volume corpus can be used for unsupervised pre-training of rich feature extractors via foundation model techniques. Downstream models can then build on these generalizations by fine-tuning for specialized tasks. The data may be collected in one or more databases and / or transmitted on or offsite.

[0062] Surface unit 234 may include transceiver 237 to allow communications between surface unit 234 and various portions of the oilfield 200B or other locations. Surface unit 234 may also be provided with or functionally connected to one or more controllers (not shown) for actuating mechanisms at oilfield 200B. Surface unit 234 may then send command signals to oilfield 200 in response to data received. Surface unit 234 may receive commands via transceiver 237 or may itself execute commands to the controller. A processor may be provided to analyze the data (locally or remotely), make the decisions and / or actuate the controller. In this manner, oilfield 200B may be selectively adjusted based on the data collected. This technique may be used to optimize (or improve) portions of the field operation, such as controlling drilling, weight on bit, pump rates, or other parameters. These adjustments may be made automatically based on computer protocol, and / or manually by an operator. In some cases, well plans may be adjusted to select optimum (or improved) operating conditions, or to avoid problems.

[0063] Surface unit 234 illustrates a potential integration point for foundation model deployment to optimize oilfield operations. For example, the controllers, processors, and transceivers could leverage fine-tuned foundation models that ingest sensor streams and prescribe automated actions or manual interventions. As another example, a model adapted to recognize unsafe drill vibration patterns from real-time block position, hookload, and torque telemetries could trigger adaptive guidance of the drill plan without operator input. This would improve reliability and efficiency. The model could be further updated based on measured outcomes, withhuman experts reviewing results. Thus, the oilfield 200B described in FIG. 2B demonstrates opportunities for using trainable universal foundation models specialized for control tasks. This allows updating surface equipment with Al assists that use limited labeled data from individual wellsites.

[0064] FIG. 2C illustrates a wireline operation being performed by wireline tool 206.3 suspended by rig 228 and into wellbore 236. Wireline tool 206.3 is adapted for deployment into wellbore 236 for generating well logs, performing downhole tests and / or collecting samples. Wireline tool 206.3 may be used to provide another method and apparatus for performing a seismic survey operation. Wireline tool 206.3 may, for example, have an explosive, radioactive, electrical, or acoustic energy source 244 that sends and / or receives electrical signals to surrounding subterranean formations 202 and fluids therein.

[0065] Wireline tool 206.3 may be operatively connected to, for example, geophones 218 and a computer 222.1 of a seismic truck 206.1 of FIG. 2A. Wireline tool 206.3 may also provide data to surface unit 234. Surface unit 234 may collect data generated during the wireline operation and may produce data output 235 that may be stored or transmitted. Wireline tool 206.3 may be positioned at various depths in the wellbore 236 to provide a survey or other information relating to the subterranean formation 202.

[0066] Sensors, such as gauges, may be positioned about oilfield 200C to collect data relating to various field operations as described previously. A sensor can be positioned in wireline tool 206.3 to measure downhole parameters, which relate to, for example porosity, permeability, fluid composition and / or other parameters of the field operation. This provides additional unlabeled subsurface attribute data to augment pre-training of reusable foundation models. Specifically, wireline sensors profile subsurface formations by recording properties like resistivity, porosity, permeability, and composition as a function of depth. Deploying instruments across diverse geology and well types provides information-rich signals portraying spatial variability within and across sites. Learning from such heterogeneous unlabeled wireline data enables a foundation model to capture complex inter-property relationships that generalize across distinct downstream analysis tasks. This learning enables effective transfer learning. Once trained at scale, a model can be efficiently fine-tuned using limited labels from a new location to perform accurate wirelineinterpretation, formation evaluation, or casing inspection for that site. Thus, wireline tools generate useful unlabeled training examples for building reusable models.

[0067] FIG. 2D illustrates a production operation being performed by production tool 206.4 deployed from a production unit or Christmas tree 229 and into completed wellbore 236 for drawing fluid from the downhole reservoirs into surface facilities 242. The fluid flows from reservoir 204 through perforations in the casing (not shown) and into production tool 206.4 in wellbore 236 and to surface facilities 242 via gathering network 246.

[0068] Sensors, such as gauges, may be positioned about oilfield 200D to collect data relating to various field operations as described previously. As shown, a sensor may be positioned in production tool 206.4 or associated equipment, such as Christmas tree 229, gathering network 246, surface facility 242, and / or the production facility, to measure fluid parameters, such as fluid composition, flow rates, pressures, temperatures, and / or other parameters of the production operation. FIG. 2D illustrates hydrocarbons being produced from reservoir 204 via wellbore 236. Operational telemetry and periodically sampled properties from production data encode rich unlabeled signals useful for pre-training foundation models. Patterns in multiphase flow rate, phase fractions, wellhead and bottomhole pressures relate to subsurface architectures and fluid types. Gathered across producing assets, such data can be used by models in a broad manner, for example, as the data may be applicable to various production engineering analyses. While FIGS. 2A-2D illustrate tools used to measure properties of an oilfield, it will be appreciated that the tools may be used in connection with non-oilfield operations, such as gas fields, mines, aquifers, storage or other subterranean facilities. Also, while certain data acquisition tools are depicted, it will be appreciated that various measurement tools capable of sensing parameters, such as seismic two- way travel time, density, resistivity, production rate, etc., of the subterranean formation and / or its geological formations may be used. Various sensors may be located at various positions along the wellbore and / or the monitoring tools to collect and / or monitor the desired data. Other sources of data may also be provided from offsite locations. While FIGS. 2A-2D depict oilfield tools, unlabeled data from comparable equipment in gas fields, mines, aquifers, storage facilities, etc. offers additional perspectives to augment foundation model training diversity to strengthen model versatility. The field configurations of FIGS. 2A-2D are intended to provide a brief description of an example of a field usable with data collection environments. Part of, or the entirety of, oilfield 200 (e.g., 200A, 200B, 200C, and / or 200D) may be on land, water, and / or sea. Also, while a singlefield measured at a single location is depicted, oilfield applications may be utilized with any combination of one or more oilfields, one or more processing facilities and one or more wellsites.

[0069] FIG. 2E illustrates a schematic view, partially in cross section, of oilfield 200E having data acquisition tools 202.1, 202.2, 202.3 and 202.4 positioned at various locations along oilfield 200 for collecting data of reservoir 204 in accordance with implementations of various technologies and techniques described herein. Data acquisition tools 202.1-202.4 may be the same as data acquisition tools 206.1-206.4 of FIGS. 2A-2D, respectively, or others not depicted. As shown, data acquisition tools 202.1-202.4 generate data plots or measurements 208.1-208.4, respectively. These data plots are depicted along oilfield 200 to demonstrate the data generated by the various operations.

[0070] Data plots 208.1-208.3 are examples of static data plots that may be generated by data acquisition tools 202.1-202.3, respectively; however, it should be understood that data plots 208.1- 208.3 may also be data plots that are updated in real time. These measurements may be analyzed to better define the properties of the formation(s) and / or determine the accuracy of the measurements and / or for checking for errors. The plots of each of the respective measurements may be aligned and scaled for comparison and verification of the properties.

[0071] Static data plot 208.1 is a seismic two-way response over a period of time. Static plot208.2 is core sample data measured from a core sample of the reservoir 204. The core sample may be used to provide data, such as a graph of the density, porosity, permeability, or some other physical property of the core sample over the length of the core. Tests for density and viscosity may be performed on the fluids in the core at varying pressures and temperatures. Static data plot208.3 is a logging trace that typically provides a resistivity or other measurement of the formation at various depths.

[0072] A production decline curve or graph 208.4 is a dynamic data plot of the fluid flow rate over time. The production decline curve typically provides the production rate as a function of time. As the fluid flows through the wellbore, measurements are taken of fluid properties, such as flow rates, pressures, composition, etc.

[0073] Other data may also be collected, such as historical data, user inputs, economic information, and / or other measurement data and other parameters of interest. As described below, the static and dynamic measurements may be analyzed and used to generate models of thesubterranean formation to determine characteristics thereof. Similar measurements may also be used to measure changes in formation aspects over time.

[0074] The subterranean structure 204 has a plurality of geological formations 206.1-206.4. As shown, this structure has several formations or layers, including a shale layer 206.1, a carbonate layer 206.2, a shale layer 206.3 and a sand layer 206.4. A fault 207 extends through the shale layer 206.1 and the carbonate layer 206.2. The static data acquisition tools are adapted to take measurements and detect characteristics of the formations.

[0075] While a specific subterranean formation with specific geological structures is depicted, it will be appreciated that oilfield 200E may contain a variety of geological structures and / or formations, sometimes having extreme complexity. In some locations, typically below the water line, fluid may occupy porous spaces of the formations. Each of the measurement devices may be used to measure properties of the formations and / or its geological features. While each acquisition tool is shown as being in specific locations in oilfield 200E, it will be appreciated that one or more types of measurement may be taken at one or more locations across one or more fields or other locations for comparison and / or analysis.

[0076] The data collected from various sources, such as the data acquisition tools of FIG. 2E, may then be processed and / or evaluated. Typically, seismic data displayed in static data plot 208.1 from data acquisition tool 202.1 is used by a geophysicist to determine characteristics of the subterranean formations and features. The core data shown in static plot 208.2 and / or log data from well log 208.3 are typically used by a geologist to determine various characteristics of the subterranean formation. The production data from graph 208.4 is typically used by the reservoir engineer to determine fluid flow reservoir characteristics. The data analyzed by the geologist, geophysicist and the reservoir engineer may be analyzed using modeling techniques.

[0077] Moreover, the diverse data collected by the various acquisition tools in FIG. 2E can provide a training corpus for developing foundation models. Rather than relying on manual labeling by individual domain experts during supervised training, a universal model can assimilate volumes of unlabeled measurements to learn underlying representations. For example, relationships between seismic images, well logs, and production time series characterize linkages between structures, lithology, and fluid flow. Jointly analyzing these signals without restrictive annotations enables a model to capture data fundamentals that can generalize across applications.

[0078] Once trained, domain-specific fine-tuning that leverages limited labels adapts the common foundation model to focused tasks (e.g. identifying productive zones from seismic images for geophysicists, predicting rock mechanical properties from well data for engineers, or forecasting reservoir performance for modelers).

[0079] FIG. 3A illustrates an oilfield 300 for performing production operations in accordance with implementations of various technologies and techniques described herein. As shown, the oilfield 300 has a plurality of wellsites 302 operatively connected to central processing facility 354. The oilfield configuration of FIG. 3A is not intended to limit the scope of the oilfield application system. Part, or all, of the oilfield may be on land and / or sea. Also, while a single oilfield with a single processing facility and a plurality of wellsites is depicted, any combination of one or more oilfields, one or more processing facilities and one or more wellsites may be present.

[0080] Each wellsite 302 has equipment that forms wellbore 336 into the earth. The wellbores extend through subterranean formations 306 including reservoirs 304. These reservoirs 304 contain fluids, such as hydrocarbons. The wellsites draw fluid from the reservoirs and pass them to the processing facilities via surface networks 344. The surface networks 344 have tubing and control mechanisms for controlling the flow of fluids from the wellsite to processing facility 354.

[0081] As illustrated in FIG. 3A, the example oilfield 300 with wellsites 302 draw hydrocarbons from subsurface reservoirs 304 via wellbores 336 and surface networks 344 for processing at facility 354. Large volumes of unlabeled sensor and operational data are generated across the hydrocarbon supply chain described with respect to FIG. 3A. In examples, patterns in measurements from wellsite equipment and flowing fluids relate operating configurations and subsurface properties to production rates. Accumulating such unlabeled data from various sites provides data for self-supervision for use with training foundation models encompassing crossdomain relationships.

[0082] Fine-tuning pretrained models using limited labeled data from specific wellsites then enables optimized control and forecasting applications. As such, training and adaptation of foundation models helps to enhance hydrocarbon extraction efficiency.

[0083] FIG. 3B illustrates a side view of a marine-based survey 360 of a subterranean subsurface 362 in accordance with one or more implementations of various techniques describedherein. Subsurface 362 includes seafloor surface 364. Seismic sources 366 may include marine sources such as vibroseis or airguns, which may propagate seismic waves 368 (e.g., energy signals) into the Earth over an extended period of time or at a nearly instantaneous energy provided by impulsive sources. The seismic waves may be propagated by marine sources as a frequency sweep signal. For example, marine sources of the vibroseis type may initially emit a seismic wave at a low frequency (e.g., 5 Hz) and increase the seismic wave to a high frequency (e.g., 80-90Hz) over time.

[0084] The component(s) of the seismic waves 368 may be reflected and converted by seafloor surface 364 (i.e., reflector), and seismic wave reflections 370 may be received by a plurality of seismic receivers 372. Seismic receivers 372 may be disposed on a plurality of streamers (i.e., streamer array 374). The seismic receivers 372 may generate electrical signals representative of the received seismic wave reflections 370. The electrical signals may be embedded with information regarding the subsurface 362 and captured as a record of seismic data.

[0085] In one implementation, each streamer may include streamer steering devices such as a bird, a deflector, a tail buoy and the like, which are not illustrated in this application. The streamer steering devices may be used to control the position of the streamers in accordance with the techniques described herein.

[0086] In one implementation, seismic wave reflections 370 may travel upward and reach the water / air interface at the water surface 376, a portion of reflections 370 may then reflect downward again (i.e., sea-surface ghost waves 378) and be received by the plurality of seismic receivers 372. The sea-surface ghost waves 378 may be referred to as surface multiples. The point on the water surface 376 at which the wave is reflected downward is generally referred to as the downward reflection point.

[0087] The electrical signals may be transmitted to a vessel 380 via transmission cables, wireless communication or the like. The vessel 380 may then transmit the electrical signals to a data processing center. Alternatively, the vessel 380 may include an onboard computer capable of processing the electrical signals (i.e., seismic data). Those skilled in the art having the benefit of this disclosure will appreciate that this illustration is highly idealized. For instance, surveys may be of formations deep beneath the surface. The formations may typically include multiple reflectors, some of which may include dipping events, and may generate multiple reflections(including wave conversion) for receipt by the seismic receivers 372. Tn one implementation, the seismic data may be processed to generate a seismic image of the subsurface 362.

[0088] Typically, marine seismic acquisition systems tow each streamer in streamer array 374 at the same depth (e.g., 5-10m). However, marine-based survey 360 may tow each streamer in streamer array 374 at different depths such that seismic data may be acquired and processed in a manner that avoids the effects of destructive interference due to sea-surface ghost waves. For instance, marine-based survey 360 of FIG. 3B illustrates eight streamers towed by vessel 380 at eight different depths. The depth of each streamer may be controlled and maintained using the birds disposed on each streamer.

[0089] As described above, a seismic survey vessel 380 towing an array of streamers 374 with sensors to measure reflections from subsurface structures. The streamer depths are staggered to collect richer data less affected by ghosting interference. While advanced acquisition improves imagery, manually annotating the vast volumes for supervised learning is impractical thereby restricting model development. Using a foundation model can overcome this impracticality by enabling reusable seismic machine learning tools to be derived from such diverse surveys without labeling. By pre-training on unlabeled field records, models learn useful conceptual representations of geology, fluids, textures etc. Specialized versions can then be adaptively tailored to particular seismic interpretation tasks like salt delineation using limited samples.

[0090] FIG. 3C depicts a marine electromagnetic survey system 382 in accordance with implementations of various technologies described herein. The electromagnetic survey system 382 may use CSEM survey techniques, but other electromagnetic survey techniques may also be used. Marine electromagnetic surveying may be performed by a survey vessel 384 that moves in a predetermined pattern along the surface 385 of a body of water such as a lake or the ocean. The survey vessel 384 is configured to pull a towfish (an electric source) 386, which is connected to a pair of electrodes 388. During the survey, the vessel may stop and remain stationary for a period while obtaining measurements, while in some circumstances, the vessel may remain underway while obtaining measurements.

[0091] At the source 386, a controlled electric current may be generated and sent through the electrodes 388 into the seawater. For instance, the electric current generated may be in the range between about 0.01 Hz and about 20 Hz. The current creates an electromagnetic field 390 in thesubsurface area 392 to be surveyed below the sea floor 393. The electromagnetic field 390 may also be generated by magneto-telluric currents instead of the source 386. The survey vessel 384 may also be configured to tow a sensor cable 394. The sensor cable 394 may be a marine towed cable. The sensor cable 394 may contain sensor housings 395, telemetry units 396, and current sensor electrodes (not illustrated). The sensor housings 395 may contain voltage potential electrodes for measuring the electromagnetic field 390 strength created in the subsurface area 392 during the surveying period. The current sensor electrodes may be used to measure electric field strength in directions transverse to the direction of the sensor cable 394 (the y- and z-directions). The telemetry units 396 may contain circuitry configured to determine the electric field strength using the electric current measurements made by the current sensor electrodes. While a marinebased electromagnetic survey is described in regard to FIG. 3C, a land-based electromagnetic survey may also be used in accordance with implementations of various techniques described herein.

[0092] FIG. 3D depicts an embodiment of seismic system 310 in which a plurality of tow vessels 311 is employed to enable seismic profiling, e.g. three-dimensional vertical seismic profiling or rig / offset vertical seismic profiling. In FIG. 3D, a marine system is illustrated as including a rig 312, a plurality of vessels 311, and one or more acoustic receivers 313. Although a marine system is illustrated, other embodiments of the disclosure may not be limited to this example. A person of ordinary skill in the art will recognize that teachings of the disclosure may be used in land or offshore systems. However, offshore systems are described herein to simplify the disclosure and to facilitate explanation.

[0093] Although two vessels 311 are illustrated in FIG. 3D, a single vessel 311 with multiple source arrays 315 or multiple vessels 311 each with single or multiple sources 315 may be used. In some applications, at least one source / source array 315 may be located on the rig 312 as represented by the rig source in FIG. 3D. As the vessels 311 travel on predetermined or systematic paths, their locations may be recorded through the use of navigation system 318. In some cases, the navigation system 318 utilizes a global positioning system (GPS) 319 to record the position, speed, direction, and other parameters of the tow vessels 311.

[0094] As illustrated, the global positioning system 319 may utilize or work in cooperation with satellites 320, which operate on a suitable communication protocol, e.g. VSATcommunications. The VSAT communications may be used, among other things, to supplement VHF and UHF communications. The GPS information can be independent of the VSAT communications and may be input to processing system or other suitable processors to predict the future movement and position of the vessels 311 based on real-time information. In addition to predicting future movements, the processing system also can be utilized to provide directions and coordinates as well as to determine initial shot times, as described above. A control system effectively utilizes a processing system in cooperation with source controller and synchronization unit to synchronize the sources 315 with the downhole data acquisition system 321.

[0095] As illustrated, the one or more vessels 311 each tow one or more acoustic sources / source arrays 315. The source arrays 315 include one or more seismic signal generators 317, e.g. air guns, configured to create a seismic / sonic disturbance. In the embodiment illustrated, the tow vessels 311 comprise a master source vessel 328 (Vessel A) and a slave source vessel 329 (Vessel B). However, other numbers and arrangements of tow vessels 311 may be employed to accommodate the parameters of a given seismic profiling application. For example, one source315 may be mounted at rig 312 (see FIG. 3D) or at another suitable location, and both vessels 311 may serve as slave vessels with respect to the rig source 316 or with respect to a source at another location.

[0096] However, a variety of source arrangements and implementations may be provided as desired for a given application. When utilizing dithered timing between the sources, for example, the master and slave locations of the sources can be adjusted according to the parameters of the specific seismic profiling application. In some applications, one of the source vessels 311 (e.g. source vessel A 356 in FIG. 3D) may serve as the master source vessel while the other source vessel 311 serves as the slave source vessel (e.g., 329) with dithered firing. However, an alternate source vessel 311 (e.g. source vessel B in FIG. 3D) may serve as the master source vessel while the other source vessel 311 serves as the slave source vessel with dithered firing.

[0097] Similarly, the rig source 316 may serve as the master source while one of the source vessels 311 (e.g. vessel A 328) serves as the slave source vessel with dithered firing. The rig source316 also may serve as the master source while the other source vessel 311 (e.g. vessel B 329) serves as the slave source vessel with dithered firing. In some applications, the rig source 316 may serve as the master source while both of the source vessels 311 serve as slave source vessels eachwith dithered firings. These and other arrangements may be used in achieving the desired synchronization of sources 315 with the downhole acquisition system 321.

[0098] The acoustic receivers 313 of data acquisition system 321 may be deployed in borehole 330 via a variety of delivery systems, such as wireline delivery systems, slickline delivery systems, and other suitable delivery systems. Although a single acoustic receiver 313 could be used in the borehole 330, the illustrated embodiment comprises a plurality of acoustic receivers 313 that may be located in a variety of positions and orientations. The acoustic receivers 313 may be configured for sonic and / or seismic reception. Additionally, the acoustic receivers 313 may be communicatively coupled with processing equipment 314 located downhole. By way of example, processing equipment 314 may comprise a telemetry system for transmitting data from acoustic receivers 313 to additional processing equipment 331 located at the surface, e.g. on the rig 312 and / or vessels 311.

[0099] Depending on the specifics of a given data communication system, examples of surface processing equipment 33 Imay comprise a radio repeater 332, an acquisition and logging unit 333, and a variety of other and / or additional signal transfer components and signal processing components. The radio repeater 332 along with other components of processing equipment 331 may be used to communicate signals, e.g. UHF and / or VHF signals, between vessels 311 and rig 312 and to enable further communication with downhole data acquisition system 321.

[0100] It should be noted the UHF and VHF signals can be used to supplement each other. In general, the UHF band supports a higher data rate throughput but can be susceptible to obstructions and has less range. The VHF band is less susceptible to obstructions and has increased radio range but its data rate throughput is lower. In FIG. 3D, for example, the VHF communications are illustrated as "punching through" an obstruction in the form of a production platform.

[0101] In some applications, the acoustic receivers 313 are coupled to surface processing equipment 331 via a hardwired connection. In other embodiments, wireless or optical connections may be employed. In still other embodiments, combinations of coupling techniques may be employed to relay information received downhole via the acoustic receivers 313 to an operator and / or control system, e.g. control system, located at least in part at the surface.

[0102] In addition to providing raw or processed data uphole to the surface, the coupling system, e.g. downhole processing equipment 314 and surface processing equipment 331, may bedesigned to transmit data or instructions downhole to the acoustic receivers 313. For example, the surface processing equipment 331 may comprise synchronization unit, which coordinates the firing of sources 315, e.g. dithered (delayed) source arrays, with the acoustic receivers 313 located in borehole 330. According to one embodiment, the synchronization unit uses coordinated universal time to ensure accurate timing. In some cases, the coordinated universal time system is employed in cooperation with global positioning system 319 to obtain UTC data from the GPS receivers of GPS system 319.

[0103] The seismic system 310 depicted in FIG. 3D synchronizes and integrates surface sources with downhole receivers to enrich subsurface coverage for enhanced foundation model training. By combining borehole measurements with larger-scale reflections at the surface, hybrid seismic sensing provides insights into the complex subsurface structures. Pre-training on such diverse unlabeled signals allows foundation models to interrelate complex structures such that downstream models can then be efficiently tailored to make predictions about stratigraphy, for example, at a new drill site from limited labeled examples. This transfer learning increases accuracy and accelerates deployment to optimize well and drill positioning relative to faults and reservoirs.

[0104] FIG. 3D illustrates one example of a system for performing seismic profiling that can employ simultaneous or near-simultaneous acquisition of seismic data. By way of example, the seismic profiling may comprise three-dimensional vertical seismic profiling but other applications may utilize rig / offset vertical seismic profiling or seismic profiling employing walkaway lines. A source 315 located on rig 50, on a stationary vessel 311, and / or on another stationary vessel or structure, can provide an offset source.

[0105] As an example, the overall seismic system 310 may employ various arrangements of sources 315 on vessels 22 and / or rig 50 with each location having at least one source / source array 315 to generate acoustic source signals. The acoustic receivers 313 of downhole acquisition system 321 are configured to receive the source signals, at least some of which are reflected off a reflection boundary located beneath a sea bottom. The acoustic receivers 313 generate data streams that are relayed uphole to a suitable processing system via downhole telemetry / processing equipment 314.

[0106] While the acoustic receivers 313 generate data streams, the navigation system determines a real-time speed, position, and direction of each vessel 22 and also estimates initialshot times accomplished via signal generators of the appropriate source arrays 315. The source controller may be part of surface processing equipment 331 (located on rig 312, on vessels 311, or at other suitable locations) and is designed to control firing of the acoustic source signals so that the timing of an additional shot time (e.g. a shot time via slave source vessel 329) is based on the initial shot time (e.g. a shot time via master source vessel 328) plus a dither value.

[0107] The synchronization unit of, for example, surface processing equipment 331, coordinates the firing of dithered acoustic signals with recording of acoustic signals by the downhole acquisition system 321. Processor system is configured to separate a data stream of the initial shot and a data stream of the additional shot via the coherency filter. As discussed above, however, other embodiments may employ pure simultaneous acquisition and / or may not perform separation of the data streams. In such cases, the dither is effectively zero.

[0108] After an initial shot time at T=0 (TO) is determined, subsequent firings of acoustic source arrays 315 may be offset by a dither. The dithers can be positive or negative and sometimes are created as pre-defined random delays. Use of dithers facilitates the separation of simultaneous or near-simultaneous data sets to simplify the data processing. The ability to have the acoustic source arrays 315 fire in simultaneous or near-simultaneous patterns reduces the overall amount of time used for three-dimensional vertical seismic profiling source acquisition. This, in turn, reduces rig time. As a result, the overall cost of the seismic operation is reduced, rendering the data intensive process much more accessible.

[0109] If the acoustic source arrays used in the seismic data acquisition are widely separated, the difference in move-outs across the acoustic receiver array of the wave fields generated by the acoustic sources 315 can be sufficient to obtain a clean data image via processing the data without further special considerations. However, even when the acoustic sources 315 are substantially colocated in time, data acquired by any of the methods involving dithering of the firing times of the individual sources 315 described herein can be processed to a formation image leaving hardly any artifacts in the final image. This is accomplished by taking advantage of the incoherence of the data generated by one acoustic source 315 when seen in the reference time of the other acoustic source 315.

[0110] The seismic system 310 depicted in FIG. 3D synchronizes and integrates surface sources with downhole receivers to enrich subsurface coverage for enhanced foundation modeltraining. By combining borehole measurements with larger-scale reflections at the surface, hybrid seismic sensing provides insights into the complex subsurface structures. Pre-training on such diverse unlabeled signals allows foundation models to interrelate complex structures such that downstream models can then be efficiently tailored to make predictions about stratigraphy, for example, at a new drill site from limited labeled examples. This transfer learning increases accuracy and accelerates deployment to optimize well and drill positioning relative to faults and reservoirs.[0U1] As described above, FIG. 3D illustrates concurrent seismic excitation from multiple surface ships and downhole receivers to enhance subsurface illumination over legacy sequential profiling. This simultaneous shooting acquires richer data. While human labeling would maximize value, requiring interpreters to annotate enormous volumes is impractical. Fortunately, foundation models provide an alternative. By pre-training on raw unconventional records like the information described in FIG. 3D, seismic machine learning models can be trained on rich representations without explicit supervision. Specialized versions can then be adapted to effectively analyze novel surveys for exploitation with minimal labeling.

[0112] Attention is now directed to methods, techniques, and workflows for processing and / or transforming collected data that are in accordance with some embodiments. Some operations in the processing procedures, methods, techniques, and workflows disclosed herein may be combined and / or the order of some operations may be changed. Those with skill in the art will recognize that in the geosciences and / or other multi-dimensional data processing disciplines, various interpretations, sets of assumptions, and / or domain models such as velocity models, may be refined in an iterative fashion; this concept is applicable to the procedures, methods, techniques, and workflows as discussed herein. This iterative refinement can include use of feedback loops executed on an algorithmic basis, such as at a computing device (e.g., computing system 1800, FIG. 18), and / or through manual control by a user who may make determinations regarding whether a given step, action, template, or model has become sufficiently accurate.Examples for Developing Foundation Models

[0113] FIG. 4 depicts an exemplary flow diagram of a method 400 for training a seismic foundation model - a machine learning model pre-trained on extensive raw or processed subsurface imagery to learn broadly useful data representations without manual labeling. in accordance withexamples of the present disclosure. FIG. 4 illustrates the training of a seismic foundation model as an example of a foundation model for a single data modality (e.g., seismic data). Other examples of training foundation models for a single data modality are further described below in FIG. 8 and FIG. 12. While FIG. 4, FIG. 8, and FIG. 12, illustrate examples of single modality data foundation models, such as seismic, log, CSEM, or MT data, foundation models trained for single modality data are not limited to these examples.

[0114] In examples, the method 400 for training a seismic foundation model begins at 402, which may be triggered by a user request, a scheduled task, or an automated process based on data availability. Upon starting, the method 400 includes loading seismic volume data 404 (e.g., a first set of seismic volume or a first set of seismic data) and training 406 a seismic foundation model 408 based on the seismic volume data 404. Seismic volume data 404 may refer to a three- dimensional dataset generated by seismic surveys, which are used primarily in the exploration for oil, gas, and other subsurface resources. As described with respect to FIGS. 2A-3D, this data can be created by sending seismic waves into the Earth using sources like controlled explosions or specialized equipment. As these waves travel through the Earth, they encounter different layers and materials, causing reflections at various boundaries. These reflected waves are then recorded by sensors (geophones or hydrophones) placed on the surface or in the ocean. The recorded data is processed to create a three-dimensional representation of the subsurface, showing the geological structures and layers. Each point in the seismic volume data 404 represents a seismic reflection with its amplitude, which can indicate the presence of different geological features, such as rock types, faults, and fluid-filled reservoirs.

[0115] In some examples, the training includes updating 416 by loading new seismic volume data 418 and continuing to train 406 the seismic foundation model 408. Once the seismic foundation model 408 is trained, a quality check 410 can be performed of the seismic foundation model 408 to determine whether or not to save the seismic foundation model 408. For example, if the quality check 410 indicates that the trained seismic foundation model 408 is “good” or meets or exceeds a quality threshold value, then the trained seismic foundation model 408 can be saved at 412, e.g., for implementation and the method 400 can stop at 414. If the quality check 410 indicates that the trained seismic foundation model 408 is “fair” or “poor” or fails to meet or exceed a quality threshold value, then the trained seismic foundation model 408 can be further trained on additional and / or new seismic volume data 418.

[0116] In some examples, the training of the foundation model for a single modality data can be performed over several tasks via self-supervision (e.g., training the model based on labels generated from self-supervised learning). For example, the tasks can include interpolation, denoising, and reconstruction, using modified (corrupted) version of the original data as input. Further, the results from the tasks can be evaluated against the original (uncorrupted) data. If there is a large error and / or misfit determined during the evaluation, this may be indicative of the need for further training. Further, in some cases, the training of the foundation model can be continuous as new or updated data is received.

[0117] As one example implementation of the method 400 of FIG. 4, one or more seismic datasets are loaded containing raw or processed subsurface imaging volumes gathered across diverse sites and geologies. The seismic foundation model 408 is then trained 406 to perform reconstruction tasks, like imputing missing traces using surrounding context. These selfsupervised objectives derive supervisory signals from the unannotated data itself to update model parameters.

[0118] In examples, the seismic foundation model 408 may comprise a deep neural network optimized for generating spatiotemporal representations in seismic data through iteratively reducing reconstruction losses. The quality check 410 can subsequently evaluate the fidelity of modeled outputs compared to actual ground truths on validation samples, determining whether learned embeddings are sufficiently descriptive before saving the seismic model (e.g., 412) and operational deployment.

[0119] In some examples, periodic retraining of the seismic foundation model 408 can incorporate new seismic volume data 418 into the seismic foundation model 408 to enhance model generalization. Fine-tuning for downstream tasks may then be initiated from this updated starting point. While a seismic focus provides specialization, multi-modal embodiments, such as those described in the present disclosure, alternatively integrate well log data or other physical measurements concurrently within the unified model.

[0120] FIG. 5 is an exemplary flow diagram of implementing and / or using a pre-trained seismic foundation model (or large seismic model) for a specific application. Other examples of implementing foundation models based on single data modality are further described below in FIG. 6, FIG. 9, FIG. 10, and FIG. 13 While FIG. 5, FIG. 6, FIG. 9, FIG. 10, and FIG. 13illustrate examples of implementing single modality data foundation models, such as seismic, log, CSEM, or MT data, implementing such foundation models are not limited to these examples.

[0121] As depicted in FIG. 5, a pre-trained seismic foundation model (e.g., seismic foundation model saved at 412 of FIG. 4) can be applied to a downstream task, such as but not limited to fault detection. In some examples, the downstream task can be for a single data modality. In one example, a method 500 of implementing and / or using the pre-trained seismic foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) for specific application (e.g., fault detection) is described. The method 500 begins at 502, which may be triggered by a user request, a scheduled task, or an automated process based on data availability. Upon starting, the method 500 can proceed to load seismic volume data at 504 (e.g., a second set of seismic volume or a second set of seismic data). In examples, a seismic volume data 504 is a 3D data structure containing processed outputs from seismic sensors traversing a surveyed area, with fluctuations representing subsurface geology useful for exploration. Multiple seismic volumes covering different regions can be analyzed using techniques described in FIG. 5. Seismic volume data 504 can capture both raw pre-processed signals as well as post-processing outputs depending on the stage of analysis.

[0122] In some examples, the method 500 can select and annotate representative inline, z, and / or crossline slices. The method 500 further includes loading 512 the pre-trained large seismic foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) along with the selected representative inline slices (e.g., slices corresponding to a constant x position), crossline slices (e.g., slices corresponding to a constant y position), and / or z slices (e.g., slices corresponding to a constant z / depth position) 506 having expert labels (e.g., labeled training set 508) to train 514 a downstream task model (e.g., a fault model 516). In some examples, a subset of the foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) can be trained; or the subset of the founding model (e.g., the pre-trained seismic foundation model 408 of FIG. 4) can fine-tune another subset of the foundation model. For example, the trained seismic foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) can be used to train the downstream task model (e.g., a fault model 516).

[0123] In some examples, the downstream task model (e.g., a fault model 516) can be evaluated via a quality check 520 (e.g., using unlabeled testing data 510) to determine if thedownstream task model (e.g., fault model 516) is accurate. For example, the prediction 518 of the downstream task model (e.g., fault model 516) in training can be checked and if the prediction 518 based on the unlabeled testing data 510 is accurate or “good” or meets or exceeds a quality threshold value, then the downstream task model (e.g., a fault model 516) can be saved at 522 (e g., for implementation) as a downstream task model. The method 500 can then stop at 524. If the prediction 518 is incorrect or “fair” or fails to meet or exceeds a quality threshold value, then the downstream task model (e.g., a fault model 516) can undergo further training 514 until a threshold is met during the quality check 520 to save the downstream task model 522.

[0124] FIG. 6 is an exemplary flow diagram of implementing and / or using a pre-trained seismic foundation model (or large seismic model) for a specific application. Other examples of implementing foundation models based on single data modality are further described below in FIG. 9, FIG. 10, and FIG. 13. While FIG. 9, FIG. 10 and FIG. 13 illustrate examples of implementing single modality data foundation models, such as seismic, log, CSEM, or MT data, implementing such foundation models are not limited to these examples.

[0125] As depicted in FIG. 6, a pre-trained seismic foundation model (e.g., seismic foundation model saved at 412 of FIG. 4 can be applied to a downstream task, such as but not limited to selecting and interpreting facies on representative inline and crossline slices. In some examples, the downstream task can be for a single data modality. In one example, a method 600 of implementing and / or using the pre-trained seismic foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) for specific application (e.g., selecting and interpreting facies on representative inline and crossline slices) is described. The method 600 can begin at 602, which may be triggered by a user request, a scheduled task, or an automated process based on data availability. Upon starting, the method 600 can proceed to load seismic volumes at 604 (e.g., a second set of seismic volume or a second set of seismic data).

[0126] In some examples, the method 600 can select and annotate representative inline, z, and / or crossline slices. The method 600 further includes loading 612 the pre-trained large seismic foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) along with the selected representative inline slices (e.g., slices corresponding to a constant x position), crossline slices (e.g., slices corresponding to a constant y position), and / or z slices (e.g., slices corresponding to a constant z / depth position) 606 having expert labels (e.g., labeled training set608) to train 614 a downstream task model (e.g., facies model 616). In some examples, a subset of the foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) can be trained; or the subset of the founding model (e.g., the pre-trained seismic foundation model 408 of FIG. 4) can fine-tune another subset of the foundation model. For example, the trained seismic foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) can be used to train the downstream task model (e.g., a facies model 616).

[0127] In some examples, the downstream task model (e.g., a facies model 616) can be evaluated via a quality check 620 (e.g., using unlabeled testing data 610) to determine if the downstream task model (e.g., facies model 616) is accurate. For example, the prediction 618 of the downstream task model (e.g., facies model 616) in training can be checked and if the prediction 618 based on the unlabeled testing data 610 is accurate or “good” or meets or exceeds a quality threshold value, then the downstream task model (e.g., a facies model 616) can be saved at 622 (e.g., for implementation) as a downstream task model. The method 600 can then stop at 624. If the prediction 618 is incorrect or “fair” or fails to meet or exceeds a quality threshold value, then the downstream task model (e.g., facies model 616) can undergo further training 614 until a threshold is met during the quality check 620 to save the downstream task model 622.

[0128] As an example, FIGS. 7A-7C depict comparisons between the prediction of the facies model using a pre-trained foundation model (e.g., FIG. 7A) and a prediction of the facies model without using the pre-trained foundation model (e.g., FIG. 7C), where FIG. 7B depicts a ground truth or manual interpretation of the facies model. Fine-tuning the foundation model to obtain the facies model 616 that generates FIG. 7A shows better continuation and prediction of facies, particularly for minor facies like thin slope valley channel systems (e.g., areas 702A and ground truth area 702B) and mass transport deposit (e.g., areas 704A and ground truth area 704B), when compared to the facies model that does not use the pre-trained foundation model (e.g. 702C and 704C of FIG. 7C).

[0129] The training 514 and 614 of the downstream task model (e.g., a fault model 516 or a facies model 616) for specific applications of fault detection and facies classification are two examples of downstream tasks. In other examples, the training 514 and 614 of the downstream task model (e.g., a fault model 516 or a facies model 616) can be for various tasks including, but not limited to, applications in stratigraphy interpretation, seismic processing and imaging, seismicinversion, and seismic interpretation. Tn such cases, once the downstream task model is saved, the downstream task model can be implemented to perform one or more of the downstream tasks including, but not limited to, applications in seismic processing, seismic imaging, and seismic interpretation. Additional examples of seismic application can include inversion and quantitative interpretation. Further, foundation models can be built for pre-stack and post-stack seismic data solutions.

[0130] FIG. 8 depicts an exemplary flow diagram directed to building a well log (or wellbore) foundation model (or a large well log model). Similar to the seismic foundation model 408 of FIG. 4, FIG. 8 depicts an example of a well log foundation model 808 for a single data modality (e g., wellbore data). An exemplary embodiment of implementation of a well log (or wellbore) foundation model can be found in PCT Application No. PCT / US2023 / 082115 filed on December 1, 2023, which claims priority to U.S. Provisional Patent Application No. 63 / 385,704, filed on December 1, 2022, both of which are incorporated herein by reference in their entirety. An example of building of a well log foundation model 808 is described below.

[0131] In examples, a computing system can obtain unlabeled log data. For instance, the unlabeled log data may be acquired from sensor(s) of a wireline tool and / or wellbore as described in any of FIGS. 2A-3D. Additionally or alternatively, the unlabeled log data may be from multiple wellbores, multiple oilfields, and / or multiple geographic locations. In some examples, the data acquired from the sensor(s) may be labeled at acquisition. These labels may be removed before sending to the computing system. For instance, the labels may be removed at a surface unit or another computing system. Alternatively, the labels may be removed within the computing system. In other words, the log data may be labeled when received by the computing system, but the computing system obtains the unlabeled log data by removing the labels from the labeled log data before transmitting the unlabeled log data to the neural network(s).

[0132] Constructing the foundational model (e.g., well log foundation model 808) includes performing machine learning that may utilize one or more neural networks. This machine learning may be trained with self-supervised tasks at scale on a large and varied data set, which includes the unlabeled log data. Multiple different types of measurements from one or more formations and formation types may be included in the unlabeled log data. As discussed in greater detail herein below, the self-supervised training tasks may include transforming the unlabeled log data.Examples of such transformation include adding noise to the data and applying a controlled distortion to the data. In some embodiments, adding noise before or after applying distortion to the data may comprise a transformation of the unlabeled log data.

[0133] Examples of sources of unlabeled log data may include neutron porosity (NPOR) logs, gamma ray (GR) logs, compressional slowness (DTC), shear slowness (DTS), bulk density (RHOB) logs, spontaneous potential (SP) logs, caliper (CALI) logs, shallow resistivity (LLS) logs, deep induction (ILD) logs, photoelectric (PEF) logs, and / or any other suitable log types that may be used in performing any suitable downstream applications that may be desirable in relation to wellbores. Compressional slowness is an alternative name for transit time of compressional waves or delta T compression (DTC). Shear slowness is an alternative name for transit time of shear waves or delta T of shear.

[0134] Constructing the foundational model (e.g., well log foundation model 808) may include the gathering of the data (e.g., well logs) at scale, training the foundational model, and evaluating the foundational model’s performance. The training and evaluating may be recursive. Training / evaluation may use self-supervised paradigms that incorporate statistical analysis, domain-driven alterations, and derived curves from the input well logs to force the foundational model to learn representations of the well logs. For instance, the domain-driven alterations may include injecting synthetic error (e.g., noise) into extracted sections of the well logs. The injected error may be representative of one or more types of systematic measurement error that may be experienced in well logs. For example, such errors may include lateral shift for neutron and gamma ray logs, scaling for gamma ray logs, small vertical shifts for one or more logs, synthetic alterations, resampling borehole effect on density and neutron porosity logs, more generally multiplicative and additive noise based on one or more probability distributions. Similarly, the statistical analysis may include attempting to make predictions of parameters using a portion of the data (e.g., in past parameters using present data in a log, in future parameters using present data in a log, or a combination of using past and future data to predict present missing data, and so forth).

[0135] For instance, if the predictions can be verified in the data, the prediction may be evaluated to determine the accuracy of the foundational model (e.g., well log foundation model 808). Such tweaks and / or predictions may continue to further improve the foundational model untila target time and / or convergence (when training loss settles to within an error range of a final value or additional training will not improve the foundational model further) is reached. Once the foundational model (e.g., well log foundation model 808) is constructed, the resulting data / algorithm may be saved and stored to any suitable location, such as a database. Additionally or alternatively, the foundational model (e.g., well log foundation model 808) may be stored in a computing system of a surface unit that may not have a robust Internet connection due to the potentially remote nature of oilfields.

[0136] The computing system storing or having access to the foundational model (e.g., well log foundation model 808) then fine-tunes the foundational model for a downstream application. In other words, the computing system adapts the foundational model (e.g., well log foundation model 808) to solve for specific tasks. Indeed, the foundational model (e.g., well log foundation model 808) may be broadly applicable to multiple different (and even unrelated) downstream tasks. The fine-tuning may be used to refine the foundational model (e.g., well log foundation model 808) for one of multiple available downstream tasks that may be used to solve specific wellbore log problems. For instance, the multiple available downstream tasks may include, but not be limited to, outlier detection, wellbore log correlation to correlate new incoming data to a type of log or characteristic, correcting wellbore log errors, performing formation estimation of specific properties, performing zonation, predicting missing basic and advanced wellbore log data for one or more intervals, marking boundaries between zones, and / or any other tasks related to the log data.

[0137] Refining / fme-tuning the foundational model (e.g., well log foundation model 808) may include adapting the foundational model to solve for specific tasks. The refinement enables the computing system to implement at least one of the downstream applications based at least in part on the fine-tuned variant of the foundational model. The method may also include a control signal being sent by or responsive to the fine-tuned foundational model, the control signal may request or require the performance of a physical wellsite action. Responsive to the control signal, the physical wellsite action is taken, either automatically or by human intervention. The wellsite action may be based upon the one or more results, the equipment and processes, or a combination thereof. The wellsite action may be or include generating and / or transmitting a control signal (e.g., using a computing system) that causes a physical action to occur at a wellsite. The wellsite action may also or instead include performing the physical action at the wellsite. The physical action mayinclude selecting where to drill a wellbore, drilling the wellbore, varying a weight and / or torque on a drill bit that is drilling the wellbore, varying a drilling trajectory of the wellbore, varying a concentration and / or flow rate of a fluid pumped into the wellbore, or the like.

[0138] Implementing the downstream application using the fine-tuned foundational model takes advantage of the features and information extraction inherent in the foundational model (e.g., well log foundation model 808) and permits employing a wide variety of learning paradigms for specific tasks at hand. For instance, the implementation of the downstream application allows for use of supervised approaches, use of a small, labeled dataset (few-shot learning) approach, classification, self-supervised approaches such as log corrections, and unsupervised clustering, among other possibilities. The refinement and / or implementation may use the addition of new data types or new datasets, changes in the model structure, and / or retraining of part or all the parameters (e.g., weights and biases) from the foundational model. However, compared to traditional techniques, where the deep network is fully trained from the start, the fine-tuned network based in the foundational model (e.g., well log foundation model 808) presents faster convergence, higher stability, better generalization, reduced risk of overfitting in small training data, and in many cases includes increased accuracy.

[0139] Further, a single pre-trained foundational model (e.g., well log foundation model 808) provides an opportunity for better streamlining of log workflows using the common foundation of the foundational model. In other words, the foundational model enables a “head start” on the specific downstream applications enabling more streamlined processing and performance of the downstream tasks. It also enables the reusage of all or most of the weights from the foundational model (e.g., well log foundation model 808) by keeping them unaltered (freezing them) and using in context learning, few-shot learning, and soft prompt techniques when solving multiple downstream applications. Performance of a wellsite action will have some overlapping results as implementing the downstream application, as presented here.

[0140] The faster convergence and the reduced risk of overfitting in the presence of smaller amounts of data also may enable more user interaction for exploration and discovery in wellbore log-related products due to a smaller and more manageable data set and deploying lighter computational solutions in locations (e.g., oilfield) without high connectivity or access to a server with high computational capacity. Another advantage of using the foundational model (e.g., welllog foundation model 808) is the ability to generalize the solution across different fields, different acquisition tools, and the ability to be incorporated into multiple downstream tasks related to wellbore logs.

[0141] As described above, one of the biggest challenges when applying machine learning methods is insufficient training labels. For example, building a machine learning model with a small amount of training data can lead to poor generalization during inference. However, to overcome the challenges, the foundation model (e.g., well log foundation model 808), can be built for wellbore data (e.g., wireline logs, core cuttings, mud logs, cone penetration test data, etc.) that can extract large amount, complex features from data that can be used for various downstream tasks, including but not limited to outlier detection, log correction, missing log prediction, lithofacies and rock property interpretation, depth shifting, and marker detection (as long as the downstream task contains similar input logs such as a subset of the triple-combo and quad-combo logs).

[0142] In one example, a method 800 for building a well log foundation model 808 (or large well log foundation model) begins at 802, which may be triggered by a user request, a scheduled task, or an automated process based on data availability. Upon starting, the method 800, can proceed with loading well log data 804 (e.g., a first set of well log data) and training 806 a well log foundation model 808 based on the well log data 804. In some cases, the training 806 of the well log foundation model 808 includes updating 816 with loading new well log data 818 and continuing to train 806 the well log foundation model 808. Once the well log foundation model 808 is trained, a quality check 810 can be performed of the well log foundation model 808 to determine whether or not to save 812 the trained well log foundation model at 812. For example, if the quality check 810 indicates that the well log foundation model 808 is “good” or meets or exceeds a quality threshold value, then the well log foundation model 808 can be saved at 812 (e.g., for implementation) and the method 800 can stop at 814. If the quality check 810 indicates that the well log foundation model 808 is “fair” or “poor” or fails to meets or exceeds a quality threshold value, then the well log foundation model 808 can be further trained on additional and / or updated well log data (e.g., 818).

[0143] FIG. 9 is an exemplary flow diagram directed to implementing and / or using a pretrained large well log foundation model (e.g., the pre-trained well log foundation model saved at812 of FIG. 8) (or large well log foundation model) for a specific application. An exemplary embodiment of implementation of a well log (or wellbore) foundation model can be found in PCT Application No. PCT / US2023 / 082115 filed on December 1, 2023, which claims priority to U.S. Provisional Patent Application No. 63 / 385,704, filed on December 1, 2022, both of which are incorporated herein by reference in their entirety. As depicted in FIG. 9, the pre-trained large well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) can be applied to a downstream task, such as but not limited to, well marker propagation (picking), outlier detection, log correction, pay zone calculation, elastic properties, log prediction, formation interpretation, geomechanical applications, or marker detection. This flow diagram of FIG. 9 is an example of a single data modality.

[0144] In one example, a method 900 of implementing and / or using the pre-trained well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) for a specific application (e.g., marker propagation) is described. The method 900 can start at 902 and proceed to load well log data at 904 (e.g., a second set of well log data) and representative markers can be selected and / or annotated at 906. The method 900 further includes loading (e.g., 912) the pre-trained large well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) along with the selected representative markers with expert labels (e.g., labeled training data 908) to train a downstream task model e.g., a marker propagation model 916 to predict 918 propagated markers. In some examples, a subset of the foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) can be trained or fine-tune another subset of the foundation model. For example, the well log foundation model can be used to train the downstream task model (e.g., marker propagation model 916).

[0145] In some cases, the downstream task model (e.g., the marker propagation model 916) can be evaluated via a quality check 920 (e.g., using unlabeled testing data 910) to determine if the model (e.g., marker propagation model 916) is accurate or otherwise meets or exceeds a quality threshold. For example, the prediction 918 of the marker propagation model 916 in training can be evaluated and if the prediction 918 based on the unlabeled testing data 910 is accurate or “good”, or meets or exceeds a quality threshold value, then the marker propagation model 916 can be saved at 922 for implementation as a downstream task model. If the prediction 918 is incorrect or “fair” or fails to meet or exceed a quality threshold value, then the marker propagation model 916 canundergo further training (e.g., 914) until a threshold is met during the quality check 920 process to save the downstream task model at 922. The method 900 may stop at 924.

[0146] FIG. 10 is an exemplary flow diagram directed to implementing and / or using a pretrained well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) (or large well log foundation model) for a specific application. As depicted in FIG. 10, the pre-trained large well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) can be applied to a downstream task, such as but not limited to, well marker propagation (picking), outlier detection, log correction, pay zone calculation, elastic properties, log prediction, formation interpretation, geomechanical applications, or marker detection. This flow diagram of FIG. 10 is an example of a single data modality.

[0147] In one example, a method 1000 of implementing and / or using the pre-trained well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) for a specific application (e.g., log interpretation) is described. The method 1000 can start at 1002 and proceed to load well log data at 1004 (e.g., a second set of well log data) and representative wells can be selected and / or interpreted at 1006. The method 1000 further includes loading (e.g., 1012) the pre-trained large well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) along with the selected representative markers with expert labels (e.g., labeled training set 1008) to train a downstream task model e.g., a log interpretation model 1016 to predict 1018 propagated markers. In some examples, a subset of the foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) can be trained or fine-tune another subset of the foundation model. For example, the well log foundation model can be used to train the downstream task model (e.g., log interpretation model 1016).

[0148] In some cases, the downstream task model (e.g., the log interpretation model 1016) can be evaluated via a quality check 1020 (e.g., using unlabeled testing data 1010) to determine if the model (e.g., log interpretation model 1016) is accurate or otherwise meets or exceeds a quality threshold. For example, the prediction 1018 of the log interpretation model 1016 in training can be checked and if the prediction 1018 based on the unlabeled testing data 1010 is accurate or “good”, or meets or exceeds a quality threshold value, then the log interpretation model 1016 can be saved at 1022 for implementation as a downstream task model. If the prediction 1018 is incorrect or “fair” or fails to meet or exceed a quality threshold value, then the log interpretationmodel 1016 can undergo further training (e.g., 1014) until a threshold is met during the quality check 1020 process to save the downstream task model at 1022. The method 1000 may stop at 1024.

[0149] FIG. 11A illustrates a diagram where data predictions are generated using an example of a downstream task with the well log interpretation model 1016 in FIG. 10. Initially, the well log foundation model - specifically, a pre-trained version stored as item 812 in FIG. 8 — is refined through a fine-tuning process. This process adapts the foundation model (e.g., pre-trained version stored as item 812 in FIG. 8) into the well log interpretation model 1016 in FIG. 10, specifically for tasks related to well log interpretation. The fine-tuning involves using well logs that have already been labeled or interpreted (for instance, those marked as item 1008 in FIG. 10) to train the model, enabling it to make predictions on well logs that haven't been labeled or interpreted.

[0150] In this specific downstream task, the pre-trained well log foundation model (again, item 812 in FIG. 8) is fine-tuned to become the well log interpretation model 1016. This fine-tuning process can involve a variable number of wells, with the quantity ranging from as few as one well to up to nine wells, as shown in FIG. 11B. This approach demonstrates how the model is adapted from a general foundation to a specialized interpretation tool through selective training on a limited dataset.

[0151] Example predictions from this downstream task are depicted in FIG. 11A. More specifically, the left panel presents the variation of the testing performance when the number of training wells increases. As depicted in FIG. 11B, the log interpretation model 1016 fine-tuned from the foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) with only 2 wells outperforms a model that is not a fine-tuned foundation model using 9 wells, as indicated by prediction accuracy measured by mean-absolute error (MAE). The right panel in FIG. 11A depicts the interpreted logs (including PHIT (porosity, total), VCL (volume of clay), TOC (total organic carbon), and SW (water saturation) for each section from left to right, respectively) with one labeled well for training, where the lighter curves (or lines) correspond to ground truth labels and the darker curves (or lines) correspond to model predictions. As depicted in FIG. 11A, the interpretation results using the fine-tuned foundation model (right column in each section (e.g., 1102B, 1104B, 1106B, 1108B) generates predictions generally closer to the ground truth data thanthe predictions made without using the fine-tuned foundation model (left column in each section (e.g., 1102A, 1104A, 1106A, 1108A).

[0152] FIG. 12 is an exemplary flow diagram directed to building a CSEM / MT foundation model (or large domain model or large CSEM / MET model). FIG. 12 provides an example of training a foundation model for a single data modality (e.g., CSEM / MT data).

[0153] In one example, a method 1200 for building a CSEM or MT foundation model 1208 (or large CSEM or MT model) begins at 1202, which may be triggered by a user request, a scheduled task, or an automated process based on data availability. Upon starting, the method 1200, can proceed with loading CSEM or MT data at 1204 (e.g., a first set of CSEM or MT data) and training, at 1206 a CSEM or MT foundation model 1208 based on the CSEM or MT data loaded at 1204. In some examples, the training 1206 includes updating 1216 the CSEM or MT foundation model 1208 by loading new CSEM or MT data 1218 and continuing to train 1206 the CSEM or MT foundation model 1208. Once the CSEM or MT foundation model 1208 is trained, a quality check 1210 can be performed of the CSEM or MT foundation model 1208 to determine whether or not to save, at 1212, the CSEM or MT foundation model 1208. For example, if the quality check 1210 indicates that the CSEM or MT foundation model 1208 is “good” or meets or exceeds a quality threshold value, then the CSEM or MT foundation model 1208 can be saved at 1212, e.g., for implementation, and method 1200 can stop at 1214. If the quality check 1210 indicates that the CSEM or MT foundation model 1208 is “fair” or “poor” or fails to meet or exceed a quality threshold value, then the CSEM or MT foundation model 1208 can further trained at 1206 on additional and / or new CSEM or MT data 1218.

[0154] FIG. 13 is an exemplary flow diagram directed to implementing and / or using a pretrained CSEM / MT foundation model (e.g., saved pre-trained CSEM / MT foundation model at 1212 of FIG. 12) (or large CSEM / MT model) for a specific application. As described in FIG. 13, a pretrained large CSEM / MT foundation model can be applied for a downstream task, such as but not limited to, detection of subsurface conductivity anomalies, estimation of conductivity anomaly geometry, or quantitative evaluation of conductivity values of subsurface anomalies.

[0155] In one example, a method 1300 of implementing and / or using the pre-trained CSEM / MT foundation model for specific application (e.g., predicting conductivity anomaly information) is described. The method 1300 may begin at 1302 and proceed to loading CSEM / MTdata at 1304 (e.g., a second set of CSEM / MT data) and select a set or subset of CSEM / MT data with single or multiple frequency components at 1306. The method 1300 further includes loading the pre-trained large CSEM / MT foundation model at 1312 (e.g., saved pre-trained CSEM / MT foundation model at 1212 of FIG. 12) along with the selected CSEM / MT data with expert labels (e g., labeled training set (e.g., 1308) to train a downstream task model (e.g., anomaly detection model 1316) to generate predictions (e.g., 1318) regarding an anomaly geometry and / or conductivity values of anomalies from the anomaly detection model 1316, where such prediction 1318 may include, but is not limited to, conductivity anomaly information. In some cases, a subset of the foundation model (e.g., the pre-trained CSEM / MT foundation model saved at 1212 of FIG. 12) can be trained or fine-tune another subset of the foundation model. For example, the CSEM / MT foundation model (e.g., the pre-trained CSEM / MT foundation model saved at 1212 of FIG. 12) can be used to train the downstream task model (e.g., anomaly geometry and / or conductivity values of anomalies detection model).

[0156] In some cases, the downstream task model (e.g., anomaly detection model 1316) can be evaluated via a quality check 1320 (e.g., using unlabeled testing data 1310) to determine if the downstream task model (e.g., anomaly geometry and / or conductivity values of anomalies detection model 1316) is accurate. For example, the prediction 1318 of the model in training 1314 can be check at 1320 and if the prediction 1318 based on the unlabeled testing data 1310 is accurate or “good”, or meets or exceeds a quality threshold value, then the anomaly detection model 1316 can be saved at 1322 (e.g., for implementation) as a downstream task model. If the prediction 1318 is incorrect or “fair” or fails to meet or exceed a quality threshold value, then the anomaly detection model 1316 can undergo further training 1314 until a threshold is met during the quality check 1320 to save the downstream task model at 1322. The method 1300 may stop at 1324.

[0157] FIGS. 14A and 14B depict a flow diagram directed to implementing and / or using a first foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) and a second foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) (trained separately) for a joint downstream task. As an example, the flow diagram of FIGS. 14A and 14B illustrate building a marker propagation model 1416 based on a well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) (or large well log foundation model) and a stratigraphy model 1428 based on a seismic foundation model (e.g., thepre-trained seismic foundation model saved at 412 of FIG. 4) (or large seismic model) for a joint downstream task.

[0158] However, the first foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) and the second foundation model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) are not limited to these two types of foundation models and other foundation models based on different types of can be considered (e.g., CSEM / MT data) for performing a joint downstream task. The flow diagram of FIGS. 14A and 14B illustrate the usage of multiple foundation models for a joint downstream task involving multi-data modality (e.g., seismic and well log data).

[0159] That is, a seismic model (e.g., 408) (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) and a well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8) can be trained (e.g., separately) to perform ajoint downstream task, e.g., stratigraphy interpretation. For example, a method 1400 can start at 1402 and load 912 a pretrained well log foundation model (e.g., the pre-trained well log foundation model saved at 812 of FIG. 8), load well markers 906, and load well log data 804 to train 1414 a marker propagation model 1416, which can predict 1418 propagated markers. The predicted 1418 propagated markers can undergo a quality check 1420 to determine if the marker propagation model 1416 is predicting accurate results. In some instances, the training 1414 may utilize labeled training data 908 while the quality check 1420 may utilize unlabeled testing data 910. In some examples, if the predicted propagation markers are good or meets or exceeds a quality threshold value, the propagated markers can be saved at 1422 and provided (e.g., via continuation marker 1424), as input to train 1426 the stratigraphy model 1428. The method 1400 can load seismic volume data 404, load well log data 804, and utilize the pre-trained seismic model (e.g., the pre-trained seismic foundation model saved at 412 of FIG. 4) to train at 1426 the stratigraphy model 1428. Stratigraphic volumes can be predicted at 1430 and undergo an quality check at 1432 using the unlabeled testing set 1433, where if the quality is good or meets or exceeds a quality threshold value, the stratigraphic volume can be saved at 1434. Otherwise, the stratigraphy model 1428 can continue training at 1426. The method 1400 can then stop at 1436.

[0160] FIG. 15 depicts a flow diagram directed to building ajoint seismic-well log foundation model. In examples, the flow diagram depicts building (or training jointly) a multi-modalfoundation model involving multi -data modality (e.g., seismic, well log, modeling, etc. data). In examples, a method 1500 for building a joint seismic-well log model 1508 begins at 1502, which may be triggered by a user request, a scheduled task, or an automated process based on data availability. Upon starting, the method 1500, can proceed to loading seismic volume data and associated well log data 1504 and training 1506 a joint seismic-well log model 1508 based on the seismic volume data and associated well log data. In some examples, the training includes updating 1516 by loading new seismic volume data and associated well log data 1518 and continuing to train 1506 the joint seismic-well log model 1508. Once the joint seismic-well log model 1508 is trained, a quality check 1510 can be performed of the joint seismic-well log model 1508 to determine whether or not to save the joint seismic-well log model 1508. For example, if the quality check 1510 indicates that the joint seismic-well log model 1508 is “good” (or meets or exceeds a quality threshold value, then the joint seismic-well log model 1508 can be saved at 1512, e.g., for implementation, and the method 1500 can stop at 1514. If the quality check 1510 indicates that the joint seismic-well log model 1508 is “fair” or “poor” or fails to meet or exceed a quality threshold value, then the joint seismic-well log model 1508 can be further trained on additional and / or new seismic volume and associated well log data 1518.

[0161] FIG. 16 is an exemplary flow diagram directed to implementing and / or using a pretrained joint seismic-well log model (e.g., the pre-trained joint seismic-well log model 1508 saved at 1512 of FIG. 15) for a specific application. As illustrated in FIG. 16, the pre-trained large joint seismic-well log model (e.g., the pre-trained joint seismic-well log model 1508 saved at 1512 of FIG. 15) can be applied to a downstream task, e.g., stratigraphy interpretation. The flow diagram depicted in FIG. 16 is an example of a jointly-trained multi-modal foundation model applied to a downstream task involving multiple data modalities (e.g., seismic and well log). For example, the workflow illustrates using seismic volumes and well log data as inputs to a joint seismic-well log model.

[0162] As depicted in FIG. 16, a pre-trained joint seismic-well log model (e.g., the pre-trained joint seismic-well log model 1508 saved at 1512 of FIG. 15) can be applied to a downstream task, such as but not limited to stratigraphic volume predictions. In one example, a method 1600 of implementing and / or using the pre-trained foundation model (e.g., the pre-trained joint seismic- well log model 1508 saved at 1512 of FIG. 15) for specific application is described. In such a method 1600, the method 1600 can start at 1602 and proceed to load well log data at 904, seismicvolume data 404, and representative markers can be selected and / or annotated at 906. The method 1600 further includes loading 1612 the joint seismic-well log model (e.g., the pre-trained joint seismic-well log model 1508 saved at 1512 of FIG. 15) with the selected representative markers with expert labels (e.g., labeled training data 908) to train a downstream task model e.g., a stratigraphy model 1616 to predict 1618 stratigraphic volumes utilizing well log data and seismic volumes.

[0163] In some cases, the downstream task model (e.g., stratigraphy model 1616) can be evaluated via a quality check 1620 (e.g., using unlabeled testing data 1633) to determine if the downstream task model (e.g., stratigraphy model 1616) is accurate. For example, the prediction 1618 of the downstream task model (e.g., stratigraphy model 1616) in training can be checked and if the prediction 1618 based on the unlabeled testing data 1633 is accurate or “good” or meets or exceeds a quality threshold value, then the downstream task model can be saved at 1622 (e.g., for implementation) as a downstream task model. The method 1600 can then stop at 1624. If the prediction 1618 is incorrect or “fair” or fails to meet or exceeds a quality threshold value, then the downstream task model (e.g., stratigraphy model 1616) can undergo further training 1614 until a threshold is met during the quality check 1620 to save the downstream task model 1622.Example Method for Developing a Machine Learning Model

[0164] FIG. 17 depicts an example method 1700 obtaining a domain-specific model targeted for a downstream task. In one aspect, method 1700 can be implemented by the computing system 1800 of FIG. 18.

[0165] Method 1700 starts at block 1702 with obtaining a plurality of unlabeled geospatially- indexed datasets from different geographic locations representing different subsurface geologic conditions.

[0166] Method 1700 proceeds to block 1704 with training a neural network model on the plurality of unlabeled geospatially-indexed datasets in a self-supervised manner to obtain a foundation model that develops generalizable representations capturing shared interrelationships across the different subsurface geologic conditions.

[0167] Method 1700 then proceeds to block 1706 with obtaining a labeled geospatially- indexed training dataset specific to a designated survey area.

[0168] Method 1700 then proceeds to block 1708 with adapting parameters of the foundation model through retraining using the labeled geospatially-indexed training dataset to develop a domain-specific model targeted for a prediction task in the designated survey area.

[0169] Method 1700 then proceeds to block 1710 with implementing the domain-specific model on new geospatially-indexed input data from the designated survey area to generate predictions tied to locations for the targeted prediction task.

[0170] In some embodiments of method 1700, the plurality of unlabeled geospatially-indexed datasets comprises at least two of seismic survey datasets, well log datasets, or electromagnetic reading datasets.

[0171] In some embodiments of method 1700, training the neural network model comprises: corrupting the unlabeled geospatially-indexed datasets to generate training input-output pairs; and updating parameters of the neural network model using the training input-output pairs to reduce differences between inputs and reconstructed outputs of the neural network model in a selfsupervised manner. The training can be performed for a single data modality, such as seismic data, well log data, or electromagnetic data, or for multiple data modalities. In the case of multi-modal training, the foundation models can be trained separately for each data modality and then combined for a downstream task, or the foundation models can be trained jointly using a shared architecture that learns to extract features and representations from multiple data types simultaneously.

[0172] In some embodiments of method 1700, programmatically corrupting the unlabeled geospatially-indexed datasets comprises applying at least one transformation to the unlabeled geospatially-indexed data sets, wherein the at least one transformation includes at least one of adding of random noise, masking random slices, or injecting synthetic errors reflective of sensor mis-calibrations. Training the neural network in a self-supervised manner involves algorithmically corrupting the original unlabeled geospatially-indexed data to generate modified versions, then having the model attempt to reconstruct the originals from these altered inputs. By systematically distorting through additions of random noise, spatial masking, time-shifts or other perturbations, synthetic training exemplars are manufactured. The model optimizations then gradually improve reconstruction fidelity over multiple iterations by updating network parameters until the introduced corruptions are effectively nullified. This forces the model to internally develop rich implicit representations that capture structure within the unlabeled training data distributions required tocomplete the substitute supervised labeling tasks. Various algorithmic data corruption techniques can be utilized to expand diversity of the unlabeled data for improved model robustness. Addition of random noise across readings simulates realistic instrumental observation error. Masking of certain time or depth slices forces reliance on contextual inferences. Permuting the sequence order requires reassembly relying on data patterns. Synthetically injecting anomalies that mimic sensor mis-calibration artifacts induces stronger numeracy. Combining these and other synthetic constructions yields augmented variability in the self-supervised training data for developing more representations.

[0173] In some embodiments of method 1700, the foundation model includes a deep neural network model comprising at least one encoder portion configured to develop the generalizable representations capturing shared interrelationships.

[0174] In some embodiments of method 1700, the at least one encoder portion develops the generalizable representations by transforming the unlabeled geospatially-indexed datasets into compact feature embeddings that represent intrinsic subsurface properties reusable across the different geographic locations and subsurface geologic conditions for adaptation to new measurement types and survey areas. By training these encoders across diverse unlabeled geospatial datasets, they develop embeddings that encode essential characteristics generalizable across not just the training data but also to new unlabeled measurements. That is, the encoders learn projections into an embedding space that retains properties reusable even when analyzing out-of-sample inputs from unfamiliar surveys. The encodings form a dictionary of canonical representations capturing subsurface essence - stratigraphic discontinuities, structural repetition, textural anomalies etc.

[0175] In some embodiments of method 1700, obtaining the labeled geospatially-indexed training dataset comprises obtaining expert labels for at least one of seismic images, well logs, or electromagnetic survey outputs from the designated survey area.

[0176] In some embodiments of method 1700, adapting parameters of the foundation model comprises performing a few-shot learning process using the labeled geospatially-indexed training dataset to customize the foundation model for the designated survey area. Obtaining comprehensive labeled training data sufficient to train full foundation models is infeasible in geoscience domains. Few-shot learning instead adapts models using small labeling sets. Forexample, annotating merely 2-5 representative instances of seismic signatures, log traces or electromagnetic outcrops from a survey area can provide a miniature supervision set. This lighttouch labeling reduces resource overhead. By leveraging learnings accumulated within pretrained foundation models, customized solutions are rapidly trainable from scarce samples. The approach concentrates expertise: instead of exhaustively annotating many data exemplars, scientists provide time to label a fraction of the exemplars that would otherwise be needed to train a conventional foundation model.

[0177] In some embodiments, method 1700 further comprises progressively growing a geographic coverage of the domain-specific model by repeating the adapting of the parameters of the foundation model with additional labeled datasets.

[0178] Note that FIG. 17 is just one example of a method, and other methods including fewer, additional, or alternative steps are possible consistent with this disclosure.Example Computing System

[0179] FIG. 18 depicts an example computing system 1800 in accordance with some embodiments. The computing system 1800 can be an individual computer system 1801 A or an arrangement of distributed computer systems. The computer system 1801A includes one or more geosciences analysis modules 1802 that are configured to perform various tasks according to some embodiments, such as one or more methods disclosed herein. To perform these various tasks, geosciences analysis module 1802 executes independently, or in coordination with, one or more processors 1804, which is (or are) connected to one or more storage media 1806. The processor(s) 1804 is (or are) also connected to a network interface 1808 to allow the computer system 1801A to communicate over a data network 1810 with one or more additional computer systems and / or computing systems, such as 1801B, 1801C, and / or 1801D (note that computer systems 1801B, 1801C and / or 1801D may or may not share the same architecture as computer system 1801A, and may be located in different physical locations, e.g., computer systems 1801A and 1801B may be on a ship underway on the ocean, while in communication with one or more computer systems such as 1801C and / or 1801D that are located in one or more data centers on shore, other ships, and / or located in varying countries on different continents). Note that data network 1810 may be a private network, it may use portions of public networks, it may include remote storage and / or applications processing capabilities (e.g., cloud computing).

[0180] A processor can include a microprocessor, microcontroller, processor module or subsystem, programmable integrated circuit, programmable gate array, or another control or computing device.

[0181] The storage media 1806 can be implemented as one or more computer-readable or machine-readable storage media. Note that while in the example embodiment of Fig. 18 storage media 1806 is depicted as within computer system 1801A, in some embodiments, storage media 1806 may be distributed within and / or across multiple internal and / or external enclosures of computing system 1801 A and / or additional computing systems. Storage media 1806 may include one or more different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; optical media such as compact disks (CDs) or digital video disks (DVDs), BluRays or any other type of optical media; or other types of storage devices. Note that the instructions discussed above can be provided on one computer-readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine- readable storage media distributed in a large system having possibly plural nodes and / or non- transitory storage means. Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components. The storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.

[0182] It should be appreciated that computer system 1801 A is one example of a computing system, and that computer system 1801 A may have more or fewer components than shown, may combine additional components not depicted in the example embodiment of FIG. 18, and / or computer system 1801 A may have a different configuration or arrangement of the components depicted in FIG. 18. The various components shown in FIG. 18 may be implemented in hardware, software, or a combination of both, hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0183] It should also be appreciated that while no user input / output peripherals are illustrated with respect to computer systems 1801A, 1801B, 1801C, and 1801D, many embodiments of computing system 1800 include computer systems with keyboards, mice, touch screens, displays, etc. Some computer systems in use in computing system 1800 may be desktop workstations, laptops, tablet computers, smartphones, server computers, etc.

[0184] Further, the steps in the processing methods described herein may be implemented by running one or more functional modules in information processing apparatus such as general purpose processors or application specific chips, such as ASICs, FPGAs, PLDs, or other appropriate devices. These modules, combinations of these modules, and / or their combination with general hardware are included within the scope of protection.

[0185] Note that FIG. 18 is just one example of a processing system consistent with aspects described herein, and other processing systems having additional, alternative, or fewer components are possible consistent with this disclosure.Example Clauses

[0186] Implementation examples are described in the following numbered clauses:

[0187] Clause 1: A method comprising: obtaining a plurality of unlabeled geospatially- indexed datasets from different geographic locations representing different subsurface geologic conditions; training a neural network model on the plurality of unlabeled geospatially-indexed datasets in a self-supervised manner to obtain a foundation model that develops generalizable representations capturing shared interrelationships across the different subsurface geologic conditions; obtaining a labeled geospatially-indexed training dataset specific to a designated survey area; adapting parameters of the foundation model through retraining using the labeled geospatially-indexed training dataset to develop a domain-specific model targeted for a prediction task in the designated survey area; and implementing the domain-specific model on new geospatially-indexed input data from the designated survey area to generate predictions tied to locations for the targeted prediction task.

[0188] Clause 2: A method in accordance with Clause 1, wherein the training data comprises at least one of seismic data, well log data, or both seismic and well log data.

[0189] Clause 3 : A method in accordance with any one of Clauses 1-2, wherein training the neural network model comprises: corrupting the unlabeled geospatially-indexed datasets to generate training input-output pairs; and updating parameters of the neural network model using the training input-output pairs to reduce differences between inputs and reconstructed outputs of the neural network model in a self-supervised manner.

[0190] Clause 4: A method in accordance with Clause 3, wherein programmatically corrupting the unlabeled geospatially-indexed datasets comprises applying at least one transformation to the unlabeled geospatially-indexed data sets, wherein the at least one transformation includes at least one of adding of random noise, masking random slices, or injecting synthetic errors reflective of sensor mis-calibrations.

[0191] Clause 5: A method in accordance with any one of Clauses 1-4, wherein the foundation model includes a deep neural network model comprising at least one encoder portion configured to develop the generalizable representations capturing shared interrelationships.

[0192] Clause 6: A method in accordance with any one of Clauses 1-5, wherein obtaining the labeled geospatially-indexed training dataset comprises obtaining expert labels for at least one of seismic images, well logs, or electromagnetic survey outputs from the designated survey area.

[0193] Clause 7: A method in accordance with Clause 6, wherein adapting parameters of the foundation model comprises performing a few-shot learning process using the labeled geospatially-indexed training dataset to customize the foundation model for the designated survey area.

[0194] Clause 8: A method in accordance with any one of Clauses 1-7, further comprising progressively growing a geographic coverage of the domain-specific model by repeating the adapting of the parameters of the foundation model with additional labeled datasets.

[0195] Clause 9: A processing system, comprising: a memory comprising computerexecutable instructions; and a processor configured to execute the computer-executable instructions and cause the processing system to perform a method in accordance with any one of Clauses 1-8.

[0196] Clause 10: A processing system, comprising means for performing a method in accordance with any one of Clauses 1-8.

[0197] Clause 1 1 : A non-transitory computer-readable medium storing program code for causing a processing system to perform the steps of any one of Clauses 1-8.

[0198] Clause 12: A computer program product embodied on a computer-readable storage medium comprising code for performing a method in accordance with any one of Clauses 1-8.Additional Considerations

[0199] The preceding description is provided to enable any person skilled in the art to practice the various embodiments described herein. The examples discussed herein are not limiting of the scope, applicability, or embodiments set forth in the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments. For example, changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as appropriate. For instance, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Also, features described with respect to some examples may be combined in some other examples. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects set forth herein. In addition, the scope of the disclosure is intended to cover such an apparatus or method that is practiced using other structure, functionality, or structure and functionality in addition to, or other than, the various aspects of the disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0200] As used herein, a phrase referring to “at least one of’ a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, acc, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).

[0201] As used herein, the term “determining” encompasses a wide variety of actions. For example, “determining” may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like. Also, “determining” may include receiving (e.g., receiving information), accessing (e.g.,accessing data in a memory) and the like. Also, “determining” may include resolving, selecting, choosing, establishing and the like.

[0202] The methods disclosed herein comprise one or more steps or actions for achieving the methods. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Further, the various operations of methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software component(s) and / or module(s), including, but not limited to a circuit, an application specific integrated circuit (ASIC), or processor. Generally, where there are operations illustrated in figures, those operations may have corresponding counterpart means-plus- function components with similar numbering.

[0203] The following claims are not intended to be limited to the embodiments shown herein, but are to be accorded the full scope consistent with the language of the claims. Within a claim, reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. No claim element is to be construed under the provisions of 35 U.S.C. §112(f) unless the element is expressly recited using the phrase “means for” or, in the case of a method claim, the element is recited using the phrase “step for.” All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims.

Claims

CLAIMSWhat is claimed is:

1. A method for obtaining a domain-specific model targeted for a downstream task, comprising: obtaining a plurality of unlabeled geospatially-indexed datasets from different geographic locations representing different subsurface geologic conditions; training a neural network model on the plurality of unlabeled geospatially-indexed datasets in a self-supervised manner to obtain a foundation model that develops generalizable representations capturing shared interrelationships across the different subsurface geologic conditions; obtaining a labeled geospatially-indexed training dataset specific to a designated survey area; adapting parameters of the foundation model through retraining using the labeled geospatially-indexed training dataset to develop a domain-specific model targeted for a prediction task in the designated survey area; and implementing the domain-specific model on new geospatially-indexed input data from the designated survey area to generate predictions tied to locations for the targeted prediction task.

2. The method of Claim 1, wherein the plurality of unlabeled geospatially-indexed datasets comprises at least two of seismic survey datasets, well log datasets, or electromagnetic reading datasets.

3. The method of Claim 1, wherein training the neural network model comprises: corrupting the unlabeled geospatially-indexed datasets to generate training inputoutput pairs; and updating parameters of the neural network model using the training input-output pairs to reduce differences between inputs and reconstructed outputs of the neural network model in a self-supervised manner.

4. The method of Claim 3, wherein programmatically corrupting the unlabeled geospatially-indexed datasets comprises applying at least one transformation to the unlabeled geospatially-indexed datasets, wherein the at least one transformation includes at least one of adding of random noise, masking random slices, or injecting synthetic errors reflective of sensor mis-calibrations.

5. The method of Claim 1, wherein the foundation model includes a deep neural network model comprising at least one encoder portion configured to develop the generalizable representations capturing shared interrelationships.

6. The method of Claim 1, wherein obtaining the labeled geospatially-indexed training dataset comprises obtaining expert labels for at least one of seismic images, well logs, or electromagnetic survey outputs from the designated survey area.

7. The method of Claim 6, wherein adapting parameters of the foundation model comprises performing a few-shot learning process using the labeled geospatially-indexed training dataset to customize the foundation model for the designated survey area.

8. The method of Claim 7, further comprising progressively growing a geographic coverage of the domain-specific model by repeating the adapting of the parameters of the foundation model with additional labeled datasets.

9. A computing system, comprising: one or more processors; and a memory system comprising one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations, the operations comprising: obtaining a plurality of unlabeled geospatially-indexed datasets from different geographic locations representing different subsurface geologic conditions; training a neural network model on the plurality of unlabeled geospatially- indexed datasets in a self-supervised manner to obtain a foundation model thatdevelops generalizable representations capturing shared interrelationships across the different subsurface geologic conditions; obtaining a labeled geospatially-indexed training dataset specific to a designated survey area; adapting parameters of the foundation model through retraining using the labeled geospatially-indexed training dataset to develop a domain-specific model targeted for a prediction task in the designated survey area; and implementing the domain-specific model on new geospatially-indexed input data from the designated survey area to generate predictions tied to locations for the targeted prediction task.

10. The computing system of Claim 9, wherein the plurality of unlabeled geospatially- indexed datasets comprises at least two of seismic survey datasets, well log datasets, or electromagnetic reading datasets.

11. The computing system of Claim 9, wherein training the neural network model comprises: corrupting the unlabeled geospatially-indexed datasets to generate training inputoutput pairs; and updating parameters of the neural network model using the training input-output pairs to reduce differences between inputs and reconstructed outputs of the neural network model in a self-supervised manner.

12. The computing system of Claim 9, wherein the foundation model includes a deep neural network model comprising at least one encoder portion configured to develop the generalizable representations capturing shared interrelationships.

13. The computing system of Claim 9, wherein obtaining the labeled geospatially- indexed training dataset comprises obtaining expert labels for at least one of seismic images, well logs, or electromagnetic survey outputs from the designated survey area.

14. The computing system of Claim 13, wherein adapting parameters of the foundation model comprises performing a few-shot learning process using the labeled geospatially-indexed training dataset to customize the foundation model for the designated survey area.

15. The computing system of Claim 14, wherein the instructions, when executed by the at least one of the one or more processors, cause the computing system to perform operations further comprising progressively growing a geographic coverage of the domain-specific model by repeating the adapting of the parameters of the foundation model with additional labeled datasets.

16. A method for developing a machine learning model for a geoscience application, comprising: obtaining a first set of data representing subsurface properties or structures; training a foundation model using the first set of data in an unsupervised manner to learn representations of the subsurface; obtaining a second set of data and corresponding labels for a specific geoscience task; adapting the foundation model for the specific geoscience task using the second set of data and corresponding labels to generate a task-specific model; and deploying the task-specific model to generate predictions for new data related to the specific geoscience task.

17. The method of Claim 16, wherein the first set of data and the second set of data comprise seismic data.

18. The method of Claim 16, wherein adapting the foundation model comprises fine- tuning the foundation model using the second set of data and corresponding labels.

19. The method of Claim 16, further comprising: evaluating the task-specific model using a quality control process; and storing the task-specific model for future use when the task-specific model meets a quality threshold based on the quality control process.

20. The method of Claim 16, further comprising:obtaining a third set of data and corresponding labels for a second specific geoscience task; adapting the foundation model for the second specific geoscience task using the third set of data and corresponding labels to generate a second task-specific model; and deploying the second task-specific model to generate predictions for new data related to the second specific geoscience task, wherein the first set of data, the second set of data, and the third set of data comprise seismic data.

Images