Large language model parsing of scientific instrument workflow interruptions

Abstract

Systems / techniques are provided for facilitating large language model parsing of scientific instrument workflow interruptions. In various embodiments, the system may cause a scientific instrument (e.g., charged particle microscope, chromatograph, mass spectrometer) to perform a workflow on a sample. In various aspects, the system may retrieve runtime data recorded by the scientific instrument during the workflow in response to the scientific instrument generating an error message to interrupt the workflow. In various cases, the system may synthesize first text that interprets why the workflow is interrupted via execution of a large language model on error messages and runtime data.

Description

Background Technology

[0001] Historically, the field of scientific instruments has been limited by their operational complexity. This complexity can hinder the effective troubleshooting of errors experienced by scientific instruments. Summary of the Invention

[0002] The following summary is presented to provide a basic understanding of one or more embodiments. This summary is not intended to identify key or essential elements, or to depict any scope of a particular embodiment or any scope of the claims. Its sole purpose is to present the concepts in a simplified form as a prelude to the more detailed description that follows. In one or more embodiments described herein, devices, systems, computer-implemented methods, apparatuses, or computer program products are described that facilitate the parsing of large language models that disrupt scientific instrument workflows.

[0003] According to one or more embodiments, a system is provided. The system may include a non-transitory computer-readable storage memory that stores computer-executable components. The system may also include a processor that is operatively coupled to the non-transitory computer-readable storage memory and is executable of the computer-executable components stored in the non-transitory computer-readable storage memory. In various embodiments, the computer-executable components may include a workflow component that enables a charged particle microscope to perform a workflow on a sample. In various aspects, the computer-executable components may include a status component that, in response to the charged particle microscope generating an error message that interrupts the workflow, retrieves runtime data recorded by the charged particle microscope during the workflow. In various cases, the computer-executable components may include a model component that can synthesize first text explaining why the workflow was interrupted by performing a large language model on the error message and the runtime data.

[0004] According to one or more embodiments, a computer-implemented method is provided. In various embodiments, the computer-implemented method may include enabling a charged particle microscope to perform a workflow on a sample by a device operatively coupled to a processor. In various aspects, the computer-implemented method may include retrieving runtime data recorded by the charged particle microscope during the workflow by the device and in response to the charged particle microscope generating an error message that interrupts the workflow. In various cases, the computer-implemented method may include synthesizing first text explaining why the workflow was interrupted by the device and via performing a large language model on the error message and the runtime data.

[0005] According to one or more embodiments, a computer program product is provided for facilitating the parsing of large language models for scientific instrument workflow interruptions. In various embodiments, the computer program product may include a non-transitory computer-readable storage medium containing program instructions. In various aspects, the program instructions may be executed by a processor to cause the processor to cause the scientific instrument to perform a workflow on a sample. In various cases, the program instructions may be executed by a processor to cause the processor to retrieve runtime data recorded by the scientific instrument during the workflow in response to the scientific instrument generating an error message that interrupted the workflow. In various cases, the program instructions may be executed by a processor to cause the processor to synthesize first text explaining why the workflow was interrupted by performing a large language model on the error message and the runtime data. In various aspects, the scientific instrument may be a charged particle microscope, a chromatograph, or a mass spectrometer. Attached Figure Description

[0006] Various embodiments will be readily understood through the following detailed description taken in conjunction with the accompanying drawings. For ease of description, the same reference numerals indicate the same structural elements. The embodiments are illustrated in the figures by way of example rather than limitation. These figures are not necessarily drawn to scale.

[0007] Figure 1 Example non-limiting block diagrams of scientific instrument modules according to various embodiments described herein are shown.

[0008] Figure 2 Example non-limiting flowcharts illustrating computer-implemented methods according to various embodiments described herein are provided.

[0009] Figure 3 A block diagram of an example non-restricted system for facilitating the parsing of large language models that facilitate the interruption of scientific instrument workflows, according to one or more embodiments described herein, is illustrated.

[0010] Figure 4 A block diagram of an example non-restrictive system for facilitating the parsing of large language models that facilitate the interruption of scientific instrument workflows, according to one or more embodiments described herein, is illustrated. The system includes instrument workflows and error messages.

[0011] Figure 5 Example non-limiting block diagrams illustrating how error messages can interrupt instrument workflows according to one or more embodiments described herein are shown.

[0012] Figure 6 A block diagram of an example non-limiting system for facilitating the parsing of large language models that facilitate the interruption of scientific instrument workflows, according to one or more embodiments described herein, is illustrated, the system including instrument runtime data.

[0013] Figure 7Example non-limiting block diagrams illustrating instrument runtime data according to one or more embodiments described herein are shown.

[0014] Figure 8 A block diagram of an example non-limiting system for facilitating the parsing of large language models that facilitate the disruption of scientific instrument workflows, according to one or more embodiments described herein, is illustrated. The system includes a technical document library and solutions.

[0015] Figures 9-14 Example non-limiting block diagrams illustrating solutions for generating error messages based on instrument runtime data and technical documentation libraries, according to one or more embodiments described herein.

[0016] Figure 15 A block diagram of an example non-limiting system for facilitating the parsing of large language models that facilitate the interruption of scientific instrument workflows, according to one or more embodiments described herein, is illustrated. The system includes solution feedback and a set of technical document editors.

[0017] Figure 16 Example non-limiting block diagrams illustrating how one or more technical document edits can be generated based on solution feedback are provided according to one or more embodiments described herein.

[0018] Figure 17 Example non-limiting block diagrams illustrating how large language models can be trained according to one or more embodiments described herein are shown.

[0019] Figure 18 A block diagram illustrating an example non-limiting operating environment in which one or more embodiments described herein may be facilitated.

[0020] Figure 19 Example networking environments are illustrated that can operate to perform the various specific implementations described herein.

[0021] Figure 20 Example dual-beam microscopes that can be implemented according to the various implementation schemes described herein are illustrated. Detailed Implementation

[0022] The following detailed description is merely illustrative and is not intended to limit the implementation and / or application or use of the embodiments. Furthermore, there is no intention to be bound by any express or implied information presented in the preceding background or invention summary or detailed description sections.

[0023] One or more embodiments will now be described with reference to the accompanying drawings, wherein the same reference numerals are used throughout to refer to the same elements. In the following description, numerous specific details are set forth for purposes of explanation in order to provide a more thorough understanding of one or more embodiments. However, it will be apparent that in various cases one or more embodiments may be practiced without these specific details.

[0024] The various operations can be described sequentially as multiple discrete actions or operations in a manner most conducive to understanding the subject matter disclosed herein. However, the described order should not be construed as implying that these operations must depend on the order. Specifically, these operations may be performed in an order different from the order presented. The described operations may be performed in an order different from the described embodiments. Various additional operations may be performed, or the described operations may be omitted in additional embodiments.

[0025] While some elements may be represented in the singular (e.g., "processing device"), any suitable element may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as being performed by a processing device may be implemented as different operations of those operations performed by different processing devices. As used herein, the phrase "based on" should be understood to mean "at least partially based on," unless otherwise specified.

[0026] Scientific instruments are highly complex combinations of hardware and software that facilitate high-precision measurements of physical samples (e.g., semiconductor wafers, wafers, aqueous mixtures, biological tissues) in scientific, laboratory, research, or clinical operating environments through the complex arrangement of actuated components (e.g., ion sources, electron sources, optical lenses or apertures, optical plates or deflectors, columns, coils, heaters, coolers, fluid valves, fluid pumps, circuit switches, sample stages), sensors (e.g., ion detectors, electron detectors, voltmeters, thermistors, potentiometers, pressure gauges), or consumables (e.g., carrier fluids, calibrators, filters, reactive gases). For example, charged particle microscopes (e.g., scanning electron microscopes (SEM), transmission electron microscopes (TEM), electron energy loss microscopes (EELM)) are a type of scientific instrument that captures or generates microscopic or nanoscale images or energy spectra of physical samples. As another example, mass spectrometers that can be coupled to or mounted on a chromatograph are another type of scientific instrument that captures or generates ion abundance data (e.g., chromatograms, mass spectra) associated with the chemical or molecular composition of a physical sample.

[0027] Historically, the field of scientific instruments has been limited by their operational complexity. In other words, because scientific instruments can have such complex structures, their operation or use can be quite complex. In fact, for a user to be competent or confident in using a scientific instrument to analyze clinical or laboratory samples, they typically require extensive specialized training, education, or certification regarding the instrument. For example, a user might learn how to properly operate the graphical user interface (GUI) or physical controls of a charged particle microscope or mass spectrometer by studying instrument courses lasting several weeks or months. After all, a charged particle microscope or mass spectrometer may at first glance appear to have a large or overwhelming number of configurable software or hardware settings, buttons, knobs, sliders, or options. A user who has not undergone such extensive learning may not be able to operate a charged particle microscope or mass spectrometer effectively. For example, a user who attempts to operate a charged particle microscope or mass spectrometer without attending a proper instrument course lasting several weeks or months is likely to damage the sample or the instrument itself.

[0028] Given that different types of scientific instruments can operate or function differently from each other, this need for extensive learning or skills can be significantly amplified. That is, specialized instrument training, education, or certification is generally not transferable between different types of scientific instruments. For example, any training that prepares someone to operate a SEM does not necessarily prepare them to operate a TEM, and certainly not a mass spectrometer. As another example, any training that prepares someone to operate an SEM of type A does not necessarily prepare them to operate an SEM of type B. As yet another example, any training that prepares someone to operate an SEM of type A and software version C does not necessarily prepare them to operate an SEM of type A and software version D.

[0029] Due to the immense operational complexity of scientific instruments, users frequently experience and are unable to resolve hardware- or software-related malfunctions that occur during their operation. To troubleshoot or otherwise address such malfunctions, manufacturers typically provide technical support services to users. For example, manufacturers may employ numerous field engineers, technical experts, or research and development scientists to interact with users, with the goal of troubleshooting any instrument malfunctions encountered by those users. Depending on the severity or extent of any given malfunction (e.g., it might be a simple, recurring problem; or it might be a previously unseen software malfunction or hardware failure), these technical support services can consume an excessive amount of time (e.g., hours, days, or even weeks) to assess the given malfunction, collectively discuss solutions, and actually implement or execute those solutions to resolve the malfunction. These difficulties can amplify exponentially as the number of scientific instruments a manufacturer provides to different users increases.

[0030] Therefore, it may be desirable to have systems or techniques that can reduce the amount of time required to troubleshoot or resolve malfunctions that may occur during the operation of scientific instruments.

[0031] The various implementations described herein address this technical problem. One or more implementations described herein may include systems, computer-implemented methods, devices, or computer program products that facilitate the resolution of large language models (LLMs) that disrupt scientific instrument workflows. In other words, the various implementations described herein utilize large language models (LLMs) (such as ChatGPT) to quickly and efficiently troubleshoot scientific instrument malfunctions. In fact, when the various implementations described herein are implemented, errors, failures, or other malfunctions that disrupt scientific instrument workflows can be resolved in just seconds or minutes, rather than hours, days, or weeks. Therefore, users of such scientific instruments do not need to wait excessively long periods for troubleshooting services or repairs provided, collaboratively discussed, or developed by field engineers, technical experts, or research scientists. Instead, as described herein, LLMs can automatically provide, collaboratively discuss, or develop such troubleshooting services or repairs in real time, which can be considered desirable, beneficial, or advantageous.

[0032] The inventors have devised various embodiments for reducing the amount of time consumed in troubleshooting scientific instruments. As described herein, this reduction in time consumption can be achieved through specific implementations such as fault-triggered LLM analysis and synthesis; or LLM editing with feedback adjustment based on analysis references.

[0033] First, consider fault-triggered LLM resolving synthesis. In various implementations, a scientific instrument may be loaded with a given sample and may initiate or begin performing a given workflow on that sample (e.g., imaging workflow, injection workflow, milling workflow, spatial repositioning workflow, thermal conditioning workflow). In various ways, the given workflow may be interrupted by a specific fault. For example, the scientific instrument may throw or present one or more error codes that stop or pause the workflow. Rapid diagnosis, troubleshooting, or resolution of a specific fault is expected, but obtaining such diagnosis, troubleshooting, or resolution from field engineers, technical experts, or research scientists associated with the manufacturer of the scientific instrument can be highly time-consuming (e.g., potentially taking up to several weeks, depending on the complexity of the specific fault). Therefore, in various ways, the scientific instrument may respond to a specific fault by extracting or retrieving its runtime data associated with the given workflow (e.g., by extracting or retrieving any images or mass spectra captured by parts of the scientific instrument during the given workflow, by extracting or retrieving any values ​​that the controllable operating parameters of the scientific instrument were set to during the given workflow, or by extracting or retrieving any other data captured by any other sensors of the scientific instrument during the given workflow). In various scenarios, runtime data and error codes generated by scientific instruments can be fed into the LLM as collective input prompts, enabling the LLM to generate various synthetic text content about a specific fault. Some of this synthetic text content may be one or more natural language sentences describing or explaining what appears to cause the scientific instrument to experience a specific fault. Other portions of the synthetic text content may be one or more natural language sentences describing or explaining how to resolve, remedy, or correct a specific fault. Still other portions of the synthetic text content may be one or more lines of computer-executable code that, when executed, actually implements any software-related steps, actions, or modifications that will resolve, remedy, or correct a specific fault. In other words, error codes and runtime data generated by scientific instruments can be considered to convey (possibly in a hidden, uninterpretable, or otherwise obscure manner) at least a certain amount of measurable or quantifiable information unique to or specific to a particular fault, and the LLM can be considered to utilize such information to predict or infer how to resolve the specific fault. In some implementations, error codes and runtime data can be enhanced in a retrieved-enhanced generative (RAG) manner through relevant technical documentation associated with the scientific instrument (e.g., through the scientific instrument's operating manual, through the scientific instrument's design blueprints, through coding scripts or files associated with the scientific instrument). In any case, synthesized text content can be considered as teaching or explaining why a particular fault occurred or how to prevent it from occurring, and LLM can consume only a few seconds to generate such synthesized text content. Therefore, as described in this article, triggering LLM synthesis for fault-solving in response to a specific fault interrupting a given workflow can help ensure rapid resolution of specific faults.Compare this to the number of days or weeks that field engineers, technical experts, or research scientists typically spend troubleshooting complex malfunctions that plague their scientific instruments.

[0034] Next, consider the LLM editing of feedback regulation for parsing references. As mentioned above, the LLM can generate synthetic text content in response to a given workflow being interrupted by a specific fault. This synthetic text content explains or describes why (from the LLM's perspective) the specific fault occurred or how (again, from the LLM's perspective) the specific fault is resolved. Also as mentioned above, this synthetic text content can be based on one or more relevant technical documents associated with scientific instruments and considered as RAG references to the LLM. Now, in various ways, field engineers, technical experts, or research scientists can provide feedback on the synthetic text content. In various cases, this feedback can be natural language text written by the field engineer, technical expert, or research scientist, commenting on or otherwise identifying shortcomings or inaccuracies in the synthetic text content. In response, this feedback, the synthetic text content, and the RAG references upon which the LLM relies to generate the synthetic text content can all be fed to the LLM as collective input prompts, thereby causing the LLM to produce additional fragments of the synthetic text content. In various cases, the additional fragments of the synthesized text content can be one or more edits (e.g., word deletion, word insertion). If those edits have been incorporated into the corresponding RAG references in the RAG references before the synthesis text content was generated, these edits will result in the synthesized text content lacking or otherwise not possessing the defects or inaccuracies specified in the feedback. In other words, feedback can be thought of as precisely locating the specific inaccuracies in the LLM's synthesized troubleshooting analysis for a particular fault, and the LLM can use this feedback to: determine which parts of which RAG references cause the LLM to produce those specific inaccuracies; and synthesize edits to those parts of those RAG references to prevent the LLM from producing those same specific inaccuracies in the future. By enabling the LLM to infer how its RAG references should be edited to avoid identifying any analytical defects in the feedback, the LLM is more likely to provide accurate analytical inferences in response to future instrument failures. In some cases, this can be referred to as reverse RAG (e.g., RAG may involve synthesizing text content using relevant text references; in contrast, reverse RAG may involve editing relevant text references using feedback regarding the synthesized text content).

[0035] The various implementations described herein can be considered as computerized tools (e.g., any suitable combination of computer-executable hardware or computer-executable software) that facilitate the parsing of large language models that disrupt scientific instrument workflows. In various respects, such computerized tools may include workflow components, state components, model components, or execution components.

[0036] In various implementations, scientific instruments may be present. In some cases, the scientific instrument may be a charged particle microscope exhibiting any suitable design or construction (e.g., it may be a SEM, a TEM, an EELM, or a dual-beam microscope). In other cases, the scientific instrument may be a mass spectrometer equipped with a chromatograph, exhibiting any suitable design or construction (e.g., liquid chromatography hardware, gas chromatography hardware, ion chromatography hardware, matrix-assisted laser desorption / ionization (MALDI) hardware, electrospray ionization (ESI) hardware, quadrupole mass filter analyzer hardware, ion trap analyzer hardware, time-of-flight (TOF) analyzer hardware, electrostatic trap hardware). In various cases, the scientific instrument may include any suitable number of configurable operating settings. In various cases, configurable operating settings can be any suitable selective control hardware or software feature of the scientific instrument that can be directly adjusted or changed in response to electronic instructions or commands received from the user of the scientific instrument (e.g., user-controlled voltage or current settings, user-controlled temperature settings, or user-controlled actuator settings). In various aspects, any suitable sample (e.g., a semiconductor wafer or wafer for a charged particle microscope; an aqueous mixture for a mass spectrometer) can be present, currently loaded into the scientific instrument (e.g., currently located or positioned on the actuated stage of a charged particle microscope; currently inside the autosampler syringe of a mass spectrometer equipped with a chromatograph).

[0037] In various implementations, an LLM (Layered Learning Model) may exist. In various aspects, the LLM can exhibit any suitable deep learning internal architecture. For example, the LLM may include any suitable number of layers of any suitable type (e.g., input layer, one or more hidden layers, output layer, any of which may be a convolutional layer, dense layer, long short-term memory (LSTM) layer, transformer layer, nonlinear layer, pooling layer, batch normalization layer, or padding layer). As another example, the LLM may include any suitable number of neurons in various layers (e.g., different layers may have the same or different numbers of neurons). As yet another example, the LLM may include any suitable activation function in various neurons (e.g., softmax, sigmoid, hyperbolic tangent, modified linear unit) (e.g., different neurons may have the same or different activation functions). As yet another example, the LLM may include any suitable inter-neuron or inter-layer connections (e.g., forward connections, skip connections, recurrent connections).

[0038] Regardless of its specific internal architecture, the LLM can be configured as a generative text-to-text model. That is, the LLM can be configured to receive any suitable text data (which may or may not be accompanied by any suitable numerical data or any suitable graphical data) as input, and the LLM can be configured to produce synthetic text content (e.g., one or more synthetic sentences or sentence fragments) as output that is semantically or substantially based on such input text data (and based on accompanying numerical or graphical data, as appropriate).

[0039] To achieve this, the LLM can be considered to include an encoder section and a synthesizer section. In various respects, the encoder section can be any suitable upstream layer of the LLM configured to receive input text data (and any accompanying numerical or graphical data) and generate embeddings based on that input text data. In various cases, the synthesizer section can be any suitable downstream layer of the LLM configured to receive those embeddings and generate synthesized text content based on those embeddings.

[0040] In various respects, an embedding generated by the encoder portion of an LLM in response to a set of input text, numerical, or graphical data can be considered as any suitable mathematical quantity (e.g., scalar, vector, matrix, tensor, word granularity, or any suitable combination thereof) that numerically represents at least some substantial or semantic aspects of the input text, numerical, or graphical data in a low-dimensional manner. In other words, the embedding may be smaller in size or dimension than such input text, numerical, or graphical data (e.g., in some cases, one or more orders of magnitude smaller); however, despite its small size, the embedding can still be considered to substantially or semantically represent such input text, numerical, or graphical data. In yet another way, the embedding can be considered as a latent vector representation of such input text, numerical, or graphical data.

[0041] In any situation, LLM can be utilized to troubleshoot or resolve malfunctions experienced by scientific instruments. The computerized tools described herein can accomplish this in various cases.

[0042] In various implementations, computerized tools can access LLMs or scientific instruments electronically. For example, a computerized tool can connect to or communicate electronically with an LLM or scientific instrument (e.g., send electronic commands to it, read electronic signals from it). Therefore, any component of the computerized tool can interact electronically with the LLM or scientific instrument (e.g., read, write, edit, copy, manipulate, execute, activate, deactivate, modify).

[0043] In various ways, the workflow components of a computerized tool can electronically enable a scientific instrument to initiate or begin an instrument workflow for any sample currently loaded within it. In various ways, an instrument workflow can be any suitable sequence of one or more hardware or software operations that can be performed by the scientific instrument. For example, suppose the scientific instrument is a charged particle microscope. In this case, the instrument workflow could be: an imaging scheme configured or designed to capture microscopic or nanoscale images of the sample; a milling scheme configured or designed to mill or remove a certain amount of material from a designated location on the sample; a stage actuation scheme configured or designed to physically move or reorient the sample in space; or a vacuum scheme configured or designed to evacuate or degas the vacuum chamber containing the sample to a specific pressure level. As another example, suppose the scientific instrument is a mass spectrometer equipped with a chromatograph. In this context, the instrument workflow can be: a scanning or screening scheme configured or designed to capture a chromatogram or mass spectrometer of the sample; a rinsing scheme (if the sample is a clean mixture) configured to rinse the various channels or conduits of the mass spectrometer with the sample; or a temperature control scheme configured or designed to raise or lower the temperature of the various components of the mass spectrometer (e.g., oven heating column, autosampler syringe) to a specified temperature level. In some cases, the instrument workflow can be instructed or provided by the user of the scientific instrument. For example, the user can interact with any suitable human-machine interface of the scientific instrument (e.g., touchscreen, keyboard, voice control system) to notify the scientific instrument of the instrument workflow. In any case, the scientific instrument may initiate or attempt to execute an instrument workflow on the sample.

[0044] Instrument workflows can be interrupted by error messages in various ways. That is, a scientific instrument may abruptly or suddenly stop or pause its workflow and generate an error message while attempting to execute any sequence of hardware or software operations that constitute the instrument workflow. In other words, an instrument workflow may consist of multiple operations to be performed sequentially, and the scientific instrument may stop performing these multiple operations after executing any first or initial operation included in the instrument workflow but before executing any last or final operation included in the instrument workflow, and instead display an error message. In any case, an error message can be any suitable electronic data (explicitly or implicitly) indicating that the scientific instrument cannot complete or terminate the instrument workflow due to experiencing some malfunction. As some examples, error messages can be: structured text (e.g., alphanumeric identifiers) that corresponds in some way to or is associated with the occurrence of the malfunction; unstructured text (e.g., one or more natural language sentences or sentence fragments) that semantically describe or state the occurrence of the malfunction or some other detail; or visual symbols (e.g., prohibit / slash circle symbols, frown symbols) that visually indicate the occurrence of the malfunction or some other detail.

[0045] In various implementations, the status component of a computerized tool can electronically enable a scientific instrument to access or collect instrument runtime data in response to the generation of error messages. In various aspects, instrument runtime data can be any suitable electronic data generated, recorded, used, or otherwise tracked by the scientific instrument during partial execution or partial operation of its instrument workflow. As some examples, runtime data may include: any values ​​or states of configurable operating parameters of the scientific instrument during partial execution or partial operation of its instrument workflow; any imaging or spectral data partially captured by the scientific instrument during partial execution or partial operation of its instrument workflow; or measurements recorded by any auxiliary or accessory sensors (e.g., pressure sensors, cameras, microphones, temperature sensors, humidity sensors) incorporated into or built into the scientific instrument during partial execution or partial operation of its instrument workflow. Therefore, runtime data can be considered as conveying the operational status of the scientific instrument before and during the interruption of its instrument workflow due to a malfunction.

[0046] In various implementations, the model components of computerized tools can electronically generate solutions to faults by performing LLM on error messages and instrument runtime data.

[0047] More specifically, there may be causal prompts, which can be unstructured or plain text, that ask or command to determine what caused the scientific instrument to generate an error message. In various ways, model components can cascade error messages, instrument runtime data, and causal prompts together. In various cases, the model components can feed this cascade into the input layer of the LLM, which can perform a forward pass through one or more hidden layers of the LLM, and the output layer of the LLM can compute a first natural language response based on the activations provided by one or more hidden layers of the LLM.

[0048] In various scenarios, the first natural language response can be synthetic text based on error messages and instrument runtime data, and substantially or semantically responding to causal prompts. In other words, the first natural language response can be unstructured or plain text describing or explaining what (e.g., inferred or predicted by the LLM) has caused the scientific instrument to interrupt its workflow. Even further, the first natural language response can be a textual statement of a hardware-related or software-related failure of the scientific instrument that the LLM considers the root cause of the error message. That is, error messages and instrument runtime data can be considered as a snapshot of the scientific instrument's internal workings provided to the LLM during the scientific instrument's attempt to execute its workflow, and the LLM can use this snapshot to infer or predict which specific hardware or software of the scientific instrument failed and thus interrupted the workflow. In other words, the LLM can use any differences, inconsistencies, or other interrelationships between the internal workings the scientific instrument is expected to have during the execution of its workflow and the internal workings the scientific instrument actually has during the workflow to infer or predict why the workflow was interrupted. For example, if the instrument workflow is a specific type of imaging scheme or imaging scan, an LLM can expect the instrument runtime data to present or behave in a certain way (e.g., with or without certain visual, auditory, or digital patterns or artifacts). The degree or manner in which the instrument runtime data deviates from this expectation can be considered at least partially indicative of a problem with the scientific instrument to the LLM.

[0049] In some cases, a parsing prompt may be present, which can be unstructured or plain text, and its query or command determines how to parse or correct any hardware or software specified in the first natural language response. In various ways, model components can cascade error messages, instrument runtime data, the first natural language response, and the parsing prompt. In various cases, the model components can feed this cascade to the input layer of the LLM, which can complete the forward pass through one or more hidden layers of the LLM, and the output layer of the LLM can compute a second natural language response based on the activations provided by one or more hidden layers of the LLM.

[0050] In various scenarios, the second natural language response can be a composite text based on error messages, instrument runtime data, and the first natural language response, substantially or semantically responding to the parsed prompts. In other words, the second natural language response can be unstructured or plain text that describes or explains what hardware or software adjustments (such as those inferred or predicted by the LLM) will resolve any problems causing the scientific instrument to interrupt its workflow. Again, the second natural language response can be textually stating what sequence of maintenance, repair, or development actions (if performed on the scientific instrument) will resolve or remedy any hardware or software fault indicated in the first natural language response. That is, error messages and instrument runtime data can be considered as a snapshot of the scientific instrument's internal workings provided to the LLM during the scientific instrument's attempt to execute its workflow, the first natural language response can be considered as indicating a specific hardware or software failure that has been inferred to have interrupted the instrument workflow, and the LLM can use such information to infer or predict what specific remedial steps should be taken to repair such hardware or software failure. After all, different types of instrument workflows may experience corresponding types of hardware or software failures, and different types of hardware or software failures may require corresponding sequences of remedial steps.

[0051] In some cases, code hints may be present, which can be unstructured or plain text. These hints, through queries or commands, generate an encoded script whose execution will realize any software action or step specified in the second natural language response. In various ways, model components can cascade error messages, instrument runtime data, second natural language responses, and code hints. In various cases, the model components can feed this cascade into the input layer of the LLM, which completes the forward pass through one or more hidden layers of the LLM, and the output layer of the LLM can compute the encoded script based on the activations provided by one or more hidden layers of the LLM.

[0052] In various cases, the coding script can be written in any suitable programming syntax and can substantially or functionally satisfy code hints. In other words, the coding script can be a synthesized line of computer-executable code configured to execute or implement any remedial software action or operation specified in the second natural language response at runtime. Furthermore, the second natural language response can semantically interpret what software adjustments are made to the source code files of a scientific instrument to resolve any faults that disrupt the instrument's workflow, and the coding script can be configured to actually implement or execute those software adjustments.

[0053] Now, in some cases, the level of accuracy, completeness, or specificity or detail demonstrated by the first natural language response, the second natural language response, or the coded script can be increased or otherwise improved by allowing the LLM to consider supplementary information about the scientific instrument.

[0054] As a non-restrictive example, a document library may exist that includes multiple documents. In various cases, each of the multiple documents can be any suitable electronic file (e.g., a Word-doc file, a Portable Document Format (PDF) file, a web page file) that describes, explains, or otherwise indicates any suitable technical information regarding the design, manufacture, operation, maintenance, or troubleshooting of any suitable scientific instrument in textual form (or in some cases graphically or numerically). For example, the various documents may be maintenance manuals or technical manuals (or parts thereof) for some corresponding scientific instruments. In various cases, any of the multiple documents may have been written or has been written by or is written by a technician or engineer responsible for the design, development, prototyping, revision, manufacture, or research of any suitable scientific instrument (e.g., via any suitable word processing software, computer-aided design software, or quantitative analysis software). It should be noted that in some cases, any document may be presented or otherwise have any suitable length or size (e.g., one or several pages in length; dozens of pages in length; hundreds of pages in length). In any case, the model component can electronically search the document library for one or more documents substantially related to error messages or instrument operation data. In some aspects, the model component... This can be achieved through embedding search. For example, the encoder portion of an LLM can be used to generate specific embeddings for error messages and instrument runtime data; the encoder portion can be used to generate corresponding embeddings for each document in the document repository; and any document whose embedding is closest to or most similar to a specific embedding can be considered relevant to the error message or instrument runtime data. Therefore, in various ways, model components can cascade those relevant documents with error messages and instrument runtime data, and model components can generate a first natural language response, a second natural language response, or a coding script (in addition to any appropriate hints) by performing LLM on this amplified cascade. In various cases, relevant documents can be considered to provide the LLM with deeper or richer information about how the scientific instrument is known or expected to perform various instrument workflows, how the scientific instrument is known or expected to respond to various types of failures, how the scientific instrument should be repaired when affected by corresponding failures, or how fragments of the scientific instrument's source code should be written or formatted, and such deeper or richer information allows the LLM to make the first natural language response, the second natural language response, or the coding script more accurate or detailed. It should be understood or otherwise recognized that this can be considered an exploitation of RAG.

[0055] In any case, the first natural language response, the second natural language response, or the coded script can be considered together as a solution to any failure that disrupts the instrument workflow.

[0056] In various implementations, the solution generated by LLM can be considered easy or not difficult to implement (e.g., requiring only minor changes to software parameters or simple adjustments to hardware components). In this case, the execution component of the computerized tool can present the solution electronically to the user of the scientific instrument in any suitable manner. As a non-limiting example, the execution component can visually present the first or second natural language response on any suitable computer screen or computer monitor associated with the scientific instrument, allowing the user to view or read them. As another non-limiting example, the execution component can audibly play the first or second natural language response on any suitable speaker associated with the scientific instrument (e.g., via any suitable text-to-speech technology), allowing the user to hear or hear them. In this way, the computerized tool can be considered to tell or teach the user what caused the instrument workflow to be interrupted and how the instrument workflow can be repaired. Where the solution includes a coded script, the execution component can electronically execute (or can electronically instruct the scientific instrument to execute) the coded script. Thus, any software adjustments that the LLM inference would allow for the completion of the instrument workflow can be effectively implemented. After executing the coded script, in some cases, the workflow component can instruct the scientific instrument to retry or re-attempt the instrument workflow.

[0057] However, in other implementations, the solution generated by LLM can alternatively be considered difficult or not easily implemented (e.g., extensive editing or modification of the source code of a scientific instrument, extensive redesign of the physical structure of a scientific instrument). In this case, the execution component of the computerized tool can alternatively present the solution electronically to the field engineer, technical expert, or research scientist associated with the manufacturer of the scientific instrument in any suitable manner (e.g., via visual or auditory reproduction). In this case, the first and second natural language responses can be considered to eliminate the need for collaborative troubleshooting or resolution discussions among the field engineer, technical expert, or research scientist. Furthermore, in this case, the coding script can be considered to reduce the amount of manual coding required by the field engineer, technical expert, or research scientist to resolve any faults specified in the first natural language response. For example, instead of coding from scratch, the field engineer, technical expert, or research scientist can alternatively make any edits or changes they deem appropriate to the coding script. In any case, by providing field engineers, technical experts, or research scientists with first natural language responses, second natural language responses, and coded scripts, the amount of time or effort that field engineers, technical experts, or research scientists must spend troubleshooting scientific instruments (e.g., developing or releasing remedial patches or updates for scientific instruments) can be significantly reduced.

[0058] Currently, in some implementations, as described above, one or more technical documents associated with the design, manufacture, operation, or maintenance of scientific instruments can be fed to the LLM as RAG references to help the LLM synthesize solutions (e.g., first natural language responses, second natural language responses, or coded scripts) more reliably or accurately. In some aspects, field engineers, technical experts, or research scientists can provide feedback to computerized tools (e.g., via any suitable human-machine interface of the scientific instrument) regarding the appropriateness or accuracy of the solution. In various cases, the feedback can be any suitable electronic data (e.g., structured or unstructured text) that identifies or specifies one or more troubleshooting inaccuracies or errors present or contained in the solution. In various cases, model components can utilize this feedback to help prevent the LLM from recurring those same troubleshooting inaccuracies or errors in the future. Specifically, edit prompts, which can be unstructured or plain text, can exist that ask or command how to edit any RAG references upon which the LLM-generated solution relies to avoid any inaccuracies or errors specified in the feedback. In various aspects, model components can cascade the solution, RAG references, and edit prompts together. In various cases, the model components can feed this cascade into the input layer of the LLM, which can perform forward propagation through one or more hidden layers of the LLM, and the output layer of the LLM can compute one or more text edits based on the activations provided by one or more hidden layers of the LLM.

[0059] In various cases, one or more text edits can be synthetic semantic markup that substantially satisfies editing prompts. In other words, one or more text edits can be word deletions or word insertions, and if applied to RAG references before the LLM generates the solution, the solution will not have any troubleshooting inaccuracies or errors indicated in the feedback. Again, feedback can be considered as specifying which parts of the solution are incorrectly synthesized, the LLM can be considered as predicting or inferring which pages, paragraphs, or sentences of the RAG references cause it to generate or create those incorrect parts of the solution, and the LLM can be considered as further predicting or inferring how to linguistically edit or change those pages, paragraphs, or sentences of the RAG references to avoid the same errors, inaccuracies, or errors in the future. In some cases, this can be considered reverse RAG techniques (e.g., instead of using references to synthesize text, it can be considered as using feedback about the synthesized text to change or edit references). In any case, the execution component can electronically implement one or more text edits or apply them to any RAG references that the LLM relies on when synthesizing the solution. Therefore, the likelihood of future solutions synthesized by LLM being more correct or accurate is higher (e.g., one or more text edits can be considered as repairing any part of the RAG reference that caused the LLM to malfunction, thus making the LLM less likely to malfunction in the future).

[0060] In this way, computerized tools can be considered as facilitating rapid and efficient automated troubleshooting in response to disruptions in scientific instrument workflows. Therefore, the amount of time users of scientific instruments must wait for hardware or software modifications in response to instrument malfunctions can be significantly reduced, as can the amount of work instrument manufacturers must spend on collaborative discussions and development of such hardware or software modifications.

[0061] The various implementations described herein can be used to solve problems that are inherently technical (e.g., parsing large language models to facilitate disruptions in scientific instrument workflows), non-abstract, and cannot be performed by humans as a set of mental behaviors, using hardware or software. Furthermore, some of the processes performed can be executed by dedicated computers (e.g., electron microscopes such as SEM, TEM, or EELM; mass spectrometers coupled to liquid, gas, or ion chromatographs; artificial neural networks such as LLM) to perform actions defined in the field of scientific instrumentation.

[0062] For example, actions defined in this way may include: a device operatively coupled to a processor causing a scientific instrument (e.g., a charged particle microscope, mass spectrometer) to perform a workflow on a sample; the device retrieving runtime data recorded by the scientific instrument during the workflow, in response to the scientific instrument generating an error message that interrupts the workflow; and the device synthesizing first text explaining why the workflow was interrupted by performing a large language model on the error message and the runtime data. Actions defined in this way may also include: the device synthesizing second text explaining how to parse the workflow by performing the large language model on the first text. In various aspects, actions defined in this way may also include: the device synthesizing an encoded script configured to cause the scientific instrument to parse the workflow by performing the large language model on the second text; and the device causing the scientific instrument to execute the encoded script, thereby parsing the workflow. In various cases, the workflow may be: an imaging scan of the sample; a milling operation of the sample; movement of an actuable stage holding the sample; a spectral scan of the sample; or an injection or rinsing operation of the sample. In various respects, runtime data may include: a partial image of charged particles or a partial mass spectrum of the sample captured by the scientific instrument during the workflow; or the state of one or more configurable operating parameters of the scientific instrument during the workflow. In various cases, the large language model may be based on an embedding search performed through a technical document library associated with the design, manufacture, operation, or troubleshooting of the scientific instrument to retrieve an enhanced generative synthesis of the first text, and the defined tasks may also include: receiving feedback from a user or technician regarding the first text by the device; synthesizing a text edit of a document in the technical document library retrieved through the embedding search by the device and via performing the large language model on the feedback; and updating the technical document library by the device by inserting the text edit into the document.

[0063] Actions defined in this way are inherently computerized. In fact, scientific instruments such as charged particle microscopes (e.g., SEM, TEM, EELM, dual-beam microscopes) or mass spectrometers coupled to chromatographs (e.g., liquid chromatographs, gas chromatographs, ion chromatographs) are highly technical computerized devices that include specific computerized hardware (e.g., temperature sensors, pressure sensors, voltage sensors, ion beam emitters, electron beam emitters, focusing lenses, ion detectors, electron detectors, beam apertures, fluid valves, actuable sample stages). Without a computer, scientific instruments, the operations performed by them, and the electronic data they capture cannot be realized in any reasonable or feasible way through human thought or human pen and paper. Furthermore, artificial neural networks (e.g., LLMs) are also inherently computerized constructs, comprising specific software-oriented architectures (e.g., input layers, hidden layers, or output layers, any of which may consist of trainable or non-trainable internal parameters, such as convolutional layers or LSTM layers). Without a computer, artificial neural networks cannot be trained or executed in any reasonable or feasible way through human thought or human pen and paper.

[0064] Furthermore, the various implementation schemes described herein can incorporate a wide range of teachings related to the field of scientific instruments into practical applications. As mentioned above, scientific instruments are highly complex devices that typically require extensive specialized training, education, or certification to learn how to operate, maintain, or troubleshoot them. Therefore, when a scientific instrument malfunctions, the user may not be able to confidently troubleshoot the problem themselves and may be forced to wait for a technical service engineer hired by the scientific instrument manufacturer to perform the troubleshooting. Unfortunately, it can take hours or days before any technical service engineer becomes available or even considers the problem. Moreover, once a technical service engineer finally begins to consider the problem, the amount of time required for them to diagnose and resolve it can range from as little as a few minutes to as long as several weeks or months, depending on the complexity of the problem. In fact, low-complexity problems can be easily resolved by changing only the values ​​of a few configurable operating parameters of the scientific instrument, which may only take a few minutes; medium-complexity problems may require more extensive editing of the scientific instrument's source code, which may take several days; and high-complexity problems may require creating an entire new page of source code from scratch (or even redesigning the scientific instrument's hardware), which may take weeks or months.

[0065] The various implementation schemes described in this paper can help improve this problem by enabling large language model parsing of scientific instrument workflow interruptions. That is, the various implementation schemes described in this paper can utilize LLM to reduce the amount of time or effort spent troubleshooting scientific instrument malfunctions. As described in this paper, this goal can be achieved through interruption-triggered LLM parsing synthesis; or LLM editing with feedback regulation based on RAG references.

[0066] In practice, when executing workflows (e.g., imaging scans, spectral scans, sample repositioning operations, sample heating or cooling operations), scientific instruments may be abruptly or suddenly interrupted by error messages, thus preventing the workflow from being completed. The various embodiments described herein can respond to such interruptions by collecting any runtime data generated or utilized by the scientific instrument during the partial execution of the workflow (e.g., partial or incomplete images, partial or incomplete spectra, currently assigned operating parameter values, auxiliary sensor feeds), and can feed both the error message and runtime data as input to the LLM. As described herein, when such input is accompanied by appropriate prompts (e.g., causal prompts, parsing prompts, and coding prompts as described herein), this allows the LLM to synthesize text content for troubleshooting the error message. In some cases, the synthesized text content may explain what appears to be causing the error message. In other cases, the synthesized text content may explain how to correct or eliminate the underlying cause of the error message. In even more cases, the synthesized text content may include executable lines of computer code configured to correct or eliminate the underlying cause of the error message. In any case, such synthesized text content may be created within just a few seconds after the error message occurs. Compare this to the hours, days, weeks, or even months that a field engineer or technical service specialist might spend assessing, working with, and developing solutions to, fixing, or patching error messages. Additionally, as described herein, LLMs can generate such synthetic text content by relying on one or more RAG references containing technical information about scientific instruments. Sometimes, RAG references can become incorrectly outdated. In some aspects, various implementations may involve using an LLM to update or edit the RAG references in response to feedback indicating that the synthetic text content is unsatisfactory to some extent. This concept can be considered a reverse RAG and can help prevent the LLM from repeatedly identifying any troubleshooting errors based on feedback.

[0067] Furthermore, it must be emphasized how ingenious and counterintuitive the various embodiments described herein are. In fact, the various embodiments described herein can be considered as modulated LLM synthesis of troubleshooting suggestions or solutions for runtime data collected by scientific instruments during interrupted or incomplete workflows. In some cases, such runtime data may include incomplete or partially captured electronic images, incomplete or partially captured chromatograms, or incomplete or partially captured mass spectra. Conventionally, such incomplete or partially captured data is considered corrupted, meaningless, and otherwise useless. Therefore, conventional techniques discard such incomplete or partially captured data. In stark contrast, the inventors of the various embodiments described herein recognized that, despite being incomplete, such incomplete or partially captured data can still contain potentially valuable clues about what caused the interruption of the scientific instrument's workflow. In other words, the inventors recognize that any particular type of failure experienced by a scientific instrument can leave hidden traces, markers, or patterns in such incomplete or partially captured data, and LLM can utilize such hidden traces, markers, or patterns to generate troubleshooting inferences about the scientific instrument (e.g., different types of failures and therefore different types of failure analysis can cause incomplete or partially captured data to be presented or represented in a corresponding manner). Because conventional wisdom teaches that such incomplete or partially captured data is useless and should be discarded, the various implementations described herein that utilize such incomplete or partially captured data to reduce the amount of time or effort required for troubleshooting can be considered highly counterintuitive, unexpected, unusual, or ingenious.

[0068] For at least the reasons stated above, the various implementation schemes described herein can be considered as solutions to or improvements to various problems or shortcomings in the troubleshooting of scientific instruments. Therefore, the various implementation schemes described herein can be considered as concrete and practical technical improvements in the field of scientific instruments. Consequently, the various implementation schemes described herein certainly qualify as useful and practical applications of computers.

[0069] Furthermore, the various embodiments described herein can control real-world tangible devices based on the disclosed teachings. For example, the various embodiments described herein can electronically activate, deactivate, or otherwise actuate real-world hardware (e.g., ion beam emitter, ion focusing lens, liquid-carrying valve / pump) of real-world scientific instruments (e.g., SEM, TEM, EELM, dual-beam microscope, mass spectrometer, gas chromatograph).

[0070] Figure 1 Example non-limiting block diagrams of scientific instrument module 102 according to various embodiments described herein are shown.

[0071] In various implementations, the scientific instrument module 102 may be implemented by circuitry (such as a programmed computing device), including, for example, electrical or optical components. The logic components of the scientific instrument module 102 may be included in a single computing device or, depending on the circumstances, distributed across multiple computing devices communicating with each other. (References herein) Figure 18 and Figure 20 Examples of computing devices that can implement the scientific instrument module 102 individually or in combination are discussed, and references are made to... Figure 19 Examples of systems or networks of interconnected computing devices that enable the scientific instrument module 102 to be implemented across one or more computing devices in a computer device are discussed.

[0072] Scientific instrument module 102 may include a first logic unit 104, a second logic unit 106, and a third logic unit 108. As used herein, the term "logic unit" may include means for performing a set of operations associated with the logic unit. For example, any logic element included in scientific instrument module 102 may be implemented by one or more computing devices programmed with instructions to cause one or more processing devices of the computing device to perform an associated set of operations. In a particular embodiment, a logic element may include one or more non-transitory computer-readable media having instructions on them that, when executed by one or more processing devices of the one or more computing devices, cause the one or more computing devices to perform an associated set of operations. As used herein, the term "module" may refer to a series of one or more logic elements that together perform the functions associated with the module. Different logic elements in a module may take the same form or may take different forms. For example, some logic elements in a module may be implemented by a programmed general-purpose processing device, while other logic elements in the module may be implemented by an application-specific integrated circuit (ASIC). In another example, different logic elements in a module may be associated with different sets of instructions executed by one or more processing devices. A module may omit one or more logic elements in the associated diagrams; for example, when the module will perform a subset of the operations discussed herein with reference to the module, the module may include a subset of the logic elements depicted in the associated figures.

[0073] In various embodiments, a scientific instrument corresponding to scientific instrument module 102 may be present. In various aspects, the scientific instrument may be any suitable computerized device capable of electronically measuring some scientifically relevant, clinically relevant, or research-related characteristics, properties, or attributes of an analytical sample (e.g., a known or unknown mixture, compound, or series of substances). As a non-limiting example, the scientific instrument may be a scanning electron microscope. In this case, the scientific instrument can measure or determine the surface morphology of the analytical sample. As another non-limiting example, the scientific instrument may be a transmission electron microscope. In this case, the scientific instrument can measure or determine the internal structural details of the analytical sample. As yet another non-limiting example, the scientific instrument may be an electron energy loss microscope. In this case, the scientific instrument can measure or determine the positional count or intensity of the analytical sample within a series of defined energy loss bins or bands. As a more general non-limiting example, the scientific instrument may be any suitable type of charged particle microscope (e.g., some types of microscopes may use non-electron ion beams to capture images or energy spectra or otherwise interact with the sample). As another non-limiting example, the scientific instrument may be a mass spectrometer operatively coupled to a chromatograph. In this context, scientific instruments can measure or determine the chromatogram (e.g., relative compound abundance as a function of retention time) or ion spectrum (e.g., relative ion abundance as a function of mass-to-charge ratio) of an analytical sample.

[0074] In various implementations, the first logic component 104 enables the scientific instrument to perform a workflow on the sample (e.g., imaging scan, spectral screening, milling, heating, cooling, repositioning).

[0075] In various implementations, the second logic component 106 may involve retrieving runtime data (e.g., partially captured images, partially captured spectra, operating parameter values) recorded by the charged particle microscope during the workflow in response to an error message generated by the scientific instrument that interrupts the workflow.

[0076] In various implementations, the third logic component 108 may involve synthesizing first text explaining why the workflow was interrupted by performing a large language model on error messages and runtime data. In various aspects, the third logic component 108 may also include synthesizing second text explaining how the workflow is parsed by performing a large language model on the first text. In various cases, the third logic component 108 may also include synthesizing an encoded script by the device and by performing the large language model on the second text, the encoded script being configured to cause the scientific instrument to parse the workflow. In some cases, the large language model may be based on an embedding search performed through a technical document library associated with the design, manufacture, operation, or troubleshooting of the scientific instrument to retrieve an enhanced generative synthesis of the first text, second text, or encoded script. In this case, the third logic component 108 may also include: receiving feedback from a user or technician regarding the first text, second text, or encoded script; synthesizing a text edit of a document in the technical document library retrieved through the embedding search by performing the large language model on the feedback; and updating the technical document library by inserting the text edit into the document.

[0077] Therefore, the scientific instrument module 102 can facilitate the parsing of large language models that interrupt scientific instrument workflows.

[0078] Figure 2 This is an example non-limiting flowchart of a computer-implemented method 200 according to various embodiments described herein. The operation of the computer-implemented method 200 can be used in any suitable context to perform any suitable operation (e.g., by [the relevant authority]). Figure 1 , Figure 18 , Figure 19 and Figure 20 (This can be performed or used in conjunction with any of the various modules, computing devices, or graphical user interfaces mentioned above). Figure 2 In this context, operations are each instantiated once in a specific order, but they can be reordered or repeated as needed and as appropriate (e.g., different operations can be executed in parallel where appropriate).

[0079] In various aspects, action 202 may include performing a first operation, namely, enabling a scientific instrument (e.g., a charged particle microscope) to perform a workflow on a sample by a device operatively coupled to the processor. In various cases, the first logic unit 104 may perform or otherwise facilitate action 202.

[0080] In various aspects, action 204 may include performing a second operation, namely, retrieving runtime data recorded by the scientific instrument during the workflow, in response to the device generating an error message that interrupts the workflow. In various cases, the second logic component 106 may perform or otherwise facilitate action 204.

[0081] In various cases, action 206 may include performing a third operation, namely, synthesizing a first text explaining why the workflow was interrupted by the device and by performing a large language model on the error message and the status data.

[0082] Therefore, the computer-implemented method 200 can facilitate the parsing of large language models that disrupt scientific instrument workflows.

[0083] Figure 3 A block diagram of an example non-restricted system for facilitating the parsing of large language models that facilitate the interruption of scientific instrument workflows, according to one or more embodiments described herein, is illustrated.

[0084] In various embodiments, scientific instrument 302 may be present. In various aspects, scientific instrument 302 may be as described above. That is, scientific instrument 302 may be any suitable computerized device that can utilize its constituent hardware (e.g., electron source, anode, condenser lens, condenser aperture, scanning coil, objective lens, objective aperture, deflector, condenser, astigmatism reducer, electron detector, X-ray detector, actuable sample stage, autosampler injector, oven heating column, mass analyzer, absorbent packaging material, fixed phase film, vacuum chamber, fluid conduit) to electronically capture any suitable type of measurable or quantifiable data from any suitable analytical sample. As a non-limiting example, scientific instrument 302 may be any suitable type of charged particle microscope (e.g., SEM; TEM; EELM; dual-beam microscope) that can electronically capture or generate any suitable type of charged particle images of the surface or internal portions of the analytical sample. As another non-limiting example, scientific instrument 302 may be any suitable chromatograph (e.g., liquid chromatograph; gas chromatograph; ion chromatograph; thin-layer chromatograph; affinity chromatograph) or mass spectrometer (e.g., quadrupole mass spectrometer; time-of-flight mass spectrometer; ion trap mass spectrometer; Fourier transform ion cyclotron resonance mass spectrometer; sector magnetic field mass spectrometer) that can electronically capture any suitable type of compositional ion abundance data (e.g., chromatogram, mass spectrometry) associated with the analyte sample.

[0085] Although not explicitly shown in the figures, scientific instrument 302 can be electronically integrated with any suitable human-machine interface device, either remotely or locally within the scientific instrument 302. Thus, users or technicians associated with scientific instrument 302 can interact with or otherwise control the scientific instrument. Some non-limiting examples of human-machine interface devices may be a keyboard, a keypad, a touchscreen, or a voice command system for scientific instrument 302.

[0086] In any case, scientific instrument 302 may include a plurality of configurable operating parameters 304. In various aspects, each of the plurality of configurable operating parameters 304 may be any suitable hardware-related or software-related characteristic of scientific instrument 302 that directs, influences, or otherwise instructs how scientific instrument 302 operates, functions, or works with respect to any given analytical sample, and may be selectively controlled, altered, adjusted, or otherwise set by a user or technician (e.g., via interaction with a human-machine interface device of scientific instrument 302). As a non-limiting example, any of the plurality of configurable operating parameters 304 may be a user-controllable voltage setting (e.g., beam voltage) or current setting (e.g., beam current) that allows a user or technician to selectively control the electrodes of scientific instrument 302 to selectively increase or decrease the voltage or current within or applied by scientific instrument 302. As another non-limiting example, any of the configurable operating parameters 304 could be a user-controllable temperature setting that allows a user or technician to control the heaters (e.g., stage heaters, heating coils) or coolers (e.g., cooling fans, heat pumps, refrigerators) of the scientific instrument 302 to selectively increase or decrease the temperature within or applied by the scientific instrument 302. As yet another non-limiting example, any of the configurable operating parameters 304 could be a user-controllable mechanical actuator setting that allows a user or technician to control the mechanical actuators (e.g., electric motors, sample stages, aperture diaphragms, fluid pumps or syringes, sample processing robotic arms) of the scientific instrument 302 to selectively move the mechanical actuators. As another non-limiting example, any of the configurable operating parameters 304 may be user-controllable optical device settings that allow a user or technician to control the optical elements of the scientific instrument 302 (e.g., optical lenses, optical deflectors) to selectively change the optical quality (e.g., focal spot size or position, astigmatism, defocus) applied by the scientific instrument 302.

[0087] In various respects, scientific instrument 302 may have a currently loaded sample 306 or otherwise associated with a currently loaded sample. In various cases, the currently loaded sample 306 may (as its name implies) be currently loaded on or within scientific instrument 302. As a non-limiting example, in the case where scientific instrument 302 is a charged particle microscope, the currently loaded sample 306 may be currently positioned, located, or otherwise attached to an actuable stage of scientific instrument 302 such that the currently loaded sample 306 can be analyzed or scanned by scientific instrument 302. As another non-limiting example, in the case where scientific instrument 302 is a chromatograph or mass spectrometer, the currently loaded sample 306 may be currently positioned, located, or otherwise contained within an autosampler syringe of scientific instrument 302 such that the currently loaded sample 306 can be analyzed or scanned by scientific instrument 302. In various cases, the currently loaded sample 306 may be any suitable type of medical, clinical, scientific, or laboratory sample exhibiting any suitable physical, chemical, compositional, or other properties, attributes, or characteristics. As a non-limiting example, the currently loaded sample 306 may be a thin sheet taken from a semiconductor substrate or wafer. As another non-limiting example, the currently loaded sample 306 may be a sample or fragment taken from a failed support structure. As yet another non-limiting example, the currently loaded sample 306 may be an aqueous solution or mixture containing any suitable type of solvent or solute.

[0088] In various implementations, a large language model 308 (hereinafter referred to as "LLM 308") may exist. In various aspects, LLM 308 may include an encoder portion 310 and a synthesizer portion 312. In various cases, encoder portion 310 may be considered upstream of synthesizer portion 312. Equivalently, synthesizer portion 312 may be considered downstream of encoder portion 310.

[0089] In various respects, the encoder section 310 can exhibit any suitable deep learning internal architecture. In fact, in various cases, the encoder section 310 can have an input layer, one or more hidden layers, and an output layer. In various cases, any of these layers can be coupled together by any suitable inter-neuron or inter-layer connections, such as forward connections, skip connections, or recurrent connections. Furthermore, in various cases, any of these layers can be any suitable type of neural network layer with any suitable learnable or trainable internal parameters. For example, any of such an input layer, one or more hidden layers, or output layer can be a convolutional layer, whose learnable or trainable parameters can be convolutional kernels. As another example, any of such an input layer, one or more hidden layers, or output layer can be a dense layer, whose learnable or trainable parameters can be weight matrices or bias values. As yet another example, any of such an input layer, one or more hidden layers, or output layer can be a batch normalization layer, whose learnable or trainable parameters can be shift factors or scaling factors. As another example, any of such an input layer, one or more hidden layers, or an output layer can be an LSTM layer, whose learnable or trainable parameters can be an input state weight matrix or a hidden state weight matrix. As yet another example, any of such an input layer, one or more hidden layers, or an output layer can be a transformer layer, whose learnable or trainable parameters can be single-head or multi-head attention blocks or other weight matrices. Furthermore, in various cases, any of such layers can be any suitable type of neural network layer with any suitable fixed or non-trainable internal parameters. For example, any of such an input layer, one or more hidden layers, or an output layer can be a nonlinear layer, a padding layer, a pooling layer, or a cascaded layer.

[0090] Similarly, in various cases, the synthesizer section 312 can exhibit any suitable deep learning internal architecture. In fact, in various cases, the synthesizer section 312 can have an input layer, one or more hidden layers, and an output layer. In various cases, any of these layers can be coupled together by any suitable inter-neuron or inter-layer connections (e.g., forward connections, skip connections, or recurrent connections). Furthermore, in various cases, any of these layers can be any suitable type of neural network layer with any suitable learnable or trainable internal parameters (e.g., any of such an input layer, one or more hidden layers, or output layer can be a convolutional layer, a dense layer, a batch normalization layer, an LSTM layer, or a transformer layer). Furthermore, in various cases, any of such layers can be any suitable type of neural network layer with any suitable fixed or untrainable internal parameters (e.g., any of such an input layer, one or more hidden layers, or output layer can be a non-linear layer, a padding layer, a pooling layer, or a cascaded layer).

[0091] Regardless of the specific internal architecture implemented within encoder section 310 (e.g., the specific number, type, or organization of layers), encoder section 310 can be configured to receive text data (which may be accompanied by any suitable numerical or graphical data) and generate embeddings based on such input text data. In contrast, regardless of the specific internal architecture implemented within synthesizer section 312, synthesizer section 312 can be configured to receive embeddings generated by encoder section 310 and generate synthesized text content based on such embeddings. As some non-limiting examples, LLM 308 can be any of the following: ChatGPT; Gene.AI; Ollam; Bard; or Claude.

[0092] In various implementations, system 314 may be electronically integrated with scientific instrument 302 or with LLM 308 (e.g., via any suitable wired or wireless electronic connection). As described herein, system 314 may utilize LLM 308 to expedite troubleshooting of scientific instrument 302 (e.g., reducing the days, weeks, or months of troubleshooting or solution development to just seconds or minutes).

[0093] In various aspects, system 314 may include processor 316 (e.g., computer processing unit, microprocessor) and non-transitory computer-readable storage 318 operatively or communicatively connected or coupled to processor 316. Non-transitory computer-readable storage 318 may store computer-executable instructions that, when executed by processor 316, cause processor 316 or other components of system 314 (e.g., workflow component 320, state component 322, model component 324, execution component 326) to perform one or more actions. In various embodiments, non-transitory computer-readable storage 318 may store computer-executable components (e.g., workflow component 320, state component 322, model component 324, execution component 326), and processor 316 may execute the computer-executable components.

[0094] In various implementations, system 314 can access scientific instrument 302 or LLM 308 electronically. That is, system 314 can communicate electronically with or otherwise interact electronically with scientific instrument 302 or LLM 308 (e.g., transmit electronic instructions or commands to it, receive electronic data from it) in any suitable manner. Therefore, any suitable component of system 314 can interact with, communicate with, or otherwise manipulate scientific instrument 302 or LLM 308. It should be noted that in some cases, system 314 may be implemented or hosted on scientific instrument 302 itself or on any suitable computerized workstation associated with or coupled to scientific instrument 302. In this case, system 314 may be considered to be deployed in a client-side manner (e.g., system 314 may be considered local to scientific instrument 302). However, in other cases, system 314 may alternatively be implemented or hosted remotely from scientific instrument 302, such as in a cloud computing environment. In this case, system 314 may be considered to be deployed in a server-side manner.

[0095] In various implementations, system 314 may include workflow component 320. In various aspects, as described herein, workflow component 320 enables scientific instrument 302 to initiate instrument workflows.

[0096] In various implementations, system 314 may include a status component 322. In various cases, as described herein, status component 322 may collect runtime data generated or utilized by scientific instrument 302 in response to an interruption of the instrument workflow by an error message.

[0097] In various implementations, system 314 may include model component 324. In various cases, as described herein, model component 324 may utilize runtime data and LLM 308 to troubleshoot error messages that interrupt instrument workflow.

[0098] In various implementations, system 314 may include execution component 326. In various ways, as described herein, execution component 326 may present, share, or otherwise implement any troubleshooting results generated by LLM.

[0099] It should be noted that, in various cases, workflow component 320, state component 322, model component 324, and execution component 326 can be collectively considered as one or more software components 319 of system 314. In various respects, it should be understood that, for ease of explanation and illustration, one or more software components 319 are generally described herein as comprising four components (e.g., workflow component 320, state component 322, model component 324, and execution component 326). However, one or more software components 319 are not limited to being implemented as exactly four components in every embodiment. In fact, in some embodiments, the functionality of such four components described herein can be combined in any suitable manner to be implemented in fewer than four components or by fewer than four components (e.g., in some cases, a single component can perform all the functionality described herein with respect to workflow component 320, state component 322, model component 324, and execution component 326). In other embodiments, the functionality described herein for such four components may alternatively be distributed, separated, split, or segmented in any suitable manner so as to be implemented in or by more than four components (e.g., two or more components may facilitate functionality that can be performed by workflow component 320; two or more components may facilitate functionality that can be performed by state component 322; two or more components may facilitate functionality that can be performed by model component 324; two or more components may facilitate functionality that can be performed by execution component 326).

[0100] Figure 4 A block diagram of an example non-limiting system for facilitating the parsing of large language models that can disrupt scientific instrument workflows, according to one or more embodiments described herein, is illustrated. The system includes instrument workflows and error messages.

[0101] In various embodiments, workflow component 320 may electronically command, electronically instruct, or otherwise electronically initiate, set initiate, or otherwise commence the execution of instrument workflow 402. In various aspects, instrument workflow 402 may be any suitable sequence of one or more hardware or software operations that can be performed, implemented, or carried out by scientific instrument 302. As a non-limiting example, in the case where scientific instrument 302 is a charged particle microscope, instrument workflow 402 may be a charged particle imaging scheme. In this case, instrument workflow 402 may be considered any sequence of operations or actions (e.g., determining the desired focal spot size, determining the desired beam voltage or current, determining the desired beam grating pattern) that will generate a desired image of the surface or internal volume of the currently loaded sample 306. As another non-limiting example, in the case where scientific instrument 302 is a charged particle microscope, instrument workflow 402 may be a milling scheme. In this context, instrument workflow 402 can be considered as any sequence of operations or actions that specifies the removal or etching of a desired amount of material from the currently loaded sample 306 (or possibly the deposition of a desired amount of material onto the currently loaded sample) (e.g., reducing the focal spot size, increasing the beam voltage or current, aiming the beam at a desired location on the currently loaded sample 306, injecting a reactive gas onto the currently loaded sample 306). As another non-limiting example, if the scientific instrument 302 is a charged particle microscope, instrument workflow 402 can be a spatial repositioning scheme. In this case, instrument workflow 402 can be considered as any sequence of operations or actions that specifies the movement or reorientation of the currently loaded sample 306 in any desired manner (e.g., determining a desired angular or translational position of the actuable stage of the scientific instrument 302, causing the robotic arm of the scientific instrument 302 to extend, retract, or otherwise articulate in a desired manner). As yet another non-limiting example, if the scientific instrument 302 is a charged particle microscope, instrument workflow 402 can be a vacuum scheme. In this context, instrument workflow 402 can be considered as any sequence of operations or actions that specifies any desired pressure in the vacuum chamber of scientific instrument 302 (e.g., opening or closing the loading lock door of scientific instrument 302, activating or deactivating the vacuum pump of scientific instrument 302). As another non-limiting example, if scientific instrument 302 is a chromatograph or mass spectrometer, instrument workflow 402 can be a scanning or screening scheme.In this context, instrument workflow 402 can be considered as any sequence of operations or actions that designates scientific instrument 302 to scan or screen the currently loaded sample 306 to obtain any desired chemical compound or molecule (e.g., heating the column oven of scientific instrument 302 to a desired temperature, activating the syringe pump or injector of scientific instrument 302, determining the desired ion monitoring range of scientific instrument 302). As another non-limiting example, if scientific instrument 302 is a chromatograph or mass spectrometer, instrument workflow 402 can be a sample heating protocol. In this case, instrument workflow 402 can be considered as any sequence of operations or actions that designates scientific instrument 302 to bring the currently loaded sample 306 to a desired temperature (e.g., heating the autosampler of scientific instrument 302). As yet another non-limiting example, if scientific instrument 302 is a chromatograph or mass spectrometer, instrument workflow 402 can be a sample rinsing protocol. In this context, the currently loaded sample 306 may be a cleaning agent rather than a research treasure, so the instrument workflow 402 can be considered as any sequence of operations or actions that specifies the scientific instrument 302 to flush or clean its internal conduits or tubing with the currently loaded sample 306 in a desired manner (e.g., determining the desired fluid flow rate or fluid pressure of the autosampler syringe of the scientific instrument 302).

[0102] Therefore, instrument workflow 402 can be any suitable series or sequence of automated hardware or software actions that can be performed by scientific instrument 302 to achieve some desired result or effect relative to the currently loaded sample 306. However, these are merely non-limiting examples. In some aspects, the currently loaded sample 306 may be omitted. That is, scientific instrument 302 may be empty, or may not have any sample loaded inside it, but it may still be expected that scientific instrument 302 will perform instrument workflow 402 (e.g., some workflows may exist that are expected or intended to be performed without a sample).

[0103] In various situations, instrument workflow 402 can be selected or specified by a user or technician associated with scientific instrument 302. As a non-limiting example, a user or technician may interact with or otherwise use the human-machine interface of scientific instrument 302 to select instrument workflow 402 from a list of available instrument workflow definitions. As another non-limiting example, a user or technician may interact with or otherwise use the human-machine interface of scientific instrument 302 to define instrument workflow 402 in a customized or specific manner.

[0104] Under any circumstances, workflow component 320 can cause scientific instrument 302 to begin executing any sequence of actions constituting instrument workflow 402. In various ways, scientific instrument 302 may be abruptly, unexpectedly, or otherwise unintentionally interrupted by error message 404 while executing instrument workflow 402. Regarding Figure 5 Non-limiting details are described.

[0105] Figure 5 An example non-limiting block diagram illustrating how error message 404 can interrupt instrument workflow 402 according to one or more embodiments described herein is shown.

[0106] Assume that instrument workflow 402 is for any suitable positive integer. of x A sequence of operations, where these x Each of the operations can be performed or carried out by the scientific instrument 302. In response to a command or instruction from the workflow component 320, the scientific instrument 302 can begin to proceed through the... x The sequence of operations proceeds forward, starting from or initiating with the initial operation in the sequence, and proceeding with the first operation in the sequence. x The operation ends or terminates. During the execution of instrument workflow 402, scientific instrument 302 may experience a malfunction. Such a malfunction may cause or force scientific instrument 302 to suspend instrument workflow 402 and, alternatively, generate, produce, or display an error message 404 electronically. For example, suppose that when scientific instrument 302 encounters a malfunction, for any suitable positive integer... It is attempting to execute the instrument workflow 402. j A failure occurred during operation. In response to the failure, scientific instrument 302 can avoid proceeding to the first step of instrument workflow 402. This operation can either create or throw an error message 404. In other words, the failure can be considered as causing scientific instrument 302 to pause instrument workflow 402, resulting in instrument workflow 402 being incomplete or incomplete.

[0107] In various respects, error message 404 can be any suitable electronic data exhibiting any suitable format, size, or dimension (e.g., it can be one or more scalars, one or more vectors, one or more matrices, one or more tensors, one or more strings, or any suitable combination thereof), which can be considered to convey, indicate, or otherwise represent any suitable information, nature, characteristic, or attribute relating to or concerning any malfunction that interrupts instrument workflow 402. As a non-limiting example, error message 404 can be any suitable structured text or alphanumeric identifier that indicates that a corresponding hardware or software component of scientific instrument 302 has ceased to function properly, without specifically indicating what is wrong with that hardware or software component (e.g., a first alphanumeric identifier may uniquely indicate that the oven of scientific instrument 302 has ceased to function properly, but may not indicate what is specifically wrong with the oven; a second alphanumeric identifier may uniquely indicate that the ion beam emitter of scientific instrument 302 has ceased to function properly, but may not indicate what is specifically wrong with the ion beam emitter). As another non-limiting example, error message 404 can be any suitable structured text or alphanumeric identifier indicating that a corresponding hardware or software component of scientific instrument 302 has encountered a failure of a corresponding type, category, or classification (e.g., a first alphanumeric identifier can uniquely indicate that the oven of scientific instrument 302 has overheated; a second alphanumeric identifier can uniquely indicate that the oven of scientific instrument 302 has physically broken). As yet another non-limiting example, error message 404 can be any suitable unstructured text describing that a corresponding hardware or software component of scientific instrument 302 has encountered a failure of a corresponding type, category, or classification (e.g., one or more natural language sentences or sentence fragments that semantically state or explain which part of scientific instrument 302 failed and in what manner). More generally, different components of scientific instrument 302 may fail or malfunction in different ways, and error message 404 may represent or convey such information at any appropriate level of specificity or generality (e.g., error message 404 may indicate that scientific instrument 302 has malfunctioned; error message 404 may indicate that a particular part of scientific instrument 302 has malfunctioned; or error message 404 may indicate that a particular part of scientific instrument 302 has experienced a particular type of failure).

[0108] In any case, workflow component 320 may enable scientific instrument 302 to begin executing instrument workflow 402, and error message 404 (and any potential malfunction that causes error message 404) may be considered as interrupting instrument workflow 402.

[0109] Figure 6A block diagram of an example non-limiting system for facilitating the parsing of large language models that can disrupt scientific instrument workflows, according to one or more embodiments described herein, is illustrated, the system including instrument runtime data.

[0110] In various implementations, the status component 322 may, in response to an interruption of the instrument workflow 402 by the error message 404, electronically instruct, electronically command, or otherwise electronically cause the scientific instrument 302 to retrieve, record, or recall instrument runtime data 602. In various cases, the instrument runtime data 602 may be any electronic data recorded or otherwise utilized by the scientific instrument 302 during or in conjunction with a portion of the execution of its instrument workflow 402. Regarding Figure 7 Various non-restrictive aspects are described.

[0111] Figure 7 An example non-limiting block diagram illustrating instrument runtime data 602 according to one or more embodiments described herein is shown.

[0112] In various embodiments, instrument runtime data 602 may include a plurality of configurable operating parameter states 702. In various aspects, the plurality of configurable operating parameter states 702 may correspond individually to a plurality of configurable operating parameters 304 (e.g., in a one-to-one manner). More specifically, each configurable operating parameter state in the plurality of configurable operating parameter states 702 may be one or more scalars, one or more vectors, one or more matrices, one or more tensors, one or more strings, or any suitable combination thereof, constituting or representing any state or value that a corresponding configurable operating parameter among the plurality of configurable operating parameters 304 has or is assigned during partial execution of instrument workflow 402. As a non-limiting example, the plurality of configurable operating parameter states 702 may indicate what focal spot size the scientific instrument 302 uses when attempting instrument workflow 402. As another non-limiting example, the plurality of configurable operating parameter states 702 may indicate what beam voltage the scientific instrument 302 uses when attempting instrument workflow 402. As yet another non-limiting example, multiple configurable operating parameter states 702 can indicate what oven temperature the scientific instrument 302 used when attempting instrument workflow 402.

[0113] In various embodiments where scientific instrument 302 is a charged particle microscope and instrument workflow 402 is an imaging scheme, instrument runtime data 602 may include a partially captured charged particle image 704. In various aspects, the partially captured charged particle image 704 may be any suitable partially filled or partially generated pixel array or voxel array created by scientific instrument 302 during its partial execution of instrument workflow 402. For example, it might be expected that instrument workflow 402 will produce an SEM image depicting or illustrating the currently loaded sample 306. However, due to interruption by error message 404, instrument workflow 402 may alternatively produce an incompletely formed SEM image with a large proportion of pixels that still have their default values ​​(e.g., remain zero when they should not be zero), and this incompletely formed SEM image may be considered as the partially captured charged particle image 704. In other words, when the instrument workflow 402 is interrupted, the scientific instrument 302 may be rasterizing the currently loaded sample 306, and therefore the partially captured charged particle image 704 may have large blank spaces, representing any area of ​​the currently loaded sample 306 that should or is intended to be rasterized but has not yet been rasterized. It must be emphasized that conventional techniques consider the partially captured charged particle image 704 to be wasted or corrupted and therefore useless.

[0114] In various embodiments where scientific instrument 302 is a chromatograph or mass spectrometer and instrument workflow 402 is a scanning or screening scheme, instrument runtime data 602 may include partially captured spectral data 706. In various aspects, partially captured spectral data 706 may be any suitable partially filled or partially generated chromatogram or mass spectrum created by scientific instrument 302 during partial execution of its instrument workflow 402. For example, it might be expected that instrument workflow 402 will generate a chromatogram (e.g., abundance versus retention time) and a corresponding mass spectrum (e.g., abundance versus mass-to-charge ratio) for each peak in the chromatogram for the currently loaded sample 306, where a maximum retention time of 120 seconds is achieved. However, due to interruption by error message 404, instrument workflow 402 may alternatively collect only chromatographic and mass spectrometric data for the first 60 seconds of retention time. Therefore, any chromatogram and mass spectrum generated by instrument workflow 402 will lack any chromatographic or ion peaks that appear only at retention times greater than 60 seconds. Such incomplete or incomplete chromatograms or mass spectra can be collectively considered as partially captured spectral data 706. It must be emphasized that conventional techniques consider some of the captured spectral data 706 to be wasted or destroyed, and therefore useless.

[0115] In various embodiments, instrument operation data 602 may include one or more in-instrument pressure feeds 708. Specifically, in various aspects, scientific instrument 302 may be fitted or otherwise equipped with one or more integrated or built-in pressure sensors. Non-limiting examples of such integrated or built-in pressure sensors may include: strain gauges; capacitive pressure transducers; resistive pressure transducers; piezoelectric pressure transducers; or optical pressure transducers. In any case, each of these integrated or built-in pressure sensors may continuously, persistently, or periodically monitor any solid or fluid pressure experienced or encountered by a corresponding hardware component of scientific instrument 302. Thus, each of the one or more in-instrument pressure feeds 708 may be any time series of pressure measurements recorded by the corresponding integrated or built-in pressure sensor during a portion of the execution of instrument workflow 402.

[0116] In various embodiments, instrument runtime data 602 may include one or more in-instrument video feeds 710. In particular, in various aspects, scientific instrument 302 may be fitted or otherwise equipped with one or more integrated or built-in cameras. Non-limiting examples of such integrated or built-in cameras may include: endoscopes; thermal imaging cameras; night vision cameras; or any suitable type of visible spectrum camera. In any case, each of such integrated or built-in cameras may continuously, persistently, or periodically monitor the visual appearance of a corresponding hardware component of scientific instrument 302. Thus, each of the one or more in-instrument video feeds 710 may be any time series of video frames recorded by the corresponding integrated or built-in camera during a portion of the execution of instrument workflow 402.

[0117] In various embodiments, instrument runtime data 602 may include one or more in-instrument audio feeds 712. In particular, in various aspects, scientific instrument 302 may be fitted or otherwise equipped with one or more integrated or built-in microphones. Non-limiting examples of such integrated or built-in microphones may include: dynamic microphones; condenser microphones; ribbon microphones; piezoelectric microphones; or electret microphones. In any case, each of such integrated or built-in microphones may continuously, persistently, or periodically monitor noise emitted by a corresponding hardware component of scientific instrument 302. Therefore, each of the one or more in-instrument audio feeds 712 may be any time series of audio data recorded by the corresponding integrated or built-in microphone during a portion of the execution of instrument workflow 402.

[0118] In various embodiments, instrument operation data 602 may include one or more in-instrument temperature feeds 714. Specifically, in various aspects, scientific instrument 302 may be fitted or otherwise equipped with one or more integrated or built-in temperature sensors. Non-limiting examples of such integrated or built-in temperature sensors may include: thermocouples; thermistors; semiconductor-based thermometers; thermal infrared sensors; or thermal diodes. In any case, each of these integrated or built-in temperature sensors may continuously, persistently, or periodically monitor the temperature experienced by a corresponding hardware component of scientific instrument 302. Thus, each of the one or more in-instrument temperature feeds 714 may be any time series of temperature measurements recorded by the corresponding integrated or built-in temperature sensor during a portion of the execution of instrument workflow 402.

[0119] In various embodiments, instrument operation data 602 may include one or more in-instrument humidity feeds 716. Specifically, in various aspects, scientific instrument 302 may be fitted or otherwise equipped with one or more integrated or built-in humidity sensors. Non-limiting examples of such integrated or built-in humidity sensors may include: capacitive hygrometers; resistive hygrometers; optical hygrometers; gravimetric hygrometers; piezoelectric hygrometers; or solid-state hygrometers. In any case, each of these integrated or built-in humidity sensors may continuously, persistently, or periodically monitor the humidity experienced by a corresponding hardware component of scientific instrument 302. Thus, each of the one or more in-instrument humidity feeds 716 may be any time series of humidity measurements recorded by the corresponding integrated or built-in humidity sensor during a portion of the execution of instrument workflow 402.

[0120] It should be understood or otherwise recognized that Figure 7 The illustration shows only a non-limiting example of instrument runtime data 602. In some cases, any of the following may be excluded or omitted from instrument runtime data 602: multiple configurable operating parameter states 702, partially captured charged particle images 704, partially captured spectral data 706, one or more in-instrument pressure feeds 708, one or more in-instrument video feeds 710, one or more in-instrument audio feeds 712, one or more in-instrument temperature feeds 714, or one or more in-instrument humidity feeds 716, based on data availability or as otherwise required.

[0121] In any event that the instrument workflow 402 is interrupted by error message 404, the status component 322 can electronically collect or retrieve instrument runtime data 602 from the scientific instrument 302.

[0122] Figure 8A block diagram of an example non-limiting system for facilitating the parsing of large language models that can disrupt scientific instrument workflows, according to one or more embodiments described herein, is illustrated. The system includes a technical document library and solutions.

[0123] In various implementations, model component 324 may electronically store, maintain, control, or otherwise electronically access (e.g., locally or remotely) a technical documentation library 802. In various aspects, model component 324 may electronically generate error message 404 through a solution 804 utilizing LLM 308, instrument runtime data 602, or the technical documentation library 802. About Figures 9-14 Various non-restrictive details are described.

[0124] Figures 9-14 An example non-limiting block diagram illustrating a solution 804 for generating an error message 404 based on instrument runtime data 602 and a technical documentation library 802, according to one or more embodiments described herein, is provided.

[0125] First, consider Figure 9 In various cases, a workflow indicator 902 may be present. In various cases, the workflow indicator 902 may be any suitable electronic data exhibiting any suitable format, size, or dimension (e.g., it may be one or more scalars, one or more vectors, one or more matrices, one or more tensors, one or more strings, or any suitable combination thereof), which may be considered to convey, indicate, or otherwise represent any suitable information, nature, or attribute of the instrument workflow 402. As a non-limiting example, the workflow indicator 902 may be any suitable structured text or alphanumeric identifier that indicates or uniquely corresponds to the identity of the instrument workflow 402. In some cases, the workflow indicator 902 and the error message 404 may be jointly regarded as indicating: how many total steps, actions, or operations are involved in the instrument workflow 402; and at which point in those steps, actions, or operations is the instrument workflow 402 interrupted.

[0126] In various respects, model component 324 can electronically execute LLM 308 on instrument runtime data 602, error message 404, workflow indicator 902, or any suitable combination thereof. In various cases, this execution may cause LLM 308 to generate some synthetic text 906.

[0127] More specifically, model component 324 may cascade instrument runtime data 602, error message 404, and workflow indicator 902 together. It should be noted that in some cases, such cascading may include any other combination of the foregoing items (e.g., it may include less than all of instrument runtime data 602, error message 404, and workflow indicator 902). In various cases, model component 324 may feed or route this cascade to the input layer of encoder section 310. In various cases, this cascade may complete the forward pass through one or more hidden layers of encoder section 310. In various aspects, the output layer of encoder section 310 may compute or otherwise compute embedding 904 based on activation maps or feature maps provided by one or more hidden layers of encoder section 310.

[0128] In various cases, embedding 904 can be considered a latent vector representation that encoder portion 310 believes or infers corresponds to a cascade of instrument runtime data 602, error message 404, or workflow indicator 902. More specifically, embedding 904 can be one or more scalars, one or more vectors, one or more matrices, one or more tensors, or any suitable combination thereof. In various respects, the dimension of embedding 904 (e.g., the total number or cardinality of numerical elements within embedding 904) can be smaller (e.g., in some cases, many orders of magnitude smaller) than the total or cumulative dimension of instrument runtime data 602, error message 404, or workflow indicator 902. In various cases, despite its smaller dimension, embedding 904 can still be considered to represent at least some substantial or semantic content of instrument runtime data 602, error message 404, or workflow indicator 902 (albeit in a hidden or non-obvious manner). In other words, embedding 904 can be considered a compact or compressed numerical representation of instrument runtime data 602, error message 404, or workflow indicator 902. It should be noted that the embedding 904 may be considered to represent instrument runtime data 602, error message 404, or workflow indicator 902 in a potentially, vaguely, or otherwise hidden manner, because a third party that is not connected to or related to encoder section 310 will not be able to recreate or guess instrument runtime data 602, error message 404, or workflow indicator 902 from the embedding 904 alone.

[0129] Now, in various aspects, the embedding 904 can be fed or routed to the input layer of the synthesizer section 312. In various cases, the embedding 904 can perform forward propagation through one or more hidden layers of the synthesizer section 312. In various aspects, the output layer of the synthesizer section 312 can compute or otherwise compute synthesize text 906 based on activation maps or feature maps provided by one or more hidden layers of the synthesizer section 312.

[0130] In various respects, the synthesized text 906 can be one or more declarative sentences or sentence fragments generated by the synthesizer section 312 based on the embedding 904. It should be noted that the synthesized text 906 is not an estimate or approximate reconstruction of the instrument runtime data 602, error message 404, or workflow indicator 902. Rather, the synthesized text 906 can be any suitable number of synthesized sentences that are semantically or substantially related to the embedding 904 and therefore to the instrument runtime data 602, error message 404, or workflow indicator 902. In some cases, the synthesized text 906 can be considered to contain an illusion that is semantically or substantially related to the instrument runtime data 602, error message 404, or workflow indicator 902.

[0131] In various ways, model component 324 may ignore, discard, or delete the synthesized text 906. However, model component 324 may record, save, store, or otherwise maintain the embedding 904. In other words, model component 324 may extract the embedding 904 from encoder portion 310 (e.g., from the hidden layer of LLM 308).

[0132] Now, consider Figure 10 In various respects, the technical document library 802 may include multiple technical documents 1002. In various cases, for any suitable positive integer... Multiple technical documents 1002 may include n Documents: Technical Document 1002(1) to Technical Document 1002( nIn various cases, each of the multiple technical documents 1002 may be any suitable electronic file (e.g., a Word-doc file, a PDF file, a web page file) that describes, teaches, illustrates, instructs, or otherwise conveys, in textual form (or in some cases graphically or numerically), one or more technical features, details, or aspects of any suitable scientific instrument (which may or may not include scientific instrument 302), or one or more technical features, details, or aspects of any suitable instrument workflow (which may or may not include instrument workflow 402). As some non-limiting examples, any of the multiple technical documents 1002 may be a maintenance manual, repair manual, schematic diagram, failure mode report, or any part thereof, which describe or explain: the technical or scientific design of any suitable charged particle microscope, chromatograph, or mass spectrometer (e.g., listing or illustrating the different components, parts, or subsystems of the various instruments and describing how these components, parts, or subsystems work); how to operate any suitable charged particle microscope, chromatograph, or mass spectrometer (e.g., listing or illustrating the different user-configurable settings or controls of the various instruments and explaining what such settings or controls do); any suitable information regarding the intended use or intended operation of any suitable charged particle microscope, chromatograph, or mass spectrometer (e.g., showing or explaining whether the various instruments are designed to handle or not designed to handle). This includes: various operating conditions or usage scenarios; any suitable information regarding how to maintain any suitable charged particle microscope, chromatograph, or mass spectrometer (e.g., listing, illustrating, or explaining the various maintenance tasks expected to be performed for each instrument); any suitable information regarding troubleshooting any suitable charged particle microscope, chromatograph, or mass spectrometer (e.g., describing or explaining how to resolve various malfunction symptoms for each instrument); or any suitable information regarding workflows typically performed by a charged particle microscope, chromatograph, or mass spectrometer (e.g., listing, describing, or explaining how to perform different types of instrument workflows; listing, describing, or explaining the necessary or preparatory conditions required for different instrument workflows; listing, describing, or explaining how different instrument workflows typically fail; listing, describing, or explaining how to correct common problems in different instrument workflows). As some other non-limiting examples, any of the technical documents in the plurality of technical documents 1002 may be: any source code script or file associated with any suitable charged particle microscope, chromatograph or mass spectrometer; any suitable maintenance ticket or service ticket associated with any suitable charged particle microscope, chromatograph or mass spectrometer; or any suitable form of customer complaint or customer comment associated with any suitable charged particle microscope, chromatograph or mass spectrometer.

[0133] It should be recognized and understood that any technical document in the plurality of technical documents 1002 can be any subsection or sub-section of a larger document. For example, any technical document in the plurality of technical documents 1002 can be a chapter, section, paragraph, or even sentence from a longer document.

[0134] In various respects, model component 324 can electronically generate multiple embeddings 1004 by performing LLM 308 as described above on each of the multiple technical documents 1002.

[0135] As a non-limiting example, model component 324 may perform LLM 308 on technical document 1002(1), and model component 324 may extract embedding 1004(1) from LLM 308 during the execution. More specifically, model component 324 may feed or route technical document 1002(1) to the input layer of encoder section 310, technical document 1002(1) may perform forward pass through one or more hidden layers of encoder section 310, and the output layer of encoder section 310 may compute or otherwise compute embedding 1004(1) based on activation maps or feature maps provided by one or more hidden layers of encoder section 310. It should be noted that embedding 1004(1) may have the same format, size, or dimension as embedding 904 (e.g., embedding may be a vector representing a sentence with uniform dimension or uniform size; sentence-by-sentence embeddings of a paragraph may be aggregated or averaged together to produce the embedding of that paragraph; section-by-section embeddings of a chapter may be aggregated or averaged together to produce the embedding of that section or chapter; section-by-section or chapter-by-chapter embeddings of the entire document may be aggregated or averaged together to produce the embedding of the entire document), and therefore embedding 1004(1) may be considered as a potential vector representation of technical document 1002(1). In various cases, embedding 1004(1) may then complete the forward pass through synthesizer part 312, but model part 324 may ignore, disregard, or remove any synthesized text content created by synthesizer part 312 based on embedding 1004(1).

[0136] As another non-limiting example, model component 324 may correspond to technical document 1002 ( n ) Execute LLM 308, and model part 324 can extract embedding 1004 from LLM 308 during this execution. n In fact, as described above, model component 324 can be used with technical document 1002 (…). n Feed or route to the input layer of encoder section 310, technical document 1002 ( nThe forward pass through one or more hidden layers of encoder section 310 can be completed, and the output layer of encoder section 310 can compute or otherwise compute embedding 1004 based on the activation map or feature map provided by one or more hidden layers of encoder section 310. n Therefore, embedding 1004 ( n ) can have the same format, size, or dimensions as embedded 904, and therefore embedded 1004 ( n This can be considered as technical document 1002. n The latent vector representation of ). As above, 1004 ( n Then the forward pass through synthesizer section 312 can be completed, but model component 324 can ignore, disregard, or remove synthesizer section 312 based on embedding 1004 ( n Any synthetic text content created.

[0137] In various cases, embedding 1004(1) to embedding 1004( n They can be collectively considered to form multiple embeddings 1004.

[0138] In various respects, model component 324 can electronically identify or identify a related set of technical documents 1006 by comparing embedding 904 with a plurality of embeddings 1004. Specifically, for each given embedding among the plurality of embeddings 1004, model component 324 can calculate any suitable error or similarity value between that given embedding and embedding 904. As some non-limiting examples, such error or similarity value may involve: mean absolute error (MAE) calculation; mean squared error (MSE) calculation; cosine similarity calculation; Euclidean distance calculation; or cross-entropy calculation. In any case, model component 324 can determine that the related set of technical documents 1006 is any technical document among the plurality of technical documents 1002 whose embedding (e.g., in 906) is most similar to or closest to embedding 904. As a non-limiting example, for any suitable positive integer m Model component 324 can identify multiple technical documents 1002 that have an embedding most similar to or closest to embedding 904. m A technical document, and this type m This document can be considered as a collection of related technical documents 1006. That is, the collection of related technical documents 1006 may include... m One document: Related technical document 1006(1) to related technical document 1006( m In other words, the relevant technical document 1006(1) can be any technical document among a plurality of technical documents 1002 whose embedding is closest to or most similar to embedding 904, while the relevant technical document 1006( m() can be embedded in multiple technical documents 1002, which is the first m closest or first m The most similar technical document to any document embedding a 904 error.

[0139] In any case, the relevant technical document set 1006 may be considered to be substantially or semantically related to the instrument operation data 602, error message 404, or workflow indicator 902 in some way. As a non-limiting example, it is assumed that workflow indicator 902 specifies that instrument workflow 402 is a particular imaging scheme, and it is assumed that error message 404 indicates that the ion beam emitter of scientific instrument 302 has ceased to function properly. In this context, any relevant technical document in the relevant technical document collection 1006 can be structured or unstructured text describing or explaining: what hardware or software components constitute scientific instrument 302; what imaging operation sequence constitutes instrument workflow 402; how to perform, conduct, or implement each imaging operation in those imaging operations; what focal spot size, grating pattern, or beam voltage is required for instrument workflow 402; how to design, operate, or construct the ion beam emitter; what types of malfunctions typically affect the ion beam emitter; what types of malfunctions typically result in partially captured images, partially captured spectral data, internal pressure measurements, internal video recordings, internal audio recordings, internal temperature measurements, or internal humidity measurements similar to those shown in instrument runtime data 602; or how to troubleshoot or repair any of such malfunctions. Therefore, the relevant technical document collection 1006 can be considered to provide valuable contextual information regarding instrument runtime data 602, error message 404, or workflow indicator 902.

[0140] Next, consider Figure 11In various implementations, a causal prompt 1102 may be present. In various aspects, the causal prompt 1102 may be one or more unstructured or plain text sentences or sentence fragments that request or command the identification, determination, or diagnosis of a specific hardware or software cause of any malfunction that interrupts the instrument workflow 402. In practice, the scientific instrument 302 may consist of multiple hardware or software components (e.g., an ion beam emitter, an electron detector, a collimating lens, a vacuum pump, an operating system, downloaded libraries); any given hardware or software may be capable of experiencing multiple types of failures or malfunctions (e.g., an ion beam emitter may experience cathode failure, anode failure, or insulator failure); a given type of failure or malfunction may have different underlying or root causes (e.g., insulation failure of an ion beam emitter may be caused by: dielectric breakdown via voltage surge; dielectric breakdown via overheating; dielectric breakdown due to vibration fatigue; dielectric breakdown via moisture absorption; or dielectric breakdown via degassing). Therefore, the causal prompt 1102 can be considered a request to identify a specific, subtle, potential, or root cause of any malfunction that interrupts the instrument workflow 402. As a non-limiting example, the causal relationship prompt 1102 could be the following sentence: "The workflow is specified by workflow indicator 902. The workflow is interrupted by error message 404, generating instrument runtime data 602. What specific fault caused the workflow to be interrupted?" As another non-limiting example, causal prompt 1102 could be the following sentence: "The workflow is specified by workflow indicator 902. The workflow is interrupted by error message 404, generating instrument runtime data 602. Diagnose the cause of the interruption." Therefore, in various cases, model component 324 can electronically perform LLM 308 on instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, causal relationship prompt 1102, or any suitable combination thereof. In various cases, such execution can cause LLM 308 to generate explanatory text 1104. More specifically, model component 324 can cascade instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, and causal relationship prompt 1102 together. As above, it should be noted that in various cases, this cascading can include any other combination of the foregoing items (e.g., it can include less than all of instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, and causal relationship prompt 1102). In various cases, model component 324 can feed this cascading to the input layer of encoder section 310. In various aspects, this cascade can complete the forward pass through one or more hidden layers of encoder section 310. In various cases, the output layer of encoder section 310 can compute or otherwise compute one or more embeddings (not shown) based on activation maps or feature maps provided by one or more hidden layers of encoder section 310. In various cases, the one or more embeddings can be routed to the input layer of synthesizer section 312. In various aspects, the one or more embeddings can complete the forward pass through one or more hidden layers of synthesizer section 312, and the output layer of synthesizer section 312 can compute or otherwise compute description text 1104 based on activation maps or feature maps provided by one or more hidden layers of synthesizer section 312.

[0141] In various respects, the explanatory text 1104 may be one or more unstructured or plain text statements or sentence fragments that semantically answer the causal prompt 1102. That is, the explanatory text 1104 may be synthetic text describing or stating what potential problem with scientific instrument 302 appears to have caused the instrument workflow 402 to be interrupted. In other words, the explanatory text 1104 may be natural language or plain text describing or explaining what specific hardware-related or software-related problem forced scientific instrument 302 to generate error message 404 in instrument workflow 402. In various cases, workflow indicator 902 may be considered to indicate known information about instrument workflow 402, while error message 404 and instrument runtime data 602 (such as those enhanced or supplemented by the relevant technical documentation set 1006) may be considered to be measurement or detection information about any unknown fault or failure that interrupted instrument workflow 402. In some cases, this unknown fault or failure may have left traces, markings, or other unique effects on instrument runtime data 602. After all, different potential failures can manifest in different ways (for example, some failures can manifest visually, which can uniquely affect partially captured charged particle images 704 or one or more in-instrument video feeds 710; other failures can manifest audibly, which can uniquely affect one or more in-instrument audio feeds 712; and still others can manifest thermally, which can uniquely affect one or more in-instrument temperature feeds 714). Therefore, LLM 308 can be considered as utilizing the relevant technical documentation set 1006 to identify any traces, markers, or unique effects left in the instrument operation data 602 by previously unknown faults, and such identification can therefore allow for the inference of the identity of the previously unknown fault. Thus, the explanatory text 1104 can semantically convey or indicate this identity.

[0142] Now, consider Figure 12In various implementations, a parsing prompt 1202 may be present. In various aspects, the parsing prompt 1202 may be one or more unstructured or plain text sentences or sentence fragments that request or command the identification or determination of a remedial action for any specific fault specified in the explanatory text 1104. Indeed, as stated above, a given type of failure or fault may have different underlying or root causes, and different underlying or root causes may require different remedial actions (e.g., dielectric breakdown via voltage surges may be repaired or resolved by a first fault-clearing or repair action sequence; in contrast, dielectric breakdown via overheating may not respond to a first fault-clearing or repair action sequence but may be repaired or resolved by a second fault-clearing or repair action sequence). Therefore, the parsing prompt 1202 can be considered a request to identify specific actions or programs that, if performed on scientific instrument 302, would resolve or resolve any problems causing an interruption to instrument workflow 402. As a non-limiting example, the parsing prompt 1202 could be the following sentence: "The workflow is specified by workflow indicator 902. The workflow was interrupted by error message 404, generating instrument runtime data 602. The inferred reason for the interruption is indicated by explanatory text 1104. How to parse the inferred reason?" As another non-limiting example, the parsing prompt 1202 could be the following sentence: "The workflow is specified by workflow indicator 902. The workflow was interrupted by error message 404, generating instrument runtime data 602. The inferred cause of the interruption is indicated by explanatory text 1104. Determine how to repair."

[0143] Therefore, in various cases, model component 324 can electronically perform LLM 308 on instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, explanatory text 1104, parsing prompt 1202, or any suitable combination thereof. In various cases, such execution may cause LLM 308 to generate parsing text 1204. More specifically, model component 324 may cascade instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, explanatory text 1104, and parsing prompt 1202 together. As above, it should be noted that in various cases, this cascading may include any other combination of the foregoing items (e.g., it may include less than all of instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, explanatory text 1104, or parsing prompt 1202). In various cases, model component 324 may feed this cascading to the input layer of encoder section 310. In various aspects, this cascade can complete the forward pass through one or more hidden layers of encoder section 310. In various cases, the output layer of encoder section 310 can compute or otherwise compute one or more embeddings (not shown) based on activation maps or feature maps provided by one or more hidden layers of encoder section 310. In various cases, the one or more embeddings can be routed to the input layer of synthesizer section 312. In various aspects, the one or more embeddings can complete the forward pass through one or more hidden layers of synthesizer section 312, and the output layer of synthesizer section 312 can compute or otherwise compute the parsed text 1204 based on activation maps or feature maps provided by one or more hidden layers of synthesizer section 312.

[0144] In various respects, the parsing text 1204 can be one or more unstructured or plain text statements or sentence fragments that semantically answer the parsing prompt 1202. That is, the parsing text 1204 can be synthetic text that describes or states what remedial action or sequence of operations will resolve or address any potential problem with the scientific instrument 302 specified in the explanatory text 1104. In other words, the parsing text 1204 can be natural language or plain text that forms a tutorial that teaches what specific hardware or software modifications to the scientific instrument 302 will repair or fix a specific hardware-related or software-related problem that interrupts the instrument workflow 402. Non-limiting examples of remedial hardware modifications may include: replacing consumable fluids, housings, or components of the scientific instrument 302; tightening or loosening specific bolts or fasteners of the scientific instrument 302; drying, wetting, or lubricating specified surfaces or lenses of the scientific instrument 302; leveling, reorienting, or otherwise repositioning the scientific instrument 302; or adding structural supports to the scientific instrument 302. Non-limiting examples of remedial software modifications may include: restarting or rebooting scientific instrument 302; adjusting a specific one of the multiple configurable operating parameters 304 to a specific value; or editing the source code file of scientific instrument 302. In any of these cases, LLM 308 may be considered to utilize its own inferences indicated in the explanatory text 1104 and any traces, markers, or unique patterns present in the instrument runtime data 602 (such as those supplemented or enhanced by the related technical documentation set 1006) to determine how to resolve any problem causing error message 404 to interrupt instrument workflow 402. Therefore, parsing text 1204 may semantically convey or indicate this determination.

[0145] Next, consider Figure 13 In various implementations, code hint 1302 may be present. In various aspects, code hint 1302 may be one or more unstructured or plain text sentences or sentence fragments that request or command the generation or writing of coded scripts to implement any software modifications specified in parsing text 1204. As a non-limiting example, code hint 1302 may be the following sentence: “The workflow is specified by workflow indicator 902. The workflow is interrupted by error message 404, generating instrument runtime data 602. The inferred cause of the interruption is indicated by explanatory text 1104. Parsing is indicated in parsing text 1204. A script to implement parsing is written.”

[0146] Therefore, in various cases, model component 324 can electronically execute LLM 308 on instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, explanatory text 1104, parsed text 1204, code hint 1302, or any suitable combination thereof. In various cases, such execution can cause LLM 308 to generate synthetic code 1304. More specifically, model component 324 can cascade instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, explanatory text 1104, parsed text 1204, and code hint 1302 together. As above, it should be noted that in various cases, this cascading can include any other combination of the foregoing items (e.g., it can include less than all of instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, explanatory text 1104, parsed text 1204, or code hint 1302). In various cases, model component 324 may feed the cascade to the input layer of encoder section 310. In various aspects, the cascade may perform forward propagation through one or more hidden layers of encoder section 310. In various aspects, the output layer of encoder section 310 may compute or otherwise compute one or more embeddings (not shown) based on activation maps or feature maps provided by one or more hidden layers of encoder section 310. In various aspects, the one or more embeddings may be routed to the input layer of synthesizer section 312. In various aspects, the one or more embeddings may perform forward propagation through one or more hidden layers of synthesizer section 312, and the output layer of synthesizer section 312 may compute or otherwise compute synthesized code 1304 based on activation maps or feature maps provided by one or more hidden layers of synthesizer section 312.

[0147] In various respects, the synthetic code 1304 can be one or more structured texts that semantically resolve or satisfy the parsing prompt 1202. That is, the synthetic code 1304 can be coded or a programming script configured to perform or implement any remedial software modifications specified in the parsing text 1204 at execution time. In other words, the synthetic code 1304 can be one or more lines of computer-executable code (e.g., consisting of variable definitions, function calls, for loops, if loops, or while loops) written in any suitable programming syntax, and when executed by a computing device, cause that computing device to perform any software-related actions that LLM 308 has determined will resolve any problems that interrupt the instrument workflow 402.

[0148] like Figure 14As shown, solution 804 can be considered to include explanatory text 1104, parsed text 1204, and synthetic code 1304, or to be composed of them.

[0149] It should be understood that in some implementations, solution 804 may omit or exclude any of the explanatory text 1104, parsed text 1204, or synthetic code 1304.

[0150] although Figures 9-14 The LLM 308 is described as generating the explanatory text 1104, parsing text 1204, and synthesis code 1304 sequentially, one after another, but this is merely a non-limiting example for ease of explanation and illustration. In various respects, a single unified prompt may exist, containing any combination of the contents of the causal prompt 1102, parsing prompt 1202, or code prompt 1302. In this case, the LLM 308 can be executed on the cascading of instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, and this unified prompt, and such execution may allow the LLM 308 to generate the explanatory text 1104, parsing text 1204, and synthesis code 1304 simultaneously or substantially simultaneously.

[0151] Despite Figures 9-14While not explicitly shown, it should be noted that in some implementations where the explanatory text 1104, parsed text 1204, and synthesized code 1304 are generated sequentially, the relevant technical document set 1006 may be iteratively or incrementally updated before each execution of the LLM 308. As a non-limiting example, the relevant technical document set 1006 may be initially extracted by performing an embedding search on the technical document library 802 based on the embedding 904 shown. However, after generating the explanatory text 1104, new embeddings may be created to collectively represent the instrument runtime data 602, error message 404, workflow indicator 902, and explanatory text 1104, and these new embeddings may be used to search the technical document library 802 again. Therefore, any technical document retrieved from such a new search may be considered semantically relevant not only to the instrument runtime data 602, error message 404, and workflow indicator 902, but also to the explanatory text 1104. Thus, these new relevant technical documents may be fed as input to the LLM 308 to improve the accuracy or reliability of the parsed text 1204. Similarly, after generating the parsed text 1204, another new embedding can be created to collectively represent the instrument runtime data 602, error message 404, workflow indicator 902, explanatory text 1104, and parsed text 1204, and this other new embedding can be used to search the technical document library 802. Therefore, any technical document retrieved from this other new search can be considered semantically relevant not only to the instrument runtime data 602, error message 404, workflow indicator 902, and explanatory text 1104, but also to the parsed text 1204. Thus, these other new relevant technical documents can be fed as input to the LLM 308 to improve the accuracy or reliability of the synthesized code 1304.

[0152] In any case, model component 324 can generate solution 804 by utilizing LLM 308, instrument runtime data 602, or technical documentation library 802.

[0153] In various implementations, execution component 326 may perform any suitable subsequent actions in response to the generation of solution 804. As a non-limiting example, execution component 326 may electronically transmit solution 804 (or any suitable portion thereof) to any suitable computing device. As another non-limiting example, execution component 326 may electronically present solution 804 (or any suitable portion thereof) on any suitable computer screen or display.

[0154] It should be understood that the difficulty of implementing solution 804 can range from easy to difficult. After all, some types of malfunctions can be easily resolved (e.g., changing a few operating parameter values, cleaning specific surfaces of scientific instrument 302). In contrast, other types of malfunctions may be very difficult to resolve (e.g., extensive editing of the source code of scientific instrument 302; extensive refurbishment or redesign of the physical structure of scientific instrument 302). Therefore, when solution 804 is easy or not difficult to implement, it is acceptable or appropriate for execution component 326 to electronically transmit or present solution 804 to or on a computing device associated with the end user of scientific instrument 302. On the other hand, when solution 804 is difficult or not easy to implement, it is acceptable or appropriate for execution component 326 to electronically transmit or present solution 804 to or on a computing device associated with the engineers or technical experts of the manufacturer of scientific instrument 302.

[0155] In any case, solution 804 can be generated within seconds or minutes after the instrument workflow 402 is interrupted. Therefore, solution 804 can be considered to significantly speed up the troubleshooting process for such an interruption compared to the hours, days, weeks, or months required by existing technologies.

[0156] In some embodiments of solution 804 that include synthetic code 1304, execution component 326 may electronically instruct, command, or otherwise electronically cause scientific instrument 302 to execute synthetic code 1304. Thus, scientific instrument 302 may execute any remedial software modifications specified in parsed text 1204. In response to such execution, in some cases, execution component 326 may electronically instruct, command, or otherwise electronically cause scientific instrument 302 to resume or retry instrument workflow 402 to verify whether the potential or fundamental problem specified in instruction text 1104 has been successfully fixed by executing synthetic code 1304. If scientific instrument 302 encounters the same or another interruption during such resumption or retry of instrument workflow 402, system 314 may: repeat the various actions described herein; or may automatically request electronic contact with a technical expert for troubleshooting assistance.

[0157] In some implementations, LLM 308 may infer or predict any faults that have never been seen or encountered before that cause error message 404. In this case, explanatory text 1104 may indicate that the potential cause of error message 404 is unknown or has never been seen before. In such cases, execution component 326 may, in response to explanatory text 1104 indicating that the potential cause is unknown or has never been seen before, electronically send an assistance request to any suitable computing device of a technical expert associated with the manufacturer of scientific instrument 302. In some such cases, modeling component 324 may avoid generating parsing text 1204 and synthetic code 1304. In other such cases, modeling component 324 may generate parsing text 1204 and synthetic code 1304, which can be collectively regarded as potential or proposed solutions to the unseen cause of error message 404, and execution component 326 may correspondingly send them to the computing device of the technical expert, thereby assisting the technical expert despite the unseen nature of the potential cause of error message 404.

[0158] Figure 15 A block diagram of an example non-limiting system for facilitating the resolution of large language models that can disrupt scientific instrument workflows, according to one or more embodiments described herein, is illustrated. The system includes solution feedback and a set of technical document editors.

[0159] In various implementations, system 314 may receive, retrieve, or otherwise access solution feedback 1502 electronically from any suitable source. In various respects, solution feedback 1502 may be any suitable electronic data (e.g., one or more scalars, one or more vectors, one or more matrices, one or more tensors, or one or more strings) that represents or otherwise conveys a deficiency or inadequacy of solution 804.

[0160] As a non-limiting example, as described above, execution component 326 may share or present solution 804 to or on a computing device associated with, or on, a technical expert of the manufacturer of scientific instrument 302. Thus, the technical expert is able to read or review solution 804. In some cases, the technical expert may identify one or more inaccurate or suboptimal parts of solution 804 (e.g., determining that the explanatory text 1104 describes a potential cause of the error; determining that the parsed text 1204 describes an incorrect remedial action or an incorrect order of remedial actions; determining that the synthesized code 1304 describes an incorrect function call). Therefore, solution feedback 1502 may be one or more declarative natural language sentences or sentence fragments written by the technical expert (e.g., using any suitable word processing software) and describe or explain how or why one or more specific parts of solution 804 are inaccurate.

[0161] In various aspects, model component 324 can utilize LLM 308 and solution feedback 1502 to generate one or more technical document edits 1504. About Figure 16 Various non-restrictive aspects are described.

[0162] Figure 16 Example non-limiting block diagrams illustrating how one or more technical document edits 1504 can be generated based on solution feedback 1502 according to one or more embodiments described herein.

[0163] In various implementations, an edit prompt 1602 may be present. In various aspects, the edit prompt 1602 may be one or more unstructured or plain text sentences or sentence fragments that request or instruct the editing of a corresponding relevant technical document in the relevant technical document set 1006 to avoid any inference inaccuracies specified in the solution feedback 1502. In other words, the LLM 308 may be considered to have generated the inference inaccuracies specified in the solution feedback 1502 due to being misled by certain parts of the relevant technical document set 1006 (e.g., certain chapters, certain pages, certain paragraphs, certain sentences) (e.g., possibly because those documents are outdated), and the edit prompt 1602 may be considered to require the LLM 308 to identify those certain parts of the relevant technical document set 1006 and determine how those certain parts should be edited or changed so that the LLM 308 will not be misled in the same way again in the future. As a non-limiting example, edit prompt 1602 could be the following sentence: "The workflow is specified by workflow indicator 902. The workflow is interrupted by error message 404, generating instrument runtime data 602. Troubleshooting is indicated by solution 804 and based on inferences from related technical document set 1006. The solution indicated by solution feedback 1502 is inaccurate. What edits to related technical document set 1006 will prevent the solution from being inaccurate?" As a non-limiting example, edit prompt 1602 could be the following sentence: "The workflow is specified by workflow indicator 902. The workflow is interrupted by error message 404, generating instrument runtime data 602. Troubleshooting is indicated by solution 804 and based on inferences from related technical document set 1006. What edit would bring related technical document set 1006 into agreement with solution feedback 1502?" ".

[0164] Therefore, in various cases, model component 324 can electronically perform LLM 308 on instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, solution 804, solution feedback 1502, editing prompt 1602, or any suitable combination thereof. In various cases, such execution may cause LLM 308 to produce one or more technical document edits 1504. More specifically, model component 324 may cascade instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, solution 804, solution feedback 1502, and editing prompt 1602 together. As above, it should be noted that in various cases, this cascading may include any other combination of the foregoing items (e.g., may include less than all of instrument runtime data 602, error message 404, workflow indicator 902, related technical document set 1006, solution 804, solution feedback 1502, or editing prompt 1602 together). In various cases, model component 324 may feed the cascade to the input layer of encoder section 310. In various aspects, the cascade may complete the forward pass through one or more hidden layers of encoder section 310. In various aspects, the output layer of encoder section 310 may compute or otherwise compute one or more embeddings (not shown) based on activation maps or feature maps provided by one or more hidden layers of encoder section 310. In various aspects, the one or more embeddings may be routed to the input layer of synthesizer section 312. In various aspects, the one or more embeddings may complete the forward pass through one or more hidden layers of synthesizer section 312, and the output layer of synthesizer section 312 may compute or otherwise compute one or more embeddings based on activation maps or feature maps provided by one or more hidden layers of synthesizer section 312.

[0165] In various respects, one or more technical document edits 1504 can be one or more structured or unstructured texts that semantically answer or resolve editing prompts 1602. That is, one or more technical document edits 1504 can be synthetic text that describes, states, or otherwise represents one or more text edits or text changes (e.g., word deletion or insertion; sentence deletion, insertion, or paraphrasing; paragraph deletion, insertion, or paraphrasing) that, if made to a corresponding relevant technical document in the relevant technical document set 1006, will make solution 804 consistent with solution feedback 1502. In other words, each technical document edit in one or more technical document edits 1504 can be a corresponding grammatical or semantic change, annotation, or underlining of a corresponding technical document in the relevant technical document set 1006 that, if made to the relevant technical document before generating solution 804, will prevent solution 804 from containing any inaccuracies specified in solution feedback 1502. In other words, LLM 308 can be considered as inferring which specific parts of which specific technical documents it relies on to generate Solution 804, causing LLM 308 to be misled or otherwise generate any incorrect inferences specified in Solution Feedback 1502, and LLM 308 can further infer how those specific parts of those specific technical documents should be rewritten to prevent LLM 308 from being similarly misled in the future. Therefore, one or more technical document edits 1504 can be considered as rewritten versions (or otherwise rewrite instructions) of those specific parts of those specific technical documents.

[0166] In various implementations, execution component 326 can electronically perform one or more technical document edits 1504. In other words, execution component 326 can electronically apply one or more technical document edits 1504 to corresponding related technical documents in the related technical document collection 1006. Furthermore, execution component 326 can semantically rewrite or modify one or more technical documents in the related technical document collection 1006 in any manner specified by one or more technical document edits 1504. Therefore, the substantial information contained within the technical document library 802 can be considered up-to-date, thus increasing the likelihood of inaccuracies in troubleshooting solutions inferred by LLM 308 in the future.

[0167] To increase the likelihood of the accuracy or correctness of solution 804 according to the various implementation schemes described herein, the LLM308 may first undergo training. Regarding Figure 17 An unrestricted example of this training is described.

[0168] Figure 17Example non-limiting block diagrams illustrating how an LLM 308 can be trained are shown according to one or more embodiments described herein.

[0169] In various respects, before training begins, the trainable intrinsic parameters of the LLM 308 (e.g., convolutional kernels, weight matrices, bias values) can be initialized by the system 314 in any suitable manner (e.g., via random initialization).

[0170] In various implementations, a training input 1702 and a truth note 1704 may exist. In various respects, the training input 1702 may be any suitable text, numerical, or graphical data that can be received by the LLM 308, as described herein. As some examples, if not limiting, the training input 1702 may be any suitable combination of instrument runtime data, error messages, workflow indicators, technical documents, prompts, any other suitable structured or unstructured text or numerical data, or any suitable combination or cascade thereof. In various cases, the truth note 1704 may be any correct or accurate synthetic text content (e.g., such as 1104, 1204, or 1504) or code (e.g., such as 1304) known or considered to correspond to the training input 1702.

[0171] In any case, system 314 can enable LLM 308 to execute on training input 1702, thereby causing LLM 308 to produce output 1706. More specifically, in some cases, training input 1702 can be fed or routed to the input layer of LLM 308, training input 1702 can perform forward passes through one or more hidden layers of LLM 308, and the output layer of LLM 308 can compute output 1706 based on activation maps or feature maps provided by one or more hidden layers of LLM 308.

[0172] It should be noted that the format, size, or dimensions of output 1706 can be specified by the number, arrangement, size, or other characteristics of neurons, convolutional kernels, attention blocks, or other internal parameters of the output layer (or any other layer) of the LLM 308. Therefore, output 1706 can be forced to have any desired format, size, or dimensions by adding, removing, or otherwise adjusting the characteristics of the output layer (or any other layer) of the LLM 308.

[0173] In various respects, output 1706 can be considered as predicted or inferred text content synthesized by LLM 308 based on training input 1702 (e.g., predicted or inferred explanatory text, predicted or inferred parsed text, predicted or inferred synthesized code, predicted or inferred technical documentation edits). In contrast, truth annotation 1704 can be considered as any correct or accurate text content known or considered to correspond to training input 1702 (e.g., correct or accurate explanatory text, correct or accurate parsed text, correct or accurate synthesized code, correct or accurate technical documentation edits). It should be noted that if LLM 308 has not undergone or has undergone little to no training so far, output 1706 may be highly inaccurate. In other words, output 1706 can be very different from truth annotation 1704.

[0174] In various ways, the loss 1708 between the output 1706 and the truth annotation 1704 can be calculated by system 314 (e.g., MAE, MSE, cross-entropy error). In various cases, the trainable intrinsic parameters of LLM 308 can be progressively updated based on the loss 1708 via backpropagation (e.g., stochastic gradient descent).

[0175] In various cases, this execution and update process can be repeated for any appropriate number of input annotation pairs. This ultimately allows the trainable intrinsic parameters of the LLM 308 to be iteratively optimized to accurately perform text synthesis or code synthesis. In various respects, any suitable training batch size, any suitable error / loss function, or any suitable training termination criterion can be utilized during this training.

[0176] Although the disclosure herein primarily describes LLM 308 as being trained in a supervised manner, this is merely a non-limiting example for ease of explanation and illustration. In various implementations, any other suitable training paradigm can be used to train LLM 308, such as unsupervised training, semi-supervised training, or reinforcement learning, and any of these training paradigms can be joint or non-joint.

[0177] Although the disclosure herein primarily describes the LLM 308 as being trained or configured to synthesize solution 804, these are merely non-limiting examples for ease of explanation and illustration. In various embodiments, the LLM 308 may be configured to synthesize any suitable visual graphics that complement solution 804. Non-limiting examples of such visual graphics may include: formatted reports or presentation slides based on, derived from, or illustrating solution 804; or plots, graphs, or charts based on, derived from, or illustrating solution 804. Therefore, execution component 326 may visually represent such visual graphics in addition to or otherwise incorporating solution 804.

[0178] Although the disclosure herein primarily describes the execution component 326 as visually or audibly presenting or displaying the solution 804 (or any associated visual graphics) to a user or technician of the scientific instrument 302, these are merely non-limiting examples for ease of explanation and illustration. In various embodiments, the execution component 326 may electronically transmit the solution 804 (or any associated visual graphics) to any suitable computing device associated with the scientific instrument 302. As a non-limiting example, the execution component 326 may share the solution 804 (or any associated visual graphics) with any suitable downstream software tools or applications that operate or are associated with the scientific instrument 302 (e.g., some embodiments may involve sending the solution 804 to such downstream software tools or applications rather than presenting the solution 804 to a user or technician).

[0179] In various contexts, machine learning algorithms or models can be implemented in any suitable manner to facilitate any suitable aspect described herein. To facilitate some of the machine learning aspects described above in various implementations, consider the following discussion of artificial intelligence (AI). The various implementations described herein may employ artificial intelligence to facilitate the automation of one or more features or functionalities. These components may employ various AI-based schemes to perform the various implementations / examples disclosed herein. To provide or assist in the numerous determinations (e.g., determination, ascertainment, inference, computation, prediction, prognosis, estimation, derivation, forecasting, detection, computation) described herein, the components described herein may examine all or a subset of the data to which they have been granted access and may provide reasoning or determination of the state of a system or environment from a set of observations, such as those captured via events or data. For example, determinations may be employed to identify specific contexts or actions, or to generate probability distributions of states. These determinations may be probabilistic; that is, the probability distribution of states of interest is calculated based on considerations of data and events. Determination may also refer to techniques employed to compose higher-level events from a set of events or data.

[0180] Such determinations can lead to the construction of new events or actions from observed events or a collection of stored event data, regardless of whether the events are temporally related or whether the events and data originate from one or more event and data sources. The components disclosed herein can employ various classification schemes (explicit training (e.g., via training data) and implicit training (e.g., via observed behavior, preferences, historical information, received external information, etc.)) or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, etc.) related to the execution of automatic or deterministic actions relevant to the claimed subject matter. Therefore, classification schemes or systems can be used to automatically learn and execute multiple functions, actions, or determinations.

[0181] The classifier can take the input attribute vector z = (z1, z2, z3, z4, ... z n The confidence level of an input class is mapped to the confidence level of that class, such as f(z) = confidence level. ( (Classification). This type of classification can use probabilistic or statistical analysis (e.g., considering analytical utility and cost) to determine the actions to be performed automatically. Support Vector Machines (SVMs) are an example of classifiers that can be adopted. SVMs operate by finding a hypersurface in the space of possible inputs, where the hypersurface attempts to separate triggering criteria from non-triggering events. Intuitively, this makes the classification correct for test data that is close to but not identical to the training data. Other directed and non-directed model classification methods include, for example, Naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, or probabilistic classification models that provide different independent patterns, any of which can be adopted. The classification used in this paper also includes statistical regression for developing priority models.

[0182] To provide additional context for the various implementation schemes described herein Figure 18 The following discussion is intended to provide a brief general description of a suitable computing environment 1800 in which various implementations of the embodiments described herein may be implemented. While the embodiments have been described above in the general context of computer-executable instructions that can run on one or more computers, those skilled in the art will recognize that the embodiments may also be implemented in combination with other program modules or as a combination of hardware and software.

[0183] Typically, program modules include routines, programs, components, data structures, etc., that perform specific tasks or implement specific abstract data types. Furthermore, those skilled in the art will understand that the methods of this invention can be practiced with other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (IoT) devices, distributed computing systems, and personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, each operatively coupled to one or more associated devices.

[0184] The illustrative implementation schemes described herein can also be practiced in distributed computing environments, where certain tasks are performed by remote processing devices linked via a communication network. In a distributed computing environment, program modules can reside on both local and remote memory storage devices.

[0185] Computing devices typically include a variety of media, which may include computer-readable storage media, machine-readable storage media, or communication media. These two terms are used interchangeably herein, as follows. A computer-readable storage media or a machine-readable storage media can be any available storage medium that can be accessed by a computer, and includes volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, a computer-readable storage media or a machine-readable storage media can be implemented in conjunction with any method or technology used for storing information, such as computer-readable or machine-readable instructions, program modules, structured data, or unstructured data.

[0186] Computer-readable storage media may include, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compressed optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD), Blu-ray disc (BD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, solid-state drives or other solid-state storage devices, or other tangible or non-transitory media that can be used to store desired information. In this regard, the terms “tangible” or “non-transitory” used herein to describe storage devices, memories, or computer-readable media should be understood to exclude only the propagation of transient signals themselves as a modifier, and do not waive the rights of all standard storage devices, memories, or computer-readable media that are not merely about the propagation of transient signals themselves.

[0187] Computer-readable storage media can be accessed by one or more local or remote computing devices, for example via access requests, queries or other data retrieval protocols, for various operations concerning the information stored on the media.

[0188] Communication media typically contain computer-readable instructions, data structures, program modules, or other structured or unstructured data in data signals (such as modulated data signals, such as carrier waves or other transmission mechanisms), and include any information delivery or transmission medium. The term "modulated data signal" or signal refers to a signal whose characteristics are set or altered to encode information in one or more signals. By way of example, and not limitation, communication media include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media).

[0189] Refer again Figure 18 Example environment 1800 for implementing various embodiments of the aspects described herein includes a computer 1802, which includes a processing unit 1804, system memory 1806, and a system bus 1808. The system bus 1808 couples system components, including but not limited to the system memory 1806, to the processing unit 1804. The processing unit 1804 can be any of a variety of commercially available processors. Dual microprocessors and other multiprocessor architectures may also be used as the processing unit 1804.

[0190] System bus 1808 can be any of several types of bus structures that can be further interconnected to a memory bus (with or without a memory controller), peripheral bus, and local bus using any of a variety of commercially available bus architectures. System memory 1806 includes ROM 1810 and RAM 1812. The Basic Input / Output System (BIOS) can be stored in non-volatile memory such as ROM, erasable programmable read-only memory (EPROM), or EEPROM, where the BIOS contains basic routines that facilitate, for example, transferring information between components within computer 1802 during startup. RAM 1812 may also include high-speed RAM, such as static RAM for caching data.

[0191] Computer 1802 further includes an internal hard disk drive (HDD) 1814 (e.g., EIDE, SATA), one or more external storage devices 1816 (e.g., floppy disk drive (FDD) 1816, memory stick or flash drive reader, memory card reader, etc.), and drives 1820 (e.g., solid-state drives, optical disc drives) that can read from or write to disks 1822 (e.g., CD-ROMs, DVDs, BDs, etc.). Alternatively, in cases involving solid-state drives, disks 1822 are not included unless they are separate. Although the internal HDD 1814 is illustrated as being located within computer 1802, the internal HDD 1814 may also be configured for external use in a suitable rack (not illustrated). Additionally, although not shown in environment 1800, solid-state drives (SSDs) may be used as a supplement to or alternative to the HDD 1814. HDD 1814, external storage device 1816, and drive 1820 can be connected to system bus 1808 via HDD interface 1824, external storage interface 1826, and drive interface 1828, respectively. Interface 1824 for the specific implementation of the external drive may include at least one or both of Universal Serial Bus (USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technologies. Other external drive connection technologies are also considered in the implementations described herein.

[0192] Drives and their associated computer-readable storage media provide non-volatile storage of data, data structures, computer-executable instructions, etc. For computer 1802, drives and storage media are adapted to store any data in a suitable digital format. Although the above description of computer-readable storage media refers to corresponding types of storage devices, those skilled in the art will understand that other types of computer-readable storage media (whether currently existing or developed in the future) may also be used in the example operating environment, and further, any such storage media may contain computer-executable instructions for performing the methods described herein.

[0193] The driver and RAM 1812 can store multiple program modules, including an operating system 1830, one or more application programs 1832, other program modules 1834, and program data 1836. All or part of the operating system, applications, modules, or data may also be cached in RAM 1812. The systems and methods described herein can be implemented using various commercially available operating systems or combinations of operating systems.

[0194] Computer 1802 may optionally include emulation technology. For example, a hypervisor (not shown) or other middleware may emulate the hardware environment used for operating system 1830, and the emulated hardware may optionally be different from that of operating system 1830. Figure 18The illustrated hardware. In such implementations, operating system 1830 may include one of a plurality of virtual machines (VMs) hosted at computer 1802. Furthermore, operating system 1830 may provide a runtime environment for application 1832, such as the Java Runtime Environment or the .NET Framework. A runtime environment is a consistent execution environment that allows application 1832 to run on any operating system that includes a runtime environment. Similarly, operating system 1830 may support containers, and application 1832 may be in the form of containers, which are lightweight, stand-alone, executable software packages that include, for example, code, runtime, system tools, system libraries, and settings for the application.

[0195] Furthermore, the computer 1802 may be equipped with a security module, such as a Trusted Processing Module (TPM). For example, using a TPM, the boot unit hashes the next boot unit over time and waits for the result to match a security value before loading the next boot unit. This process can occur at any layer of the computer 1802's code execution stack, such as at the application execution level or the operating system (OS) kernel level, thereby achieving security at any code execution level.

[0196] Users can input commands and information into computer 1802 using one or more wired / wireless input devices (e.g., keyboard 1838, touchscreen 1840, and pointing devices such as mouse 1842). Other input devices (not shown) may include microphones, infrared (IR) remote controls, radio frequency (RF) remote controls or other remote controls, joysticks, virtual reality controllers or virtual reality headsets, game controllers, styluses, image input devices (e.g., cameras), gesture sensor input devices, visual motion sensor input devices, emotion or face detection devices, or biometric input devices (e.g., fingerprint or iris scanners), etc. These and other input devices are typically connected to processing unit 1804 via input device interface 1844, which may be coupled to system bus 1808, but these and other input devices may also be connected via other interfaces (e.g., parallel ports, IEEE 1394 serial ports, game ports, USB ports, IR interfaces, Bluetooth, etc.). ® (Interfaces, etc.) connections.

[0197] Monitor 1846 or other types of display devices may also be connected to system bus 1808 via an interface (such as video adapter 1848). In addition to monitor 1846, the computer typically includes other peripheral output devices (not shown), such as speakers, printers, etc.

[0198] Computer 1802 can operate in a networked environment using a logical connection to one or more remote computers (such as remote computer 1850) via wired or wireless communication. Remote computer 1850 can be a workstation, server computer, router, personal computer, portable computer, microprocessor-based entertainment device, peer-to-peer device, or other common network node, and typically includes many or all of the elements described relative to computer 1802; however, for simplicity, only memory / storage device 1852 is illustrated. The depicted logical connection includes a wired / wireless connection to a local area network (LAN) 1854 or a larger network (e.g., a wide area network (WAN) 1856). Such LAN and WAN networking environments are common in offices and companies and facilitate the establishment of enterprise-wide computer networks (such as intranets) where all networks can connect to global communications networks (e.g., the Internet).

[0199] When used in a LAN networking environment, computer 1802 can connect to local network 1854 via a wired or wireless communication network interface or adapter 1858. Adapter 1858 facilitates wired or wireless communication with LAN 1854, which may also include a wireless access point (AP) configured thereon for communication with adapter 1858 in wireless mode.

[0200] When used in a WAN networking environment, computer 1802 may include modem 1860, or may be connected to a communication server on WAN 1856 via other components for establishing communication over WAN 1856 (such as via the Internet). Modem 1860 may be connected to system bus 1808 via input device interface 1844; modem 1860 may be internal or external, and may be a wired or wireless device. In a networking environment, program modules depicted relative to computer 1802 or parts thereof may be stored in remote memory / storage device 1852. It should be understood that the network connections shown are exemplary, and other components for establishing communication links between computers may be used.

[0201] When used in a LAN or WAN networking environment, computer 1802 can access cloud storage systems or other network-based storage systems that supplement or replace the external storage device 1816 described above, such as, but not limited to, network virtual machines that provide one or more aspects of information storage or processing. Typically, the connection between computer 1802 and the cloud storage system can be established, for example, via adapter 1858 or modem 1860 through LAN 1854 or WAN 1856. When computer 1802 is connected to the associated cloud storage system, external storage interface 1826 can manage the storage provided by the cloud storage system with the help of adapter 1858 or modem 1860, just as it would manage other types of external storage devices. For example, external storage interface 1826 can be configured to provide access to cloud storage sources as if these sources were physically connected to computer 1802.

[0202] Computer 1802 may be operable to communicate with any wirelessly configured device or entity (e.g., printer, scanner, desktop or portable computer, portable data assistant, communications satellite, any equipment or location associated with a wirelessly detectable tag (e.g., kiosk, newsstand, store shelf, etc.) and telephone). This may include Wi-Fi and Bluetooth. ® Wireless technology. Therefore, communication can be a predefined structure like a traditional network, or it can be self-organizing communication between at least two devices.

[0203] Figure 19 This is a schematic block diagram of a sample computing environment 1900 to which the disclosed subject matter can interact. The sample computing environment 1900 includes one or more clients 1910. Clients 1910 can be hardware or software (e.g., threads, processes, computing devices). The sample computing environment 1900 also includes one or more servers 1930. Servers 1930 can also be hardware or software (e.g., threads, processes, computing devices). For example, server 1930 can accommodate threads to perform transformations by employing one or more embodiments described herein. One possible communication between client 1910 and server 1930 may be in the form of data packets suitable for transmission between two or more computer processes. The sample computing environment 1900 includes a communication framework 1950 for facilitating communication between client 1910 and server 1930. Client 1910 is operatively connected to one or more client data repositories 1920 for storing information local to client 1910. Similarly, server 1930 is operatively connected to one or more server data repositories 1940 for storing information local to server 1930.

[0204] Figure 20The document illustrates example non-limiting apparatuses for carrying out the various embodiments described herein. Figure 20 A non-limiting example of a dual-beam system 2010 is illustrated, comprising a vertically mounted scanning electron microscope (SEM) column and a focused ion beam (FIB) column mounted at an angle of approximately 52 degrees to the vertical. Such a dual-beam system is commercially available, for example, from FEI Company, Hillsboro, Oregon, the assignee of this application. Although Figure 20 Examples of suitable microscopy hardware for implementing the various embodiments described herein are shown, but it should be understood that such microscopy hardware is non-limiting. In other words, the various embodiments described herein can be implemented in combination with any other suitable type of microscopy hardware. The dual-beam system 2010 is a non-limiting example of scientific instrument 302 or any other scientific instrument discussed above.

[0205] A scanning electron microscope 2041 and a power supply and control unit 2045 are provided together with a dual-beam system 2010. An electron beam 2043 is emitted from the cathode 2052 by applying a voltage between the cathode 2052 and the anode 2054. The electron beam 2043 can be focused into a fine point using a condenser lens 2056 and an objective lens 2058. The electron beam 2043 can be scanned in two dimensions on any suitable sample using a deflection coil 2060. The operation of the condenser lens 2056, the objective lens 2058, or the deflection coil 2060 can be controlled by the power supply and control unit 2045.

[0206] An electron beam 2043 can be focused onto a substrate 2022, which can be located on a movable XY stage 2025 within a lower chamber 2026. When electrons in the electron beam 2043 strike the substrate 2022, secondary electrons can be emitted. These secondary electrons are detected by a secondary electron detector 2040, as discussed below. A scanning transmission electron microscope (STEM) detector 2062, located below the transmission electron microscope (TEM) sample holder 2024 and the movable XY stage 2025, can collect electrons transmitted through the sample mounted on the TEM sample holder 2024, as discussed above.

[0207] The dual-beam system 2010 may also include a focused ion beam (FIB) system 2011, which may include a vacuum chamber having an upper neck portion 2012, an ion source 2014, and a focusing column 2016 including extraction electrodes and an electrostatic optics system located in the upper neck portion. The axis of the focusing column 2016 may be tilted 52 degrees (or any other suitable angular displacement) from the axis of the electron column. The ion column 2012 may include the ion source 2014, extraction electrodes 2015, focusing elements 2017, deflection elements 2020, and a focused ion beam 2018. The focused ion beam 2018 may pass from the ion source 2014 through the focusing column 2016 and be delivered toward a substrate 2022 between electrostatic deflection members schematically indicated by the number 2020. This substrate may include, for example, semiconductor devices on a movable XY stage 2025 located within a lower chamber 2026.

[0208] The movable XY stage 2025 can move vertically (along the Z-axis) in a horizontal plane (along the X and Y axes). The movable XY stage 2025 can tilt approximately sixty (60) degrees and rotate about the Z-axis. In some embodiments, a separate TEM sample stage (not shown) may be used. Such a TEM sample stage can move along the X, Y, and Z axes. A door 2061 can be opened for inserting a substrate 2022 onto the movable XY stage 2025, or also for maintaining an internal gas supply reservoir (if used). The door 2061 can be interlocked so that it cannot be opened if the system is under vacuum.

[0209] An ion pump 2068 can be used to evacuate the neck section 2012. Under the control of a vacuum controller 2032, the chamber 2026 can be evacuated using a turbomolecular and mechanical pumping system 2030. This type of vacuum system can provide approximately 1 × 10⁻⁶ vacuum within the chamber 2026. -7 5×10 -4 The vacuum between the chambers. If etching aids, etching restraint gases, or deposition precursor gases are used, the chamber floor pressure may increase, typically to about 1 × 10⁻⁶. -5 Entrust.

[0210] A high-voltage power supply 2034 can provide an appropriate accelerating voltage to the electrodes in the focusing column 2016 to excite the focused ion beam 2018. When the focused ion beam strikes the substrate 2022, material can be sputtered (i.e., physically ejected) from the sample. Alternatively, the focused ion beam 2018 can decompose a precursor gas to deposit material.

[0211] A high-voltage power supply 2034 can be connected to an ion source 2014 (which may be a liquid metal ion source) and appropriate electrodes in the ion beam focusing column 2016 to form an ion beam 2018 of approximately 1 keV to 60 keV and guide it to the sample. A deflection controller and amplifier 2036, operating according to a predetermined pattern provided by a pattern generator 2038, can be coupled to a deflection element 2020 (which may be a deflection plate), thereby allowing manual or automatic control of the focused ion beam 2018 to trace a corresponding pattern on the upper surface of the substrate 2022. In some systems, the deflection element 2020 may be placed before the final lens. When a blanking controller (not shown) applies a blanking voltage to the blanking electrode, the blanking electrode (not shown) within the ion beam focusing column 2016 causes the focused ion beam 2018 to strike the blanking aperture (not shown) instead of the substrate 2022.

[0212] Ion source 2014 can provide a metal ion beam, such as gallium. In other examples, ion source 2014 can be a plasma ion source that extracts ions from generated plasma. This source can be focused into a beam sub-1 / 10 of a micrometer wide at substrate 2022 for modifying substrate 2022 by ion milling, enhanced etching, material deposition, or for imaging substrate 2022.

[0213] A charged particle detector 2040 (such as an Everhart Thornley or multichannel board) for detecting secondary ion or electron emissions can be connected to video circuitry 2042, which supplies drive signals to video monitor 2044 and receives deflection signals from system controller 2019. The orientation of the charged particle detector 2040 within the lower chamber 2026 can vary in different embodiments. For example, the charged particle detector 2040 may be coaxial with the ion beam and include apertures for allowing the ion beam to pass through. In other embodiments, secondary particles may be collected by a final lens and then deflected off-axis for collection.

[0214] The micromanipulator 2047 can precisely move an object within a vacuum chamber. The micromanipulator 2047 may include a precision electric motor 2048 located outside the vacuum chamber to provide X, Y, Z, and θ control of a portion 2049 located within the vacuum chamber. The micromanipulator 2047 may be equipped with different end effectors for manipulating small objects. In the various embodiments described herein, the end effector may be a fine probe 2050.

[0215] The gas delivery system 2046 may extend into the lower chamber 2026 for introducing and guiding gaseous vapor to the substrate 2022. A suitable gas delivery system 2046 is described in U.S. Patent 5,851,413, entitled "Gas Delivery Systems for Particle Beam Processing," assigned to the assignee of this invention, entitled "Gas Delivery Systems for Particle Beam Processing," and another gas delivery system is described in U.S. Patent 5,435,850, entitled "Gas Injection System," also assigned to the assignee of this invention. For example, iodine may be delivered to enhance etching, or organometallic compounds may be delivered to deposit metal.

[0216] The system controller 2019 controls the operation of various components of the dual-beam system 2010. Through the system controller 2019, a user can scan the focused ion beam 2018 or electron beam 2043 in a desired manner by entering commands into any suitable user interface (not shown). Alternatively, the system controller 2019 can control the dual-beam system 2010 according to programming instructions stored in memory 2021. In various embodiments, any one of one or more software components 319 may be implemented in the system controller 2019 or otherwise executed by the system controller.

[0217] Various implementations can be systems, methods, apparatus, or computer program products at any possible level of technical detail integration. Computer program products may include computer-readable storage media (or media) having computer-readable program instructions for causing a processor to execute aspects of various implementations. Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media may also include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punched cards or raised structures in grooves on which instructions are recorded), and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses transmitted through fiber optic cables), or electrical signals transmitted through wires.

[0218] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or via a network (e.g., the Internet, a local area network, a wide area network, or a wireless network) to an external computer or external storage device. This network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the corresponding computing / processing device. The computer-readable program instructions used to perform operations in various implementation schemes can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). Computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an internet service provider via the internet). In some implementations, the electronic circuitry includes, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), which can be personalized by utilizing state information from the computer-readable program instructions to execute the instructions for various purposes.

[0219] This document describes various aspects with reference to flowchart illustrations or block diagrams of methods, apparatus (systems), and computer program products according to various embodiments. It should be understood that each block in the flowchart illustrations or block diagrams, and combinations of blocks in the flowchart illustrations or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create components for implementing the functions / actions specified in one or more blocks of the flowchart or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, or other device to operate in a particular manner. The computer-readable storage medium storing the instructions includes an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart or block diagram. The computer-readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to perform a series of operational actions on the computer, other programmable apparatus, or other device, thereby producing a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / actions specified in one or more blocks of the flowchart or block diagram.

[0220] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible specific implementations of systems, methods, and computer program products according to various embodiments. In this regard, each box in a flowchart or block diagram may represent a module, segment, or portion of instructions, containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in the boxes may not occur in the order shown in the figure. For example, two consecutively shown blocks may actually be executed substantially simultaneously, or sometimes in reverse order, depending on the functionality involved. It should also be noted that each box in a block diagram or flowchart, and combinations of boxes in a block diagram or flowchart, may be implemented by a dedicated hardware system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0221] Although the subject matter has been described above in the general context of computer-executable instructions for computer program products already running on a computer, those skilled in the art will recognize that this disclosure may also be implemented in conjunction with other program modules. Typically, program modules include routines, programs, components, data structures, etc., that perform a specific task or implement a specific abstract data type. Furthermore, those skilled in the art will recognize that various aspects can be practiced using other computer system configurations, including single-processor or multi-processor computer systems, small computing devices, mainframe computers, and computers, handheld computing devices (e.g., PDAs, telephones), microprocessor-based or programmable consumer or industrial electronics, etc. The illustrated aspects can also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected via a communication network. However, some, if not all, aspects of this disclosure can be practiced on a standalone computer. In a distributed computing environment, program modules may reside in both local memory storage devices and remote memory storage devices.

[0222] The terms “component,” “system,” “platform,” “interface,” etc., used in this application may refer to or include computer-related entities or entities associated with an operating machine having one or more specific functionalities. Entities disclosed herein may be hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process, processor, object, executable program, executing thread, program, or computer running on a processor. As an example, both an application running on a server and the server itself can be components. One or more components may reside within a process or executing thread, and components may be located on a single computer or distributed across two or more computers. In another example, a corresponding component may be executed from various computer-readable media on which various data structures are stored. These components may communicate, for example, via local or remote processes based on signals having one or more data packets (e.g., data from one component interacts with another component in a local system, a distributed system, or with other systems via a network such as the Internet). As another example, a component may be a device having specific functionalities provided by mechanical parts operated by electrical or electronic circuitry, which is operated by software or firmware applications executed by a processor. In such cases, the processor may be internal or external to the device and may execute at least a portion of the software or firmware application. As another example, a component may be a device that provides specific functionality through electronic components without mechanical parts, wherein the electronic components may include a processor or other components for executing software or firmware that at least partially endows the electronic components with functionality. In one aspect, the component may be emulated via a virtual machine, for example, within a cloud computing system.

[0223] Furthermore, the term “or” is intended to mean inclusive “or” rather than exclusive “or.” That is, unless otherwise specified or clearly apparent from the context, “X uses A or B” is intended to mean any natural inclusive permutation. That is, if X uses A; X uses B; or X uses both A and B, then “X uses A or B” is satisfied under any of the foregoing examples. As used herein, the term “and / or” is intended to have the same meaning as “or.” Furthermore, unless otherwise specified or clearly apparent from the context involving the singular form, the article “a” as used in this specification and figures should generally be interpreted as meaning “one or more.” As used herein, the terms “example” or “exemplary” are used to indicate as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited to such examples. Furthermore, any aspect or design described herein as “example” or “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor does it imply exclusion of equivalent exemplary structures and techniques known to one of ordinary skill in the art.

[0224] The disclosure herein describes non-limiting examples. For ease of description or explanation, the various parts disclosed herein use the terms "each," "every," or "all" when discussing various examples. Such use of the terms "each," "every," or "all" is not restrictive. In other words, when the disclosure herein provides a description of "each," "every," or "all" applicable to a particular object or component, it should be understood that this is only a non-limiting example, and it should also be understood that in various other examples, such description may apply to fewer than "each," "every," or "all" descriptions of that particular object or component.

[0225] As used herein, the term "processor" can refer to substantially any computing processing unit or device, including but not limited to a single-core processor; a single processor with software multithreading capabilities; a multi-core processor; a multi-core processor with software multithreading capabilities; a multi-core processor with hardware multithreading technology; a parallel platform; and a parallel platform with distributed shared memory. Additionally, a processor can refer to an integrated circuit, application-specific integrated circuit (ASIC), digital signal processor (DSP), field-programmable gate array (FPGA), programmable logic controller (PLC), complex programmable logic device (CPLD), discrete gate or transistor logic components, discrete hardware components, or any combination thereof, designed to perform the functions described herein. Furthermore, processors can utilize nanoscale architectures (such as, but not limited to, molecular and quantum dot-based transistors, switches, and gates) to optimize space usage or enhance the performance of user equipment. Processors can also be implemented as a combination of computing processing units. In this disclosure, terms such as "repository," "storage device," "data repository," "data storage device," "database," and substantially any other information storage component related to the operation and functionality of a component are used to refer to a "memory component," an entity embodied in "memory," or a component that includes memory. It should be understood that memory and / or memory components can be volatile or non-volatile memory, or may include both. By way of example, and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). Volatile memory may include RAM, which may, for example, serve as external cache memory. By way of example, and not limitation, RAM takes many forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous linked DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Furthermore, the memory components of the systems or computer-implemented methods disclosed herein are intended to include, but are not limited to, these and any other suitable types of memory.

[0226] The foregoing only includes examples of systems and computer-implemented methods. It is certainly impossible to describe every conceivable combination of components or computer-implemented methods for the purposes of describing this disclosure, but many further combinations and arrangements of this disclosure are possible. Furthermore, with regard to the use of the terms “comprising,” “having,” “possessing,” etc., in the detailed description, claims, appendices, and drawings, these terms are intended to be inclusive in a manner similar to how the term “comprising” is interpreted when used as a transitional word in the claims.

[0227] Various embodiments have been described for illustrative purposes, but these descriptions are not intended to be exhaustive or limited to the embodiments disclosed herein. Many modifications and variations are obvious without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements relative to technology on the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0228] Various non-limiting aspects are described in the following embodiments.

[0229] Example 1: A system may include: a processor that executes computer-executable components stored in a non-transitory computer-readable memory, wherein these computer-executable components may include: a workflow component that enables a charged particle microscope to perform a workflow on a sample; a status component that, in response to the charged particle microscope generating an error message that interrupts the workflow, can retrieve runtime data recorded by the charged particle microscope during the workflow; and a model component that can synthesize a first text explaining why the workflow was interrupted by performing a large language model on the error message and the runtime data.

[0230] Example 2: A system according to any of the foregoing embodiments can be implemented, wherein the model component can also synthesize a second text explaining how to parse the workflow by performing the large language model on the first text.

[0231] Example 3: A system according to any of the foregoing embodiments can be implemented, wherein the model component can also synthesize an encoding script by performing the large language model on the second text, the encoding script being configured to cause the charged particle microscope to resolve the workflow.

[0232] Example 4: A system according to any of the foregoing embodiments can be implemented, wherein these computer-executable components further include: an execution component that enables the charged particle microscope to execute the coded script, thereby parsing the workflow.

[0233] Example 5: The system according to any of the foregoing embodiments can be implemented, wherein the workflow may include: imaging scanning of the sample; milling operation of the sample; or movement of an actuable stage holding the sample.

[0234] Example 6: A system according to any of the foregoing embodiments can be implemented, wherein the runtime data may include: a partial image of the sample captured by the charged particle microscope during the workflow; or the state of one or more configurable operating parameters of the charged particle microscope during the workflow.

[0235] Example 7: A system according to any of the foregoing embodiments can be implemented, wherein the large language model can perform an embedding search based on a technical document library associated with the design, manufacture, operation, or troubleshooting of the charged particle microscope to retrieve the first text synthesized in an enhanced generative manner, and wherein the computer-executable components may further include: an execution component that can receive feedback from a user or technician regarding the first text, wherein the model component can synthesize a text edit of a document in the technical document library retrieved through the embedding search by performing the large language model on the feedback, and wherein the execution component can update the technical document library by inserting the text edit into the document.

[0236] Example 8: A system according to any of the foregoing embodiments can be implemented, wherein these computer-executable components may further include: an execution component that can visually or audibly present the first text on an electronic screen or electronic speaker.

[0237] In various implementation schemes, any one or more combinations of the embodiments described in Examples 1 to 8 may be implemented.

[0238] Example 9: A computer-implemented method may include a device operatively coupled to a processor causing a charged particle microscope to perform a workflow on a sample; the device and in response to the charged particle microscope generating an error message that interrupts the workflow retrieving runtime data recorded by the charged particle microscope during the workflow; and the device and via performing a large language model on the error message and the runtime data to synthesize a first text explaining why the workflow was interrupted.

[0239] Example 10: A computer-implemented method according to any of the foregoing embodiments can be implemented, the computer-implemented method further comprising: synthesizing a second text by means of the device and by means of performing the large language model on the first text to explain how to parse the workflow.

[0240] Example 11: A computer-implemented method according to any of the foregoing embodiments can be implemented, the computer-implemented method further comprising: synthesizing an encoding script by the device and via executing the large language model on the second text, the encoding script being configured to cause the charged particle microscope to resolve the workflow.

[0241] Example 12: A computer-implemented method according to any of the foregoing embodiments can be implemented, the computer-implemented method further comprising: the device causing the charged particle microscope to execute the coded script, thereby parsing the workflow.

[0242] Example 13: A computer-implemented method according to any of the foregoing embodiments can be implemented, wherein the workflow may include: imaging scanning of the sample; milling operation of the sample; or movement of an actuable stage holding the sample.

[0243] Example 14: A computer-implemented method according to any of the foregoing embodiments can be implemented, wherein the runtime data may include: a partial image of the sample captured by the charged particle microscope during the workflow; or the state of one or more configurable operating parameters of the charged particle microscope during the workflow.

[0244] Example 15: A computer-implemented method according to any of the foregoing embodiments can be implemented, wherein the large language model can perform an embedding search based on a technical document library associated with the design, manufacture, operation, or troubleshooting of the charged particle microscope to retrieve the first text synthesized in an enhanced generative manner, and further includes: receiving feedback from a user or technician regarding the first text by the device; synthesizing a text edit of a document in the technical document library, retrieved through the embedding search, by the device and via performing the large language model on the feedback; and updating the technical document library by the device by inserting the text edit into the document.

[0245] Example 16: A computer-implemented method according to any of the foregoing embodiments can be implemented, the computer-implemented method further comprising: visually or audibly presenting the first text on an electronic screen or electronic speaker by the device.

[0246] In various implementation schemes, any one or more combinations described in Examples 9 to 16 may be implemented.

[0247] Example 17: A computer program product for facilitating large language model parsing of scientific instrument workflow interruptions may include a non-transitory computer-readable storage medium containing program instructions. In various aspects, these program instructions may be executed by a processor to cause the processor to: cause the scientific instrument to perform a workflow on a sample; retrieve runtime data recorded by the scientific instrument during the workflow in response to the scientific instrument generating an error message that interrupts the workflow; and synthesize a first text explaining why the workflow was interrupted by performing a large language model on the error message and the runtime data.

[0248] Example 18: A computer program product described in any of the foregoing embodiments can be implemented, wherein these program instructions can be further executed to cause the processor to synthesize a second text explaining how to parse the workflow by executing the large language model on the first text.

[0249] Example 19: A computer program product described in any of the foregoing embodiments may be implemented, wherein the program instructions may be further executed to cause the processor to: synthesize an encoding script by executing the large language model on the second text, the encoding script being configured to cause the scientific instrument to parse the workflow; and cause the scientific instrument to execute the encoding script, thereby parsing the workflow.

[0250] Example 20: A computer program product described in any of the foregoing embodiments can be implemented, wherein the scientific instrument can be a charged particle microscope, a chromatograph, or a mass spectrometer.

[0251] In various implementation schemes, any one or more combinations described in Examples 17 to 20 may be implemented.

[0252] In various implementation schemes, any one or more combinations of the embodiments 1 to 20 may be implemented.

Claims

1. A system comprising: A processor that executes computer-executable components stored in a non-transitory computer-readable memory, wherein the computer-executable components include: A workflow component that enables a charged particle microscope to perform a workflow on a sample; A status component, in response to the charged particle microscope generating an error message that interrupts the workflow, retrieves runtime data recorded by the charged particle microscope during the workflow; and A model component that synthesizes a first text explaining why the workflow was interrupted by performing a large language model on the error message and the runtime data.

2. The system of claim 1, wherein the model component further synthesizes a second text explaining how to parse the workflow by performing the large language model on the first text.

3. The system of claim 2, wherein the model component further synthesizes an encoding script by performing the large language model on the second text, the encoding script being configured to cause the charged particle microscope to resolve the workflow.

4. The system of claim 3, wherein the computer-executable component further comprises: An execution unit that causes the charged particle microscope to execute the coded script, thereby parsing the workflow.

5. The system according to claim 1, wherein the workflow comprises: Imaging scan of the sample; The milling operation of the sample; or the movement of an actuated table holding the sample.

6. The system according to claim 1, wherein the runtime data includes: A partial image of the sample captured by the charged particle microscope during the workflow; Or the state that one or more configurable operating parameters of the charged particle microscope have during the workflow.

7. The system of claim 1, wherein the large language model is based on retrieving the first text synthesized in an enhanced generative manner by performing an embedding search through a technical document library associated with the design, manufacture, operation, or troubleshooting of the charged particle microscope, and wherein the computer-executable component further comprises: An execution component receives feedback from a user or technician regarding the first text, wherein a model component synthesizes a text edit of a document in the technical document library retrieved via the embedding search by executing the large language model on the feedback, and wherein the execution component updates the technical document library by inserting the text edit into the document.

8. The system of claim 1, wherein the computer-executable component further comprises: An execution component that visually or audibly presents the first text on an electronic screen or electronic speaker.

9. A computer-implemented method, the computer-implemented method comprising: Devices operatively coupled to the processor enable the charged particle microscope to perform a workflow on the sample; The device, in response to the generated error message from the charged particle microscope that interrupts the workflow, retrieves runtime data recorded by the charged particle microscope during the workflow; as well as The device synthesizes a first text explaining why the workflow was interrupted by performing a large language model on the error message and the runtime data.

10. The computer-implemented method according to claim 9, further comprising: The device synthesizes a second text explaining how to parse the workflow by performing the large language model on the first text.

11. The computer-implemented method according to claim 10, further comprising: The device synthesizes an encoding script by performing the large language model on the second text, the encoding script being configured to cause the charged particle microscope to resolve the workflow.

12. The computer-implemented method according to claim 11, further comprising: The device causes the charged particle microscope to execute the coded script, thereby resolving the workflow.

13. The computer-implemented method of claim 9, wherein the workflow comprises: Imaging scan of the sample; The milling operation of the sample; or the movement of an actuated table holding the sample.

14. The computer-implemented method of claim 9, wherein the runtime data includes: A partial image of the sample captured by the charged particle microscope during the workflow; Or the state that one or more configurable operating parameters of the charged particle microscope have during the workflow.

15. The computer-implemented method of claim 9, wherein the large language model is based on performing an embedding search through a technical document library associated with the design, manufacture, operation, or troubleshooting of the charged particle microscope to retrieve the first text synthesized in an enhanced generative manner, and the computer-implemented method further comprises: The device receives feedback about the first text from the user or technician; Text edits of documents in the technical document library, retrieved through the embedded search, are synthesized by the device and by performing the large language model on the feedback; as well as The device updates the technical document library by inserting the text edits into the document.

16. The computer-implemented method according to claim 9, further comprising: The device presents the first text visually or audibly on an electronic screen or electronic speaker.

17. A computer program product for facilitating the parsing of large language models that disrupt scientific instrument workflows, the computer program product comprising a non-transitory computer-readable storage medium containing program instructions executable by a processor to cause the processor to: Enables scientific instruments to execute workflows on samples; In response to the scientific instrument generating an error message that interrupts the workflow, retrieve runtime data recorded by the scientific instrument during the workflow; and A first text explaining why the workflow was interrupted is synthesized by performing a large language model on the error message and the runtime data.

18. The computer program product of claim 17, wherein the program instructions are further executable to cause the processor to: A second text explaining how to parse the workflow is synthesized by performing the large language model on the first text.

19. The computer program product of claim 18, wherein the program instructions are further executable to cause the processor to: An encoding script is synthesized by performing the large language model on the second text, the encoding script being configured to cause the scientific instrument to parse the workflow; and The scientific instrument executes the coded script, thereby parsing the workflow.

20. The computer program product of claim 17, wherein the scientific instrument is a charged particle microscope, a chromatograph, or a mass spectrometer.

Images