Automatic SOAP note generation using task decomposition

Abstract

Techniques for automatically generating subjective, objective, evaluation and planning (SOAP) notes are disclosed. In particular, techniques are disclosed for automatic SOAP note generation using task decomposition. A text transcription is accessed and segmented into portions. The text transcription may correspond to an interaction between the first entity and the second entity. The machine learning model cues entities and facts for extracting the respective portions, and generates SOAP note segments based at least in part on the facts. The SOAP note is generated by combining the SOAP note segments. The SOAP note may be stored in a database in association with at least one of the first entity and the second entity.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit and priority of U.S. Provisional Application No. 63 / 583,224, filed September 15, 2023, the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0003] This disclosure relates to the automatic generation of subjective, objective, evaluative, and planning (SOAP) notes. In particular, this disclosure relates to techniques for the automatic generation of SOAP notes using task decomposition. Background Technology

[0004] Clinical settings, such as healthcare facilities, often involve different healthcare providers working together and communicating with each other to treat patients. Recording patient encounters, capturing information conveyed during these encounters and / or about events that occurred before and / or after the encounter, filling in patient records such as electronic health records, and managing healthcare practices are integral parts of the practices of many healthcare providers and are important for ensuring high-quality healthcare. Traditional means of performing tasks associated with providing healthcare typically involve several different devices, such as listening devices, portable electronic devices, workstations, etc., and end-users with the training, knowledge, experience, and skills to properly utilize these devices and participate in healthcare processes. Relying on different devices and qualified end-users to perform clinical tasks and practice healthcare is cumbersome, time- and resource-intensive, expensive, and inefficient, potentially leading to lower-quality healthcare. Summary of the Invention

[0005] The techniques disclosed in this paper relate to the automatic generation of subjective, objective, evaluative, and planning (SOAP) notes. In particular, this paper discloses techniques for the automatic generation of SOAP notes using task decomposition.

[0006] In some embodiments, a computer-implemented method includes: accessing a text transcription corresponding to an interaction between a first entity and a second entity; segmenting the text transcription into a plurality of parts; for each of the plurality of parts: identifying one or more entities contained in the corresponding part using a first machine learning model prompt; extracting one or more facts from the corresponding part based at least in part on the one or more entities using a second machine learning model prompt; adding the one or more facts to a fact set; generating a set of note segments based at least in part on the fact set using a plurality of third machine learning model prompts, wherein each note segment in the set of note segments corresponds to a segment of a SOAP note; generating a SOAP note by combining the note segments in the set of note segments; and storing the SOAP note in a database associated with at least one of the first entity and the second entity.

[0007] In some embodiments, the text transcription includes a first number of tokens, wherein each of the plurality of portions includes a second number of tokens less than the first number of tokens, and wherein segmenting the text transcription into the plurality of portions includes selecting a corresponding portion of the text transcription having a number of tokens corresponding to the second number of tokens, and determining whether the corresponding portion of the text transcription meets a predetermined criterion.

[0008] In some embodiments, using a first machine learning model prompt to identify one or more entities contained in a corresponding portion includes extracting at least one entity from the corresponding portion using a named entity recognition model, and generating a first machine learning model prompt based at least in part on the corresponding portion and the at least one entity.

[0009] In some embodiments, the second machine learning model prompt includes a corresponding part, one or more entities, and a query for querying one or more machine learning models to extract one or more facts from the corresponding part.

[0010] In some embodiments, the method further includes: generating a set of categorized facts from a fact set using a plurality of fourth machine learning model prompts, wherein each categorized fact in the set of categorized facts corresponds to a fact in the fact set and is associated with a specific category label selected from a set of category labels, wherein generating a set of note segments using a plurality of third machine learning model prompts includes generating the set of note segments based at least in part on the set of categorized facts using a plurality of third machine learning model prompts.

[0011] In some embodiments, each category label in the set of category labels is associated with a corresponding SOAP note segment in a plurality of SOAP note segments, and wherein generating a set of categorized facts from a fact set using a plurality of fourth machine learning model prompts includes classifying each corresponding fact in the fact set as corresponding to a specific SOAP note in the plurality of SOAP note segments using a plurality of fourth machine learning model prompts.

[0012] In some embodiments, generating a set of note segments using multiple third machine learning model prompts at least in part based on the set of categorized facts includes: extracting information associated with the second entity from electronic records associated with the second entity; dividing the set of categorized facts into subsets of categorized facts; generating machine learning model prompts based on subsets of categorized facts within subsets of categorized facts and the information; and generating corresponding note segments in the set of note segments using the machine learning model prompts.

[0013] In some embodiments, the first entity is a healthcare provider, the second entity is a patient associated with the healthcare provider, and storing SOAP notes in a database includes storing SOAP notes in an electronic health record associated with the patient.

[0014] Some embodiments include a system comprising one or more processing systems and one or more computer-readable media storing instructions that, when executed by the one or more processing systems, cause the system to perform some or all of the operations and / or methods disclosed herein.

[0015] Some embodiments include a non-transitory computer-readable medium storing one or more instructions that, when executed by one or more processing systems, cause the systems to perform some or all of the operations and / or methods disclosed herein.

[0016] In some embodiments, an apparatus is provided that includes some or all of the components for implementing the operations and / or methods disclosed herein.

[0017] In some embodiments, a computer program product is provided that includes computer instructions that, when executed by a processor, implement some or all of the operations and / or methods disclosed herein.

[0018] The techniques described above and below can be implemented in a variety of ways and in a variety of contexts. Several example implementations and contexts are provided with reference to the following figures, as described in more detail below. However, the following implementations and contexts are only a few of the many implementations and contexts. Attached Figure Description

[0019] The features, embodiments, and advantages of this disclosure will be better understood when reading the following detailed description with reference to the accompanying drawings.

[0020] Figure 1 This is a simplified diagram of an example environment for automatically generating subjective, objective, evaluative, and planning (SOAP) notes according to certain embodiments.

[0021] Figure 2 This is a simplified diagram of another example environment for automatically generating SOAP notes, based on certain embodiments.

[0022] Figure 3 An example of a task flow for automatically generating SOAP notes, according to certain embodiments, is described.

[0023] Figure 4 An example of a SOAP note generation task including subtasks according to certain embodiments is described.

[0024] Figure 5 Examples of processing for automatically generating SOAP notes using task decomposition, according to some embodiments, are described.

[0025] Figure 6 This is a block diagram illustrating a pattern for implementing a cloud infrastructure-as-a-service system according to certain embodiments.

[0026] Figure 7 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to certain embodiments.

[0027] Figure 8 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to certain embodiments.

[0028] Figure 9 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to certain embodiments.

[0029] Figure 10 This is a block diagram illustrating an example computer system according to certain embodiments. Detailed Implementation

[0030] In the following description, specific details are set forth for purposes of explanation in order to provide a thorough understanding of certain embodiments. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and descriptions are not intended to be limiting. The word “exemplary” as used herein means “serving as an example, instance, or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as preferred or superior to other embodiments or designs.

[0031] introduction

[0032] Healthcare providers frequently document their interactions with patients. Documenting patient interactions is often an integral part of a healthcare provider's practice workflow, which includes appointment scheduling, patient registration and examinations, patient treatment, billing, and more. Furthermore, documenting patient interactions helps healthcare providers provide a cognitive framework to follow when evaluating their patients during interactions. The term "healthcare provider" generally refers to healthcare practitioners and professionals, including but not limited to: physicians (e.g., general practitioners, specialists, surgeons, etc.); nursing professionals (e.g., nurse practitioners, physician assistants, nurses, registered nurses, licensed intern nurses, etc.); and other professionals (e.g., pharmacists, therapists, technicians, technical specialists, pathologists, dietitians, nutritionists, emergency medical technicians, psychiatrists, psychologists, counselors, dentists, orthodontists, hygienists, etc.).

[0033] Healthcare providers often document patient interactions by generating notes that are structured and organized in a specific format. For example, a note documenting a patient interaction might begin by identifying the patient and listing their physical characteristics (e.g., age, height, weight, eye color, hair color, etc.), then describe the interaction between the patient and the healthcare provider (e.g., statements made, clinical information conveyed such as symptoms, questions, concerns, problems, treatments, prescribed medications, additional information, etc.), and then they might provide a closing statement (e.g., as a summary, conclusion, additional recommendations, and information). Healthcare providers typically generate these notes in real-time (i.e., during the patient interaction) or immediately afterward, and are done manually (i.e., handwritten) and / or using electronic devices (i.e., typing). In some cases, a healthcare provider's assistant (e.g., a nurse) or scribe will generate notes for the healthcare provider by observing or listening to the patient interaction and / or based on information obtained from other sources (such as clinical records, healthcare provider notes, laboratory tests, etc.). Once a note is generated, it is typically stored in a record associated with the healthcare provider and / or the patient (e.g., the healthcare provider's electronic health management system, the patient's electronic health record, etc.).

[0034] An example of notes used by healthcare providers to record patient interactions is the Subjective, Objective, Evaluation, and Planning (SOAP) note. SOAP notes provide healthcare providers with a structured, organized, and standardized format for recording patient interactions, which facilitates the sharing of patient information in a clear, concise, universal, systematic, and easy-to-read manner. Typically, SOAP notes can be used as a template to update a patient's electronic health record after a patient's interaction with a healthcare provider. By using SOAP notes, healthcare providers seeking information about a patient can access SOAP notes generated based on previous interactions with the patient and easily understand the patient's condition and what happened during the patient's previous interactions with the healthcare provider. Other benefits of using SOAP notes include promoting better data integration and accessibility between healthcare systems, promoting correct documentation and data sharing practices, and reducing the risk of errors and omissions.

[0035] SOAP notes consist of a subjective component, an objective component, and an evaluation and planning component. The subjective component (also known as the present medical history or HPI component) records information about the patient. This information may include, but is not limited to, the patient's chief complaint, symptoms, problems, concerns, medical history, present illness, family medical and social history, current medications, and past medications. The objective component (also known as the systematic review or ROS component) records information collected during patient contact, such as vital signs, findings during physical examination, and results of diagnostic tests. The information recorded in the objective component distinguishes between symptoms and signs. The evaluation and planning component summarizes the patient's condition, including the patient's age, relevant medical history, primary diagnosis, and clinical stability, and records the healthcare provider's differential diagnosis and proposed plans to address the patient's medical problems. This includes treatment plans, further diagnostic testing, referrals, and follow-up guidance. Table 1 shows an example of a SOAP note:

[0036] Table 1:

[0037]

[0038] Healthcare providers often practice healthcare in a wide range of clinical settings (e.g., hospitals, physician offices, patient homes, etc.) with varying patient care goals, and typical provider-patient interactions can differ in the number of dialogue turns, ranging from as short as approximately 50 turns to as long as approximately 200 turns. SOAP note generation is a complex task. For SOAP notes to be effective and appropriate for their intended purpose, the person or entity writing the notes should be able to disregard non-medically relevant information conveyed during the patient interaction, understand, contextualize, and summarize the medical information conveyed, and organize the summarized medical information into the structure and format of the SOAP notes. SOAP notes are typically generated manually (i.e., handwritten, typed, or a combination thereof) by multiple entities present during the patient interaction, where each entity essentially acts as a scribe, listening to and observing the patient interaction and recording one or more aspects of the interaction to be included in the appropriate section of the SOAP note. Consequently, SOAP notes often vary in style, format, consistency, writing quality, grammar, readability, etc., which may reduce their effectiveness and the benefits they provide.

[0039] Traditional approaches to overcoming these challenges rely on machine learning techniques to automatically generate SOAP notes or portions thereof from audio recordings of healthcare provider-patient interactions and other information about those interactions. However, these traditional approaches are often error-prone and inconsistent due to the quality of information collected during patient interactions and the lack of readily available, low-cost training data. Other traditional approaches to overcoming these challenges utilize pre-trained language models, such as Large Language Models (LLMs), to automate one or more tasks involved in generating SOAP notes. However, these pre-trained language models are often expensive to license and operate, lack the healthcare privacy guarantees expected by custom, law, and regulation, and perform poorly in identifying medical information, especially digital information such as medications and dosages. In some cases, open-source pre-trained language models can be used, which can reduce costs and facilitate maintaining health privacy standards; however, these models often suffer from input window token limitations, inconsistent performance, non-compliance with SOAP note structure, and are prone to errors (e.g., generating illusions, using incorrect terminology, and omitting important facts, digital information, important entities, and other important details).

[0040] The techniques disclosed in this paper overcome the aforementioned and other challenges by providing an improved technique for automatically generating SOAP notes using one or more machine learning models, particularly a technique for automatically generating SOAP notes using task decomposition. By decomposing the SOAP note generation process into subtasks, advantageous technical effects can be provided, and technical problems such as input window token limitations, inconsistent performance, error susceptibility, and non-compliance with SOAP note structure can be overcome. Furthermore, by decomposing the SOAP note generation process into subtasks, personalized control over the SOAP generation process can be imposed, which reduces the need for manual review of the generated SOAP notes (e.g., by medical professionals or scribes), improves the quality level of individual parts of the SOAP notes, thereby improving the overall quality of the SOAP notes and contributing to healthcare privacy. Thus, high-quality SOAP notes can be automatically generated, leading to higher quality healthcare without incurring additional costs in terms of time, healthcare privacy, and financial resources.

[0041] In various embodiments, a computer-implemented method includes: accessing a text transcription corresponding to an interaction between a first entity and a second entity; segmenting the text transcription into multiple parts; for each of the multiple parts: identifying one or more entities contained in the corresponding part using a first machine learning model prompt; extracting one or more facts from the corresponding part at least partially based on the one or more entities using a second machine learning model prompt; adding the one or more facts to a fact set; generating a set of note segments at least partially based on the fact set using multiple third machine learning model prompts, wherein each note segment in the set of note segments corresponds to a segment of a SOAP note; generating a SOAP note by combining the note segments in the set of note segments; and storing the SOAP note in a database associated with at least one of the first entity and the second entity.

[0042] As used herein, when an action is “based on” something, it means that the action is based at least partially on at least a portion of that thing. As used herein, the terms “similarly,” “substantially,” “about,” and “approximately” are defined as substantially, but not necessarily entirely, as specified (and include entirely as specified), as understood by one of ordinary skill in the art. In any disclosed embodiment, the terms “similarly,” “substantially,” “about,” or “approximately” may be replaced with “within [percentage]”, where percentages include 0.1, 1, 5, and 7%.

[0043] Overview

[0044] Figure 1 A simplified diagram of a sample environment for automatically generating SOAP notes is shown. Figure 1As shown, environment 100 includes one or more client devices 110 (hereinafter referred to as "client devices 110"), one or more communication channels 112 (hereinafter referred to as "communication channels"), a cloud service provider platform 114 (hereinafter referred to as "platform 114"), one or more databases 122 (hereinafter referred to as "databases 122"), and one or more LLMs 124 (hereinafter referred to as "LLMs"). Platform 114, which may be included as part of a cloud infrastructure of a cloud service provider (e.g., Oracle Cloud Infrastructure or OCI), may be configured to communicate with, send data and information to, and receive data and information from client devices 110 via communication channels 112. Furthermore, platform 114 may be configured to access and / or invoke databases 122 and LLMs 124 to obtain and / or receive data and information from databases 122 and LLMs 124. Data and information received from client devices 110, databases 122, and LLMs 124 can be used by platform 114 to perform tasks and services, such as automatically generating SOAP notes. Figure 1 Database 122 and LLM 124 are shown to be separate from platform 114, but this is not intended to limit them, and one or more of database 122 and / or one or more of LLM 124 may be included as part of platform 114 and / or cloud infrastructure that includes platform 114.

[0045] Each client device included in client device 110 can be any type of electronic device capable of: executing applications; presenting information textually, graphically, and audibly, such as via a display and speakers; collecting information via one or more sensing elements (such as image sensors, microphones, tactile sensors, touchscreen displays, etc.); connecting to a communication channel such as communication channel 112 or a network such as a wireless network, wired network, public network, private network, etc. to send and receive data and information; and / or storing data and information locally in one or more storage media of the electronic device and / or in one or more locations remote from the electronic device (such as cloud-based storage systems, platform 114, and / or database 122). Examples of electronic devices include, but are not limited to, mobile phones, desktop computers, portable computing devices, computers, workstations, laptop computers, tablet computers, etc.

[0046] In some implementations, the application may be installed on, executed on, and / or accessed by a client device included in client device 110. An end user may utilize the application and / or its user interface and / or interact with it to access, utilize, and / or interact with one or more services provided by platform 114. The client device may be configured to receive various forms of input, such as touch, text, voice, etc., and the application may be configured to transform that input into one or more messages, which may be transmitted or streamed to platform 114 using one or more communication channels in communication channels 112. Furthermore, the client device may be configured to receive messages, data, and information from platform 114 using one or more communication channels in communication channels 112, and the application may be configured to present and / or render the received messages, data, and information in one or more user interfaces of the application.

[0047] Each communication channel included in communication channel 112 can be any type of communication channel capable of facilitating communication and data and / or information transmission between one or more entities (such as client device 110, platform 114, database 122, and LLM 124). Examples of communication channels include, but are not limited to, public networks, private networks, the Internet, wireless networks, wired networks, fiber optic networks, local area networks, wide area networks, etc. Communication channel 112 can be configured to facilitate the flow of data and / or information between and within one or more entities. In some implementations, one or more messages can be used and data and / or information can be streamed according to one or more protocols. Each of the one or more messages can be a variable-length message, and each communication channel included in communication channel 112 can include a stream orchestration layer that can receive variable-length messages according to a predefined interface, such as an interface description language like AsyncAPI. Each variable-length message can include context information that can be used to determine one or more routes for the variable-length message and a text or binary payload of arbitrary length. Each route in the routing can be configured using a polyglot streaming language that is agnostic to the details of the underlying implementation of the routing task and the destination.

[0048] Each database included in database 122 may be any type of database capable of storing and managing data and / or information. The data and / or information stored in each database may include data and / or information generated, provided, and / or otherwise obtained by platform 114. Additionally or alternatively, the data and / or information stored and / or managed by each database may include data and / or information generated, provided, and / or otherwise obtained by other sources, such as client device 110 and / or LLM 124. One or more databases included in database 122 may be part of a platform for storing and managing healthcare information, such as patients' electronic health records, healthcare provider's electronic records, etc., and may store and manage healthcare provider's patients' electronic health records. An example platform is the Oracle HealthMillenium Platform. Furthermore, one or more databases included in database 122 may be provided, managed, and / or otherwise included as part of a cloud infrastructure provided by a cloud service provider (e.g., Oracle Cloud Infrastructure or OCI). Data and / or information stored and / or managed by database 122 may be accessed using one or more application programming interfaces (APIs) of database 122.

[0049] Each LLM included in LLM 124 can be any type of LLM capable of obtaining, generating, or retrieving one or more results in response to one or more inputs, such as one or more prompts. Prompts for obtaining, generating, or retrieving results from or from LLM 124 can be obtained from or generated by or retrieved from or accessed from client device 110, database 112, platform 114, and / or one or more other sources, such as the Internet. Each prompt can be configured to cause LLM 124 to perform one or more tasks, resulting in one or more results being provided or generated, etc. Prompts for LLM 124 can be pre-generated (i.e., before a specific task requires them) and / or generated in real-time (i.e., when a specific task requires them). In some implementations, prompts for LLM 124 can be manually and / or engineered by one or more machine learning models to achieve one or more desired results. In some implementations, prompts for LLM 124 can be engineered on demand (i.e., in real-time and / or as needed) and / or at specific intervals (e.g., once daily, when an authenticated user logs into platform 114). Each of the one or more prompts may include a request or query for LLM 124 or a task to be performed by LLM 124, along with contextual information. Contextual information may include information such as text transcription or portions thereof, information about entities (e.g., information about healthcare providers, information about patients such as information contained in a patient's electronic health record, etc.), and / or other information or records (e.g., laboratory results, ambient temperature, etc.). The LLMs included in LLM 124 may be pre-trained, fine-tuned, open-source, off-the-shelf, licensed, subscription-based, etc. Furthermore, the LLMs included in LLM 124 may include or have a context window of any size (i.e., capable of accepting any number of tokens) and may be able to interpret complex instructions. One or more LLMs included in LLM 124 may be provided, managed, and / or otherwise included as part of the cloud infrastructure (e.g., Oracle Cloud Infrastructure or OCI) of Platform 114 and / or the cloud infrastructure of the cloud service provider supporting Platform 114. One or more LLMs contained in LLM 124 can be accessed using one or more APIs of LLM 124 and / or platforms that host, support or provide LLM 124.

[0050] Platform 114 can be configured to include various capabilities and provide various services to subscribers (e.g., end users). In some implementations, where the end user or subscriber is a healthcare provider, the healthcare provider can utilize the various services to facilitate the observation, care, treatment, management, etc., of their patient population. For example, the healthcare provider can utilize the functionality provided by the various services offered by Platform 114 to examine and / or treat and / or facilitate the examination and / or treatment of patients; view, edit, and / or manage patients' electronic health records; perform administrative tasks such as scheduling appointments, managing patient populations, and providing customer service to facilitate the operation of the healthcare environment in which the healthcare provider practices, etc.

[0051] In some implementations, the services provided by platform 114 may include, but are not limited to, voice service 116, digital assistant service 118, and SOAP note service 120. Voice service 116 may be configured to convert audio into text, such as text transcription. For example, voice service 116 may convert an audio recording of a conversation between a healthcare provider and a patient into a text transcription of the conversation. To convert audio into text, voice service 116 may utilize one or more machine learning models, such as an automatic speech recognition (ASR) model. Where audio is streamed to platform 114 in the form of messages (as described above), where each message includes a portion of audio (e.g., a one-second clip of audio), in some implementations, platform 114 and / or voice service 116 may be configured to aggregate and combine all audio-related messages (e.g., all conversation-related messages) into audio data and / or audio files, and then convert the audio data or audio files into text and / or text transcription. In other implementations, platform 114 and / or voice service 116 may be configured to convert audio into text or text transcription when platform 114 and / or voice service 116 receive audio. The text or text transcription generated by voice service 116 may be stored within platform 114 and / or in another location (such as one or more databases of database 122), where it may be accessed by platform 114, one or more other services of platform 114 (such as digital assistant service 118 and / or SOAP note service 120), and / or LLM 124. Additionally or alternatively, the text or text transcription generated by voice service 116 may be provided to one or more other services of platform 114, such as digital assistant service 118 and / or SOAP note service 120, and / or LLM 124.

[0052] Digital assistant service 118 can be configured to serve as an AI-driven conversational interface for platform 114, capable of conversing with end users (e.g., those using client device 110) and performing functions and / or tasks based on information conveyed and / or determined from those conversations and other sources. Digital assistant service 118 can be configured with and / or be configured to access natural language understanding (NLU) capabilities, such as natural language processing, named entity recognition, intent classification, etc. In some embodiments, digital assistant service 118 can be skill-driven, comprising bots, each including one or more skills for conversing and performing functions and / or tasks. In some embodiments, digital assistant service 118 can be LLM-based and agent-driven, wherein one or more agents coordinate with one or more LLMs to converse and perform functions and / or tasks. Examples of skill-driven, LLM-based, and agent-driven digital assistants are described in U.S. Patent Application No. 17,648,376, filed January 19, 2022, and U.S. Patent Application No. 18 / 624,472, filed April 2, 2024, each of which is incorporated herein by reference as if fully set forth herein.

[0053] Digital assistant service 118 can be configured to initiate conversations, drive previously initiated conversations (e.g., by responding to turns in a conversation), and / or otherwise participate in conversations. In some implementations, digital assistant service 118 can drive and / or participate in conversations and / or dialogues in response to events occurring at client device 110, platform 114, database 122, LLM 124, and / or at the cloud infrastructure supporting platform 114. In the case of skill-driven digital assistant service 118, events can be mapped to specific skills and can trigger a process for that skill. This process can then generate response events / messages, which can be used to render a user interface, such as the user interface of a client application executing on client device 110. In the case of LLM-based and agent-driven digital assistant service 118, events can be mapped to one or more specific prompts to retrieve one or more results for that one or more prompts, which can then be used to render a user interface. In some implementations, the digital assistant service 118 may drive and / or engage in dialogue and / or conversation in response to messages received from client device 110, platform 114, database 122, LLM 124, and / or the cloud infrastructure supporting platform 114. In the case of a skill-driven digital assistant service 118, messages may be routed to a specific skill, which, as described above, can generate responsive events / messages for rendering the user interface. In the case of an LLM-based and agent-driven digital assistant service 118, metadata included in the message may be used to generate and / or access one or more specific prompts to retrieve results and / or multiple results that can be used to render the user interface.

[0054] SOAP note service 120 can be configured to automatically generate SOAP notes. To generate SOAP notes, SOAP note service 120 can be configured to access and generate SOAP notes from text transcripts. Text transcripts can be transcripts generated from audio recordings by voice service 116 and / or transcripts stored in and / or accessed from platform 114, database 122, LLM 124, and / or another location (such as client device 110). Text transcripts can correspond to interactions between a first entity (e.g., a healthcare provider) and a second entity (e.g., a patient). In some implementations, text transcripts can correspond to interactions between a first entity, a second entity, and one or more additional entities. Text transcripts can be stored in a database or storage medium in association with the first entity, the second entity, and / or other entities. For example, text transcripts can be stored in a database of database 122 in association with one or more electronic records of a healthcare provider and / or one or more electronic health records of a patient of a healthcare provider. In some implementations, text transcripts can be derived from interactive audio recordings using machine learning models, such as an ASR model of voice service 116. In some embodiments, audio corresponding to the interaction between the first entity and the second entity can be recorded using an electronic device (e.g., a client device of client device 110), transmitted to platform 114 via communication channel 112, and converted into text transcription by voice service 116. In some embodiments, the audio recording can have any length and can be recorded as part of a process for generating SOAP notes. In some embodiments, to generate SOAP notes, platform 114 can trigger client device 110 to record audio and send the recorded audio to platform 114 and / or database 122, and SOAP note service 120 can invoke voice service 116 to generate a text transcription of the recorded audio and use the text transcription, or parts or fragments thereof, to generate SOAP notes. Similarly, in some embodiments, to generate SOAP notes, client device 110 can record audio and send the recorded audio to platform 114 and / or database 122, where voice service 116 can receive and / or access the recorded audio, generate a text transcription, and provide the text transcription to database 122 and / or SOAP note service 120, where it can be used to generate SOAP notes.

[0055] In some implementations, SOAP notes can be automatically generated in response to a specific trigger or instruction. For example, an end user of client device 110 may provide client device 110 with an instruction to generate SOAP notes (e.g., a voice command). Client device 110 may provide a message describing the instruction to platform 114 and / or digital assistant 118, and platform 114 and / or digital assistant 118 may invoke one or more services of platform 114 (such as voice service 116 and SOAP note service 120) to initiate processing for generating SOAP notes. In another example, digital assistant service 118 may learn of the end user's intention to generate SOAP notes through conversation and may invoke SOAP note service 120 to initiate processing for generating SOAP notes. In yet another example, a button displayed in the graphical user interface on client device 110 may be activated, causing platform 114 and / or its services to initiate processing for generating SOAP notes. In another example, voice service 116 may issue an alert to digital assistant service 118 and / or SOAP note service 120 that text transcription is available, and digital assistant service 118 and / or SOAP note service 120 may access the text transcription and begin processing for generating SOAP notes. In some implementations, SOAP notes may be generated automatically at specific intervals (e.g., once a day, at the end of each interaction between entities, etc.).

[0056] SOAP note service 120 can generate SOAP notes from text transcription using LLM 124. To generate SOAP notes from text transcription using LLM 124, SOAP note service 120 can access one or more prompts (e.g., one or more prompts stored in platform 114, database 112, and / or another location) and provide these one or more prompts to LLM 124 to obtain one or more results (hereinafter referred to as "results"). Each of the one or more prompts may include a request or query to LLM 124 or a task to be performed by LLM 124, as well as contextual information. Contextual information may include text transcription or portions or fragments of text transcription, information about the entities involved in the interaction (e.g., information about a healthcare provider, information about a patient, such as information contained in the patient's electronic health record, etc.), and / or other information or records (e.g., laboratory results, ambient temperature, etc.). Results may include SOAP notes and / or portions or segments of SOAP notes, which are combined or assembled into SOAP notes (e.g., by LLM 124 or other processing). In some implementations, results may be processed to improve the results. For example, one or more prompts may include prompts that, when provided to LLM 124, cause LLM 124 to process SOAP notes to improve and / or enhance SOAP notes if the result includes SOAP notes, and / or process that portion or section of SOAP notes to improve and / or enhance that portion or section if the result includes a portion or section of SOAP notes.

[0057] In some implementations, SOAP notes can be generated as a single task, where one or more prompts are provided to the LLM 124 causing the LLM 124 to generate SOAP notes in a single step. In some implementations, because one or more prompts may be longer than the appropriate context window of the LLM 124 and / or include complex instructions for the LLM 124, the single task can be broken down into multiple subtasks. In some implementations, task decomposition processing can be used to generate SOAP notes, where one or more prompts are provided to the LLM 124 to perform one or more subtasks of the SOAP note generation process (e.g., prompts causing the LLM 124 to extract facts from text transcription, and prompts causing the LLM 124 to generate SOAP notes from text transcription and facts). Furthermore, a mixture of LLMs selected from the LLM 124 can be used to generate SOAP notes, such that the LLM 124 for one specific task or subtask of the SOAP note generation process is different from the LLM 124 for another specific task or subtask of the SOAP note generation process. In this way, control can be exercised over the stages and / or tasks of SOAP generation processing and / or SOAP note generation processing. This can result in higher quality SOAP notes, reduce the need for manual review of SOAP notes (e.g., by medical professionals or scribes), and facilitate debugging and troubleshooting if the generated SOAP notes have a lower-than-expected quality level. In some implementations, when using task decomposition to generate SOAP notes, each subtask can be tailored by selectively providing different prompts to the LLM 124 to obtain results with an appropriate quality level from the LLM 124.

[0058] SOAP notes generated by SOAP note service 120 may be stored within platform 114 and / or in another location (such as one or more databases of database 122), where they may be accessed by platform 114, one or more other services of platform 114 (such as digital assistant service 118), and / or LLM 124. Additionally or alternatively, SOAP notes generated by SOAP note service 120 may be provided to one or more other services of platform 114, such as digital assistant service 118 and / or LLM 124. For example, SOAP note service 120 may generate SOAP notes and provide them to digital assistant service 118, where they can be used in a conversation in which digital assistant service 118 is participating. SOAP notes generated by SOAP note service may be stored in a database or storage medium in association with one or more interacting entities. For example, SOAP notes that record interactions between a healthcare provider and a patient may be associated with or stored in the database of database 122 within one or more electronic records of the healthcare provider and / or one or more electronic health records of the patient, where they can be accessed by the healthcare provider, the patient, another healthcare provider, etc.

[0059] Although not shown, platform 114 may include other capabilities and services, such as authentication services, management services, task management services, notification services, etc. The various capabilities and services of platform 114 may be implemented using one or more computing resources and / or servers of platform 114, and provided by platform 114 through a subscription. Additionally or alternatively, although Figure 1 The platform 114 service is shown to be a separate service, but one or more services within a service can be combined with other services and / or considered as sub-services of another service. For example, such as Figure 2 As shown, in environment 200, SOAP note service 120 can be SOAP note sub-service 220, which can be a sub-service or part of digital assistant service 118.

[0060] Figure 1 and Figure 2 The environments 100 and 200 depicted are merely exemplary and are not intended to unduly limit the scope of the claimed embodiments. Those skilled in the art will recognize many possible variations, alternatives, and modifications. For example, in some embodiments, environments 100 and 200 may use... Figure 1 and Figure 2 The services shown can be implemented with more or fewer services, two or more services can be combined, or they can have different service configurations or arrangements.

[0061] Automatic SOAP Note Generation

[0062] Figure 3 An example of a task flow 300 for automatically generating SOAP notes is described. For example... Figure 3 As shown, task flow 300 includes segmentation subtask 312, entity recognition subtask 318, fact extraction subtask 320, fact merging subtask 322, and SOAP note generation subtask 326. In some implementations, task flow 300 is executed by a service of a cloud service provider platform (such as SOAP note service 120 of platform 114).

[0063] Segmentation subtask 312 is configured to segment text transcription 310 into chunks 314. In some embodiments, text transcription 310 corresponds to an interaction between a first entity and a second entity. In some embodiments, the text transcription is stored in platform 114 and / or database 122. In some embodiments, the first entity is a healthcare provider, the second entity is a patient associated with the healthcare provider, and the text transcription is associated with one or more electronic records of the healthcare provider and / or with one or more electronic health records of the healthcare provider's patient and / or stored within these one or more electronic health records in the platform and / or database. In some embodiments, text transcription 310 can be generated by converting audio to text. The audio can be an audio recording of an interaction between the first entity and the second entity. For example, an audio recording of a conversation between a healthcare provider and a patient can be converted into a text transcription of the conversation. To convert audio to text transcription, one or more machine learning models, such as an ASR model, can be utilized. The text transcription can be stored in the platform and / or database, where it can be accessed by segmentation subtask 312 and segmented into chunks 314.

[0064] Segmentation subtask 312 may segment the text transcription 310 into blocks of equal size and / or blocks of variable size. For example, each block segmented at segmentation subtask 312 has the same size as each of the other blocks segmented at segmentation subtask 312. In another example, one or more blocks segmented at segmentation subtask 312 may be of a different size than the other blocks segmented at segmentation subtask 312. In some embodiments, text transcription 310 includes a first number of tokens, and each block segmented at segmentation subtask 312 includes a second number of tokens, less than the first number of tokens. In some embodiments, segmenting text transcription 310 into blocks 314 includes selecting a corresponding portion of text transcription 310 having a number of tokens corresponding to the second number of tokens, and determining whether the corresponding portion of text transcription 310 meets a predetermined criterion. In some embodiments, the corresponding portion includes 250 tokens, which is less than the first number of tokens. In some embodiments, the corresponding portion includes a turn sequence of dialogue between a first entity and a second entity, wherein the turn sequence includes a first turn, a last turn, and turns between the first and last turns. In some embodiments, determining whether a corresponding portion of the text transcription 310 meets a predetermined criterion includes determining whether the last turn of the turn sequence includes a question or an incomplete sentence. In some embodiments, in response to determining that the last turn of the turn sequence includes a question, a turn from another part of the dialogue that answers the question is added to the corresponding portion, such that the last turn of the turn sequence in the modified corresponding portion is not a question, and the modified corresponding portion is added to block 314. In some embodiments, in response to determining that the last turn of the turn sequence is an incomplete sentence, a turn from another part of the dialogue is added to the corresponding portion, such that the last turn of the turn sequence in the modified corresponding portion is a complete sentence, and the modified corresponding portion is added to block 314.

[0065] Entity recognition subtask 318 is configured to recognize one or more entities in each block of block 314. In some embodiments, entity recognition subtask 318 is configured to recognize one or more entities in each block of block 314 using a machine learning model and machine learning model hints for that machine learning model. In some embodiments, the machine learning model is an LLM. In some embodiments, the LLM may be configured to recognize medical, clinical, or healthcare entities in the text of each corresponding block of block 314. Examples of medical, clinical, or healthcare entities include, but are not limited to, patients, doctors, drugs, drug dosages, vital signs, test or examination results, or laboratory results. In some embodiments, for each corresponding block of block 314, results are obtained from the machine learning model by providing the machine learning model hints as input to the machine learning model using machine learning model hints. The results may include a list (e.g., a comma-separated list) or table of any entities contained in the corresponding block (e.g., one entity, two entities, etc.). In some embodiments, the results may be compared with the corresponding blocks to determine whether any entities contained in the results are also not contained in the corresponding blocks. For example, a check may be performed regarding whether a token corresponding to each entity contained in the results matches any token in the corresponding block. If an entity included in the results is not included in the corresponding block, the entity can be removed from the results to produce a filtered result. In some implementations, machine learning model hints may include queries and contextual information. The query may be a query for extracting entities from the corresponding block (e.g., the query may be "Please extract all medically relevant entities of no more than 3 words from the following conversation and present them in a bulleted list format."). Contextual information may include the corresponding block (e.g., contextual information may include the text of the corresponding block). In some implementations, the machine learning model hints used at entity recognition subtask 318 may be automatically engineered by LLM 124 and / or accessed from platform 114, database 122, or another source (such as the Internet).

[0066] In some implementations, the results, or filtered results, may be combined with results obtained from another machine learning model and / or service configured to extract entities from text. In some implementations, Named Entity Recognition (NER) service 316 may be accessed and used to identify one or more entities in each block of block 314. In some implementations, the NER service may include a machine learning model (e.g., an NER model) configured to identify medical, clinical, or healthcare entities in the text of each corresponding block of block 314. In some implementations, for each corresponding block of block 314, the corresponding block may be provided to the NER service 316 to obtain results from the NER service 316. The results may include a list (e.g., a comma-separated list) of any entities contained in the corresponding block. In some implementations, the results obtained from the NER service 316 may be combined with results obtained from the machine learning model or filtered results to produce a merged result. The merged result may include a list (e.g., a comma-separated list) or a table of entities contained in the corresponding block.

[0067] Fact extraction subtask 320 is configured to extract one or more facts from each block of block 314. In some implementations, fact extraction subtask 320 is configured to extract one or more facts from each block of block 314 using a machine learning model and machine learning model hints for that machine learning model. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model may be the same as or different from the machine learning model used at entity recognition subtask 318. In some implementations, for each corresponding block of block 314, results are obtained from the machine learning model by providing the machine learning model hints as input to the machine learning model using machine learning model hints. The results may include a list (e.g., a comma-separated list) or table of any facts contained in the corresponding block (e.g., one fact, two facts, etc.). In some implementations, the machine learning model hints may include queries and contextual information. The query may be a query for extracting facts from the corresponding block (e.g., the query may be "Please generate an abstract summary of the entities mentioned in the following conversation. The summary should be suitable for physician notes. Each line should indicate whether it was stated by the physician or the patient. If important numbers are mentioned in the conversation, please ensure they are included."). Contextual information may include the results generated at entity recognition subtask 318, filtered or merged results, and corresponding blocks (e.g., contextual information may include a list of entities extracted from the text of the corresponding block and the text of the corresponding block). The results generated for each corresponding block of block 314 may be combined into a fact set for block 314. For example, the fact set for block 314 may include all facts extracted for each corresponding block of block 314. In some implementations, the machine learning model hints used at fact extraction subtask 320 may be automatically engineered by LLM 124 and / or accessed from platform 114, database 122, or another source such as the Internet.

[0068] The fact merging subtask 322 is configured to generate a set of categorized facts 324 from the fact set. In some implementations, the fact merging subtask 322 is configured to generate the set of categorized facts 324 by categorizing each fact contained in the fact set and assigning category labels to each corresponding fact contained in the fact set based on the category assigned to the corresponding fact by the categorization. The category labels assigned to the corresponding facts contained in the fact set can be selected from a set of category labels based on the category assigned to the corresponding fact. Each category label contained in this set of category labels can correspond to or belong to a specific SOAP note segment of the SOAP note to be generated (i.e., the SOAP note generated by task flow 300). For example, as described above, the SOAP note to be generated can include subjective segments, objective segments, and evaluation and planning segments, and the category labels contained in this set of category labels can be associated with subjective segments, objective segments, and evaluation and planning segments. In this way, the facts contained in the fact set can be labeled as corresponding to, belonging to, or associated with a specific note segment of the SOAP note.

[0069] Fact merging subtask 322 is configured to classify each fact contained in the fact set using a machine learning model and machine learning model hints for that machine learning model. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model may be the same as or different from one or more machine learning models used in entity recognition subtask 318 and fact extraction subtask 320. In some implementations, the machine learning model hints are used to obtain results from the machine learning model by providing the machine learning model hints as input. The results may include a fact set and category labels assigned to the facts contained in the fact set. In some implementations, each fact in the fact set may be assigned one or more category labels from a set of category labels (e.g., a fact contained in the fact set may be assigned a category label corresponding to or belonging to or associated with a subjective section of a SOAP note and a category label corresponding to or belonging to or associated with an objective section of a SOAP note).

[0070] The machine learning model hints used to generate the set of categorized facts may include queries and contextual information. Queries may be queries used to categorize facts in the fact set and assign labels to the categorized facts (e.g., a query might be "Please categorize each fact in the following list of facts as corresponding to, belonging to, or associated with a SOAP note segment in the following SOAP note segment. Please also assign a category label to each fact based on its categorization. Each fact may have more than one category label. Please generate a subset of facts organized by category label."). Contextual information may include the fact set and the list of SOAP note segments for generating the SOAP notes. In some implementations, the machine learning model hints used at the fact merging subtask 322 may be automatically engineered by LLM 124 and / or accessed from platform 114, database 122, or another source such as the Internet.

[0071] In some implementations, before classifying facts in the fact set as corresponding to, belonging to, or relating to specific SOAP note sections of the SOAP note to be generated, the fact merging subtask 322 may be configured to generate a set of filtered facts by classifying each fact contained in the fact set as either a medical fact or not a medical fact. Examples of medical facts include, but are not limited to, facts relating to medical practice and / or the treatment of diseases and / or injuries. The fact merging subtask 322 is configured to use a machine learning model and machine learning model hints for that machine learning model to classify each fact contained in the fact set as either a medical fact or not a medical fact. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model may be the same as or different from one or more machine learning models used in other subtasks of task flow 300. In some implementations, the machine learning model hints are used to obtain results from the machine learning model by providing the machine learning model with machine learning model hints as input. The results may include a set of filtered facts, wherein each fact in the set of filtered facts is medically relevant. In some implementations, where the fact merging subtask 322 generates a set of filtered facts before generating the set of categorized facts, the contextual information suggested by the machine learning model used to generate the set of categorized facts may include the set of filtered facts (i.e., it may include a set of medically relevant facts). In this way, medically relevant facts in the fact set can be labeled as corresponding to, belonging to, or associated with specific note sections of a SOAP note.

[0072] The machine learning model hints used to generate the set of filtered facts may include queries and contextual information. The query may be a query for categorizing facts in the fact set and assigning labels to the categorized facts (e.g., the query may be "Please categorize and label each fact in the following list as medically relevant or non-medically relevant. Generate a set of filtered facts based on the categorization and labels."). Contextual information may include the fact set and examples of medically relevant facts. In some implementations, the machine learning hints used to generate the set of filtered facts may be included in and / or combined with the machine learning hints used to generate the set of categorized facts, such that providing machine learning model hints to the machine learning model causes the machine learning model to generate a result comprising a set of filtered and categorized facts. In some implementations, the machine learning hints used to generate the set of filtered facts may be provided to the machine learning model before and / or after providing machine learning model hints used to generate the set of categorized facts, such that non-medically relevant facts in the fact set may be discarded before generating SOAP notes.

[0073] Any facts within this group of categorized facts that have been assigned a subjective segment category label (i.e., those facts classified as corresponding to, belonging to, or associated with a subjective segment of a SOAP note) may be organized into and / or otherwise included in a first subset of the categorized facts. Any facts within this group of categorized facts that have been assigned an objective segment category label (i.e., those facts classified as corresponding to, belonging to, or associated with an objective segment of a SOAP note) may be organized into and / or otherwise included in a second subset of the categorized facts. Any facts within this group of categorized facts that include evaluation and planning segment category labels (i.e., those facts classified as corresponding to, belonging to, or associated with an evaluation and planning segment of a SOAP note) may be organized into and / or included in a third subset of the categorized facts.

[0074] SOAP note generation subtask 326 is configured to generate SOAP notes 330 from the group of categorized facts 324. In some embodiments, SOAP generation subtask 326 is configured to generate SOAP notes 330 by generating SOAP note segments of SOAP notes 330 and combining SOAP note segments into SOAP notes 330. In some embodiments, SOAP note generation subtask 326 may include subtasks for generating SOAP note segments of SOAP notes 330 and subtasks for combining SOAP note segments into SOAP notes 330. Figure 4 An example of a SOAP note generation task, including subtasks such as SOAP note generation subtask 326, is described. Figure 4The SOAP note generation subtask 326 shown includes segment generation subtask 1 326A, segment generation subtask 2 326B, segment generation subtask N 326C, etc., used to generate SOAP note segments to be included in SOAP note 330. Figure 4 The SOAP note generation subtask 326 shown also includes a segment combination subtask 326D for combining SOAP note segments generated by segment generation subtasks 1-N 326A-326C. Each segment generation subtask can be used to generate a specific SOAP note segment of SOAP note 330. For example, as described above, the SOAP note to be generated may include subjective segments (e.g., HPI components), objective segments (e.g., ROS components), and evaluation and planning segments, and segment generation subtask 1 326A can be used to generate subjective segments, segment generation subtask 2 326B can be used to generate objective segments, and segment generation subtask N 326C can be used to generate evaluation and planning segments.

[0075] Segment generation subtask 1 326A is configured to generate corresponding SOAP note segments of SOAP note 330, such as HPI components, using a machine learning model and machine learning model hints for that machine learning model. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model may be the same as or different from one or more machine learning models used in other subtasks of task flow 300. In some implementations, the machine learning model hints are provided as input to the machine learning model, and the results are obtained from the machine learning model using the machine learning model hints. The results may include corresponding SOAP note segments. The corresponding SOAP notes included in the results may be styled and formatted in a style and format that corresponds to and / or matches the style and format of SOAP note 330. The machine learning model hints used to generate the corresponding SOAP note segments may include queries and contextual information. The query may be a query for generating the corresponding SOAP note segments and generating the corresponding SOAP note segments in the style and format of SOAP note 330. (For example, a query for generating the HPI components of a SOAP note could be, “Given the following list of facts derived from a doctor-patient conversation, write the present medical history section of a SOAP note. Do not mention any information not present in the list of facts. Use paragraph formatting and continuous narration.”) Contextual information may include facts contained in a first subset of the categorized facts. For example, contextual information may include facts from that set of categorized facts that have been assigned a subjective section category label (i.e., those facts categorized as corresponding to, belonging to, or relating to the subjective section of the SOAP note). Contextual information may also include entity information 328, such as the patient’s medical or health information (e.g., the patient’s age, gender, weight, etc.). Medical or health information may be derived from the patient’s electronic health records, such as those described above, and stored in the platform and / or database. In some implementations, the machine learning model hints used at section generation subtask 1 326A may be automatically engineered by LLM 124 and / or accessed from platform 114, database 122, or another source (such as the Internet).

[0076] Segment generation subtask 2 326B is configured to generate corresponding SOAP note segments of SOAP note 330, such as ROS components, using a machine learning model and machine learning model hints for that machine learning model. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model may be the same as or different from one or more machine learning models used at other subtasks of task flow 300. In some implementations, the machine learning model hints are used to obtain results from the machine learning model by providing them as input. The results may contain corresponding SOAP note segments. The corresponding SOAP notes contained in the results may be styled and formatted in a style and format that corresponds to and / or matches the style and format of SOAP note 330. The machine learning model hints used to generate the corresponding SOAP note segments may include queries and contextual information. The query may be a query for generating the corresponding SOAP note segments and generating the corresponding SOAP note segments in the style and format of SOAP note 330. (For example, a query for generating a ROS component of a SOAP note could be, "Are you a transcriber? Does the fact contain medical symptoms? If so, extract all patients’ medical symptom-related information from the facts mentioned below that are suitable for the System Review section of the SOAP note. Follow these instructions when extracting symptoms: 1. Extract only symptoms spoken by the patient. 2. Do not add unnecessary information to the extracted symptoms. 3. Do not include psychological symptoms. 4. If no symptom exists in the note, provide the output as 'None'. 5. Extract only medical symptoms present in the note. 6. Provide the output in a bulleted list format. 7. Emphasize capturing negative symptoms, meaning symptoms that have been denied but will still be discussed in the fact. 8. Do not capture diagnoses or diseases.") Contextual information can include facts contained in a second subset of the categorized facts. For example, contextual information can include facts in that set of categorized facts that have been assigned an objective section category label (i.e., those facts that are categorized as corresponding to, belonging to, or associated with an objective section of the SOAP note). Contextual information can also include entity information 328, such as the patient’s medical or healthcare information (e.g., the patient’s age, gender, weight, etc.). Medical or health information can be derived from a patient’s electronic health records, such as those described above, and stored on the platform and / or in a database. In some implementations, the machine learning model hints used at segment generation subtask 2326B can be automatically engineered by LLM 124 and / or accessed from platform 114, database 122, or another source (such as the Internet).

[0077] The segment generation subtask N 326C is configured to generate corresponding SOAP note segments of SOAP note 330, such as evaluation and planning components, using a machine learning model and machine learning model hints for that machine learning model. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model may be the same as or different from one or more machine learning models used at other subtasks of task flow 300. In some implementations, the machine learning model hints are used to obtain results from the machine learning model by providing the machine learning model as input. The results may include corresponding SOAP note segments. The corresponding SOAP notes included in the results may be styled and formatted in a style and format that corresponds to and / or matches the style and format of SOAP note 330. The machine learning model hints used to generate the corresponding SOAP note segments may include queries and contextual information. The query may be a query for generating the corresponding SOAP note segments and generating the corresponding SOAP note segments in the style and format of SOAP note 330. (For example, a query to generate the evaluation and planning components of a SOAP note could be: (i) “Generate the evaluation section of a SOAP note for the following list of facts derived from a doctor-patient conversation. This section should only include the doctor’s diagnosis of the patient’s problem. Do not include the doctor’s advice on how to solve the problem.”; and (ii) “Generate the planning section of a SOAP note for the following list of facts derived from a doctor-patient conversation.”). Contextual information may include facts contained in a third subset of categorized facts. For example, contextual information may include facts from that set of categorized facts that have been assigned evaluation and planning section category labels (i.e., those facts categorized as corresponding to, belonging to, or associated with the evaluation and planning section of the SOAP note). Contextual information may also include entity information 328, such as the patient’s medical or health information (e.g., the patient’s age, gender, weight, etc.). Medical or health information may be derived from the patient’s electronic health records, such as those described above, and stored in the platform and / or database. In some implementations, the machine learning model hints used at the segment generation subtask N 326C can be automatically engineered by LLM 124 and / or accessed from platform 114, database 122, or another source such as the Internet.

[0078] Segment Combining Subtask 326D is configured to combine corresponding SOAP note segments generated by Segment Generation Subtask 1 326A, Segment Generation Subtask 2 326B, and Segment Generation Subtask N 326C into SOAP Note 330. Segment Combining Subtask 326D can generate SOAP Note 330 by accessing a SOAP Note template (e.g., a template stored on the platform and / or in a database and / or accessed from another source, such as the Internet) and inserting the corresponding SOAP note segments into the corresponding template segments of the SOAP Note template. For example, subjective segments of the SOAP Note will be inserted into corresponding segments in the SOAP Note template, objective segments of the SOAP Note will be inserted into corresponding segments in the SOAP Note template, and evaluation and planning segments of the SOAP Note will be inserted into corresponding segments in the SOAP Note template. In some implementations, Segment Combining Subtask 326D can generate SOAP Note 330 by generating a data structure and storing the corresponding SOAP note segments in the data structure.

[0079] SOAP note generation subtask 326 is configured to store SOAP note 330 and a data structure including the SOAP note in platform 114 and / or database 122, and / or send SOAP note 330 and / or the data structure including the SOAP note to a remote location such as client device 110. In some embodiments, SOAP note 330 and / or the data structure including the SOAP note are stored in a database associated with at least one of the first and second entities. In some embodiments, SOAP note 330 and / or the data structure including the SOAP note are stored in the electronic health record of the interacting patient. In this way, automatically generated SOAP notes recording the contact between the healthcare provider and the patient can be stored in the patient's electronic health record, allowing one or more other entities (such as one or more other healthcare providers) to access the SOAP notes at a later time.

[0080] As described above, the machine learning models used for the subtasks of task flow 300 can be the same machine learning model. In some implementations, one or more machine learning models used for one or more subtasks within task flow 300 can be different from one or more machine learning models used in one or more other subtasks of task flow 300. In this way, a machine learning model that performs well for one specific task can be used for that specific task, and a machine learning model that performs well for another specific task can be used for that specific task. In some implementations, a task can be performed to identify the machine learning models to be used at the subtasks of task flow 300 before generating SOAP notes 300. In some implementations, once a machine learning model is identified for a specific subtask, machine learning model hints for that machine learning model can be automatically engineered and / or accessed from a cloud service provider platform, database, another storage medium, and / or another source (such as the Internet), and can be used to obtain results from the machine learning model. In this way, task flow 300 can be decomposed into subtasks, which can be executed using machine learning models that perform well for the corresponding subtasks. In some implementations, machine learning models can be evaluated based on performance (such as hallucination frequency, output structure performance, repetition performance, etc.) and can be selected in part based on their performance. While the techniques described herein are about machine learning models that accept machine learning model cues as input, this is not intended to be limiting, and machine learning models that accept other forms of input can be used at subtasks.

[0081] Explanatory methods

[0082] Figure 5 An example of a process for automatically generating SOAP notes using task decomposition is described. Figure 5 The processing described herein can be implemented using software (e.g., code, instructions, programs), hardware, or a combination thereof, executed by one or more processing units (e.g., processors, cores) of a corresponding system. The software can be stored on one or more non-transitory storage media (e.g., memory devices). Figure 5 The processes shown and described below are intended to be illustrative and not restrictive. Although Figure 5 The steps are described in a specific order or sequence, but this is not intended to be limiting. In some alternative embodiments, the steps may be performed in a different order, or some steps may be performed in parallel. In some embodiments, such as in Figure 1 and Figure 2 In the embodiments depicted, Figure 5 The processing shown can be performed by environment 100 and / or environment 200.

[0083] At box 502, the text transcription is accessed. In some embodiments, the text transcription corresponds to an interaction between a first entity and a second entity. In some embodiments, the first entity is a healthcare provider, and the second entity is a patient associated with that healthcare provider. In some embodiments, the text transcription is stored in a cloud service provider platform such as platform 114 described above and / or a database such as database 122 described above. In some embodiments, the text transcription is associated with one or more electronic records of the healthcare provider and / or with one or more electronic health records of the healthcare provider's patient and / or stored within such electronic health records on the platform and / or in the database. In some embodiments, the text transcription is generated by converting audio to text. The audio can be an audio recording of an interaction between the first entity and the second entity. For example, the audio recording can be an audio recording of a conversation between a healthcare provider and a patient, which can be converted into a text transcription of the conversation. To convert audio to text transcription, one or more machine learning models, such as an ASR model, can be utilized. The text transcription can be stored within the platform and / or database, where it can be accessed.

[0084] At box 504, the text transcription is segmented into multiple parts. In some embodiments, the text transcription includes a first number of tokens, and each of the multiple parts includes a second number of tokens, fewer than the first number. In some embodiments, segmenting the text transcription into multiple parts includes selecting a corresponding part of the text transcription having a number of tokens corresponding to the second number, and determining whether the corresponding part of the text transcription meets a predetermined criterion. In some embodiments, the text transcription is segmented into blocks of equal size and / or blocks of variable size. For example, each segmented block may have the same size as every other segmented block. In another example, one or more segmented blocks may be of a different size than other segmented blocks. In some embodiments, segmenting the text transcription into blocks includes selecting a corresponding part of the text transcription having a number of tokens corresponding to the second number, and determining whether the corresponding part of the text transcription meets a predetermined criterion. In some embodiments, the corresponding part includes 250 tokens, which is fewer than the first number of tokens. In some embodiments, the corresponding portion includes a turn sequence of dialogue between a first entity and a second entity, wherein the turn sequence includes a first turn, a last turn, and turns between the first and last turns. In some embodiments, determining whether a corresponding portion of the transcribed text meets predetermined criteria includes determining whether the last turn of the turn sequence includes a question or an incomplete sentence. In some embodiments, in response to determining that the last turn of the turn sequence includes a question, a turn from a dialogue from another portion of the transcribed text that answers the question is added to the corresponding portion, such that the last turn of the turn sequence in the modified corresponding portion is not a question, and the modified corresponding portion is added to a block. In some embodiments, in response to determining that the last turn of the turn sequence is an incomplete sentence, a turn from a dialogue from another portion of the plurality of portions is added to the corresponding portion, such that the last turn of the turn sequence in the modified corresponding portion is a complete sentence, and the modified corresponding portion is added to a block.

[0085] At box 506, for each of the multiple sections, one or more entities contained in the corresponding section are identified. In some implementations, a machine learning model and machine learning model hints for that machine learning model are used to identify one or more entities in each block of the block. In some implementations, the machine learning model is an LLM. In some implementations, the LLM may be configured to identify medical, clinical, or healthcare entities in the text of each corresponding block of the block. Examples of medical, clinical, or healthcare entities include, but are not limited to, patients, doctors, drugs, drug dosages, vital signs, test or examination results, or laboratory results. In some implementations, for each corresponding block of the block, a result is obtained from the machine learning model by providing the machine learning model hints as input. The result may include a list (e.g., a comma-separated list) or table of any entities contained in the corresponding block (e.g., one entity, two entities, etc.). In some implementations, the result may be compared with the corresponding block to determine whether any entity contained in the result is also not contained in the corresponding block. For example, a check may be performed regarding whether a token corresponding to each entity contained in the result matches any token in the corresponding block. If an entity contained in the result is also not contained in the corresponding block, that entity may be removed from the result to produce a filtered result. In some implementations, machine learning model hints may include queries and contextual information. The query may be a query for extracting entities from a corresponding block (e.g., the query could be "Please extract all medically relevant entities of no more than 3 words from the conversation below and present them in a bulleted list format."). Contextual information may include the corresponding block (e.g., contextual information may include the text of the corresponding block). In some implementations, the machine learning model hints used may be automatically engineered by one or more LLMs (such as LLM 124) and / or accessed from a platform, database, or another source (such as the Internet).

[0086] In some implementations, the results, or filtered results, may be combined with results obtained from another machine learning model and / or service configured to extract entities from text. In some implementations, a NER service may be accessed and used to identify one or more entities in each block of a block. In some implementations, the NER service may include a machine learning model (e.g., an NER model) configured to identify medical, clinical, or healthcare entities in the text of each corresponding block of a block. In some implementations, for each corresponding block of a block, the corresponding block may be provided to the NER service to obtain results from the NER service. The results may include a list (e.g., a comma-separated list) of any entities contained in the corresponding block. In some implementations, the results obtained from the NER service may be combined with results obtained from the machine learning model or filtered results to produce a merged result. The merged result may include a list (e.g., a comma-separated list) or a table of entities contained in the corresponding block.

[0087] At box 508, for each corresponding part of the plurality of sections, one or more facts are extracted from the corresponding part. In some implementations, a machine learning model and a machine learning model hint for that machine learning model are used to extract one or more facts from each block in the block. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model is the same as or different from the machine learning model used to identify one or more entities. In some implementations, for each corresponding block in the block, the machine learning model hint is used to obtain results from the machine learning model by providing the machine learning model hint as input. The results may include a list (e.g., a comma-separated list) or table of any facts contained in the corresponding block (e.g., one fact, two facts, etc.). In some implementations, the machine learning model hint may include queries and contextual information. The query may be a query for extracting facts from the corresponding block (e.g., the query may be "Please generate an abstract summary of the following conversation covering the mentioned entities. The summary should be suitable for physician notes. Each line should indicate whether it was stated by the physician or the patient. If important numbers were mentioned in the conversation, make sure to include them."). Contextual information may include the results, filtered results, or merged results, and corresponding blocks (e.g., contextual information may include a list of entities extracted from the text of the corresponding block and the text of the corresponding block). In some implementations, machine learning model hints for extracting one or more facts may be automatically engineered by one or more LLMs and / or accessed from a platform, database, or another source such as the Internet.

[0088] At box 510, one or more facts extracted for each corresponding part of the plurality of parts are added to a fact set. In some implementations, the one or more facts extracted for each corresponding part may be combined into a block fact set. For example, the block fact set may include all facts extracted for each corresponding block of the block.

[0089] At box 512, a set of categorized facts is generated. In some implementations, this set of categorized facts is generated from a fact set. In some implementations, the set of categorized facts is generated by categorizing each fact contained in the fact set and assigning category labels to each corresponding fact contained in the fact set based on the category assigned to the corresponding fact by the categorization. The category labels assigned to the corresponding facts contained in the fact set can be selected from a set of category labels based on the category assigned to the corresponding fact. Each category label contained in this set of category labels can correspond to or belong to a specific SOAP note segment of the SOAP note to be generated. For example, as described above, the SOAP note to be generated can include subjective segments, objective segments, and evaluation and planning segments, and the category labels contained in this set of category labels can be associated with subjective segments, objective segments, and evaluation and planning segments. In this way, facts contained in the fact set can be labeled as corresponding to, belonging to, or associated with specific note segments of the SOAP note.

[0090] Each fact contained in a fact set can be categorized using a machine learning model and machine learning model hints for that model. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model may be the same as or different from one or more machine learning models used to identify one or more entities and / or extract one or more facts. In some implementations, the machine learning model hints are used to obtain results from the machine learning model by providing them as input. The results may include a fact set and category labels assigned to the facts contained in the fact set. In some implementations, each fact in the fact set may be assigned one or more category labels from a set of category labels (e.g., a fact in the fact set may be assigned a category label corresponding to or belonging to or associated with a subjective section of a SOAP note and a category label corresponding to or belonging to or associated with an objective section of a SOAP note).

[0091] The machine learning model hints used to generate the set of categorized facts may include queries and contextual information. Queries may be queries used to categorize facts in the fact set and assign labels to the categorized facts (e.g., a query might be "Please categorize each fact in the following list of facts as corresponding to, belonging to, or associated with a SOAP note segment in the following SOAP note segment. Please also assign a category label to each fact based on its categorization. Each fact may have more than one category label. Please generate a subset of facts organized by category label."). Contextual information may include the fact set and the list of SOAP note segments for generating the SOAP notes. In some implementations, the machine learning model hints(s) used to generate the set of categorized facts may be automatically engineered by one or more LLMs and / or accessed from a platform, database, or another source such as the Internet.

[0092] In some implementations, a set of filtered facts can be generated by classifying each fact in the fact set as either a medical fact or not a medical fact before classifying facts in the fact set as corresponding to, belonging to, or relating to a specific SOAP note section to be generated. Examples of medical facts include, but are not limited to, facts about medical practices and / or the treatment of diseases and / or injuries. Each fact in the fact set can be classified as either a medical fact or not a medical fact using a machine learning model and machine learning model hints for that machine learning model. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model is the same as or different from one or more machine learning models used in the process 500. In some implementations, the machine learning model hints are used to obtain results from the machine learning model by providing them as input to the machine learning model. The results may include a set of filtered facts, wherein each fact in the set of filtered facts is medically relevant. In some implementations, a set of filtered facts may be generated before generating a set of classified facts, and the contextual information of the machine learning model hints used to generate the set of classified facts may include the set of filtered facts (i.e., may include a set of medically relevant facts). In this way, medically relevant facts in the fact set can be labeled as corresponding to, belonging to, or associated with specific note sections of SOAP notes.

[0093] Machine learning model hints for generating a set of filtered facts may include queries and contextual information. The query may be a query for categorizing facts in a fact set and assigning labels to the categorized facts (e.g., the query may be "Please categorize and label each fact in the following list as medically relevant or non-medically relevant. Generate a set of filtered facts based on the categorization and labels."). Contextual information may include the fact set and examples of medically relevant facts. In some implementations, machine learning hints for generating the set of filtered facts may be included in and / or combined with machine learning hints for generating the set of categorized facts, such that providing machine learning model hints to the machine learning model causes the machine learning model to generate a result comprising a set of filtered and categorized facts. In some implementations, machine learning hints for generating the set of filtered facts may be provided to the machine learning model before and / or after providing machine learning hints for generating the set of categorized facts, such that non-medically relevant facts in the fact set may be discarded before generating SOAP notes.

[0094] Any facts within this group of categorized facts that have been assigned a subjective segment category label (i.e., those facts classified as corresponding to, belonging to, or associated with a subjective segment of a SOAP note) may be organized into and / or otherwise included in a first subset of the categorized facts. Any facts within this group of categorized facts that have been assigned an objective segment category label (i.e., those facts classified as corresponding to, belonging to, or associated with an objective segment of a SOAP note) may be organized into and / or otherwise included in a second subset of the categorized facts. Any facts within this group of categorized facts that include evaluation and planning segment category labels (i.e., those facts classified as corresponding to, belonging to, or associated with an evaluation and planning segment of a SOAP note) may be organized into and / or included in a third subset of the categorized facts.

[0095] At box 514, a set of note segments is generated. In some implementations, this set of note segments is generated at least partially based on a set of facts. In some implementations, each note segment in this set of note segments corresponds to a segment of a SOAP note. For example, a note segment in this set of note segments may correspond to a subjective segment of a SOAP note (e.g., an HPI component), a note segment in this set of note segments may correspond to an objective segment of a SOAP note (e.g., a ROS component), and a note segment in this set of note segments may correspond to an evaluation and planning segment of a SOAP note. In some implementations, each note segment in this set of note segments may be generated using a machine learning model and machine learning model hints for that machine learning model. In some implementations, the machine learning model is an LLM. In some implementations, the machine learning model is the same as or different from one or more machine learning models used in processing 500 and / or used to generate other note segments in this set of note segments. In some implementations, the machine learning model hints are provided as input to the machine learning model, and the results are obtained from the machine learning model using the machine learning model hints. The results may include the corresponding SOAP note segments. The corresponding SOAP notes included in the results can be styled and formatted in a style and format that corresponds to and / or matches that of SOAP notes. Machine learning model hints used to generate the corresponding SOAP note sections can include queries and contextual information. Queries can be queries used to generate the corresponding SOAP note sections and to generate them in the style and format of SOAP notes. For example, a query to generate the HPI components of a SOAP note could be: “Given the following list of facts derived from a doctor-patient conversation, write the present medical history section of a SOAP note. Do not mention any information not present in the fact list. Use paragraph formatting and continuous narration.” In another example, the query for generating the ROS component of a SOAP note could be: “You are a scribe. Does the fact contain medical symptoms? If so, extract all patient medical symptom-related information from the facts mentioned below that are suitable for the System Review section of the SOAP note. When extracting symptoms, follow these instructions: 1. Only extract symptoms described by the patient. 2. Do not add unnecessary information to the extracted symptoms. 3. Do not include psychological symptoms. 4. If no symptoms are present in the note, provide the output as 'None'. 5. Only extract medical symptoms present in the note. 6. Provide the output in a bulleted list format. 7. Emphasize capturing negative symptoms, meaning symptoms that have been denied but will still be discussed in the fact. 8. Do not capture diagnoses or diseases.” In yet another example, the query for generating the evaluation and planning components of a SOAP note could be: (i) “Generate the evaluation section of the SOAP note for the following list of facts derived from doctor-patient conversations. This section should only include the doctor's diagnosis of the patient's problem.”Do not include the doctor's advice on how to solve the problem. ; and (ii) "Generate the planning section of the SOAP note for the following list of facts derived from the doctor-patient conversation." Contextual information may include facts contained in a first subset of categorized facts, a second subset of categorized facts, and / or a third subset of categorized facts. For example, for a subjective section, contextual information may include facts from that set of categorized facts that have been assigned a subjective section category label (i.e., those facts categorized as corresponding to, belonging to, or associated with a subjective section of the SOAP note). In another example, for an objective section, contextual information may include facts from that set of categorized facts that have been assigned an objective section category label (i.e., those facts categorized as corresponding to, belonging to, or associated with an objective section of the SOAP note). In yet another example, for evaluation and planning sections, contextual information may... This includes facts that have been assigned evaluation and planning segment category labels within the categorized set of facts (i.e., those facts classified as corresponding to, belonging to, or associated with the evaluation and planning segments of the SOAP notes). Contextual information may also include entity information, such as the patient's medical or health information (e.g., the patient's age, gender, weight, etc.). Medical or health information can be derived from the patient's electronic health records, such as those described above, and stored on the platform and / or in a database. In some implementations, machine learning model prompts for generating the corresponding note segments of the set of note segments can be automatically engineered by one or more LLMs and / or accessed from the platform, database, or another source (such as the Internet).

[0096] At box 516, a SOAP note is generated. In some implementations, a SOAP note is generated by combining note segments from the group of note segments. In some implementations, a SOAP note can be generated by accessing a SOAP note template (e.g., a template stored on a platform and / or in a database and / or accessed from another source, such as the Internet) and inserting the corresponding SOAP note segments into the corresponding template segments of the SOAP note template. For example, the subjective segments of a SOAP note can be inserted into the corresponding segments of the SOAP note template, the objective segments of a SOAP note will be inserted into the corresponding segments of the SOAP note template, and the evaluation and planning segments of a SOAP note will be inserted into the corresponding segments of the SOAP note template. In some implementations, a SOAP note can be generated by generating a data structure and storing the corresponding SOAP note segments in the data structure.

[0097] At box 518, SOAP notes are stored. In some embodiments, the SOAP notes are stored in a database associated with at least one of the first and second entities. In some embodiments, storing the SOAP notes in the database includes storing the SOAP notes in an electronic health record associated with the patient. In some embodiments, the SOAP notes and / or the data structure including the SOAP notes may be stored in a platform and / or a database, and / or the SOAP notes and / or the data structure including the SOAP notes may be sent to a remote location such as one or more client devices (such as client device 110). In some embodiments, the SOAP notes and / or the data structure including the SOAP notes are stored in a database associated with at least one of the first and second entities. In this way, automatically generated SOAP notes recording the contact between the healthcare provider and the patient may be stored in the patient's electronic health record, allowing one or more other entities (such as one or more other healthcare providers) to access the SOAP notes at a later time.

[0098] Examples of cloud infrastructure architecture

[0099] The term cloud service is generally used to refer to services provided by a cloud service provider (CSP) to users (e.g., cloud service customers) on demand (e.g., via a subscription model) using systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that make up the CSP's infrastructure are separate from the user's own on-premises servers and systems. Therefore, users can utilize cloud services provided by the CSP without having to purchase separate hardware and software resources for these services. Cloud services are designed to provide subscribers with easy, scalable access to applications and computing resources without requiring users to invest in the infrastructure used to provide the services.

[0100] Several cloud service providers offer various types of cloud services. As discussed in this article, there are various types or models of cloud services, including IaaS, Software as a Service (SaaS), Platform as a Service (PaaS), and more. Users can subscribe to one or more cloud services provided by a CSP. Users can be any entity, such as individuals, organizations, enterprises, etc. When a user subscribes to or registers for a service provided by a CSP, a lease or account is created for that user. The user can then access one or more subscribed cloud resources associated with that account.

[0101] As mentioned above, IaaS is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also provision various services to accompany these infrastructure components (example services include billing software, monitoring software, logging software, load balancing software, and clustering software, etc.). Therefore, since these services may be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0102] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the Internet and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software into that VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, resolving application issues, monitoring performance, and managing disaster recovery.

[0103] In most cases, cloud computing models will require the involvement of cloud providers. Cloud providers can, but are not necessarily, third-party providers specializing in (e.g., provisioning, renting, selling) IaaS services. Entities may also choose to deploy private clouds, thus becoming their own infrastructure service providers.

[0104] In some examples, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the processing of server preparation (e.g., installation libraries, daemons, etc.). This is typically managed by the cloud provider, below the hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for processing (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, etc., which can be started on demand).

[0105] In some examples, IaaS provisioning can refer to acquiring computers or virtual hosts for use, or even installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0106] In some cases, IaaS provisioning presents two distinct challenges. First, there's the initial challenge of provisioning the initial set of infrastructure before anything is operational. Second, once everything is provisioned, there's the challenge of evolving the existing infrastructure (e.g., adding new services, changing services, removing services, etc.). In some cases, both challenges can be addressed by enabling configuration that declaratively defines the infrastructure. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more profiles. Therefore, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they work together) can be described declaratively. In some cases, once the topology is defined, workflows for creating and / or managing the different components described in the profiles can be generated.

[0107] In some examples, the infrastructure can have many interconnected elements. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, one or more inbound / outbound traffic group rules may also be provided to define how inbound / outbound traffic to the network and one or more virtual machines (VMs). Other infrastructure elements, such as load balancers, databases, etc., may also be provided. The infrastructure can evolve incrementally as more and / or more infrastructure elements are expected and added.

[0108] In some cases, continuous deployment techniques can be used to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that they expect to deploy to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes spanning the entire world). However, in some examples, the infrastructure on which the code will be deployed must first be set up. In some cases, provisioning can be done manually, resources can be provisioned using provisioning tools, and / or once the infrastructure is provisioned, the code can be deployed using deployment tools.

[0109] Figure 6This is a block diagram 600 illustrating an example pattern of an IaaS architecture according to at least one embodiment. Service operator 602 may be communicatively coupled to a secure host lease 604, which may include a virtual cloud network (VCN) 606 and a secure host subnet 608. In some examples, service operator 602 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhone®, cellular phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display), running software (such as Microsoft Windows Mobile®) and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 6, Palm OS, etc.), and supporting the Internet, email, short message service (SMS), Blackberry®, or other communication protocols. Alternatively, client computing devices may be general-purpose personal computers, including, for example, personal computers and / or laptops running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. The client computing device can be a workstation computer running a variety of commercially available UNIX® or UNIX-like operating systems, including but not limited to any of the various GNU / Linux operating systems (such as, for example, Google Chrome OS). Alternatively or additionally, the client computing device can be any other electronic device, such as a thin client computer, an internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without Kinect® gesture input), and / or a personal messaging device capable of communicating over a network that can access VCN 606 and / or the internet.

[0110] VCN 606 may include a local peering gateway (LPG) 610, which may be communicatively coupled to a secure shell (SSH) VCN 612 via an LPG 610 included in an SSH VCN 612. SSH VCN 612 may include an SSH subnet 614, and SSH VCN 612 may be communicatively coupled to a control plane VCN 616 via an LPG 610 included in a control plane VCN 616. Furthermore, SSH VCN 612 may be communicatively coupled to a data plane VCN 618 via an LPG 610. Control plane VCN 616 and data plane VCN 618 may be contained within a service lease 619 that may be owned and / or operated by an IaaS provider.

[0111] The control plane VCN 616 may include a control plane demilitarized zone (DMZ) layer 620 that acts as a peripheral network (e.g., a portion of a corporate network between a corporate intranet and an external network). DMZ-based servers can assume limited liability and help control vulnerabilities. Furthermore, the control plane DMZ layer 620 may include one or more load balancer (LB) subnets 622, a control plane application layer 624 that may include one or more application subnets 626, and a control plane data layer 628 that may include one or more database (DB) subnets 630 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets 622 contained in the control plane DMZ layer 620 may be communicatively coupled to one or more application subnets 626 contained in the control plane application layer 624 and an Internet gateway 634 that may be contained in the control plane VCN 616. The application subnets 626 may be communicatively coupled to one or more DB subnets 630 contained in the control plane data layer 628, as well as a service gateway 636 and a Network Address Translation (NAT) gateway 638. The control plane VCN 616 may include the service gateway 636 and the NAT gateway 638.

[0112] The control plane VCN 616 may include a data plane mirror application layer 640, which may include one or more application subnets 626. The one or more application subnets 626 included in the data plane mirror application layer 640 may include a virtual network interface controller (VNIC) 642 capable of executing a compute instance 644. The compute instance 644 may communicatively couple the one or more application subnets 626 of the data plane mirror application layer 640 to the one or more application subnets 626 that may be included in the data plane application layer 646.

[0113] Data plane VCN 618 may include data plane application layer 646, data plane DMZ layer 648, and data plane data layer 660. Data plane DMZ layer 648 may include one or more LB subnets 622 communicatively coupled to one or more application subnets 626 of data plane application layer 646 and Internet gateway 634 of data plane VCN 618. One or more application subnets 626 may be communicatively coupled to service gateway 636 and NAT gateway 638 of data plane VCN 618. Data plane data layer 660 may also include one or more DB subnets 630 communicatively coupled to one or more application subnets 626 of data plane application layer 646.

[0114] The Internet gateway 634 of the control plane VCN 616 and data plane VCN 618 can be communicatively coupled to the metadata management service 662, which in turn can be communicatively coupled to the public Internet 664. The public Internet 664 can be communicatively coupled to the NAT gateway 638 of the control plane VCN 616 and data plane VCN 618. The service gateway 636 of the control plane VCN 616 and data plane VCN 618 can be communicatively coupled to the cloud service 667.

[0115] In some examples, the service gateway 636 of the control plane VCN 616 or data plane VCN 618 can make application programming interface (API) calls to the cloud service 667 without traversing the public internet 664. API calls from the service gateway 636 to the cloud service 667 can be unidirectional: the service gateway 636 can make API calls to the cloud service 667, and the cloud service 667 can send requested data to the service gateway 636. However, the cloud service 667 may not initiate API calls to the service gateway 636.

[0116] In some examples, secure host lease 604 can be directly connected to service lease 619, which would otherwise be isolated. Secure host subnet 608 can communicate with SSH subnet 614 via LPG 610, which enables bidirectional communication between otherwise isolated systems. Connecting secure host subnet 608 to SSH subnet 614 allows secure host subnet 608 to access other entities within service lease 619.

[0117] Control plane VCN 616 allows users of service lease 619 to configure or otherwise provision desired resources. Desired resources provisioned in control plane VCN 616 can be deployed or otherwise used in data plane VCN 618. In some examples, control plane VCN 616 can be isolated from data plane VCN 618, and the data plane mirror application layer 640 of control plane VCN 616 can communicate with the data plane application layer 646 of data plane VCN 618 via VNIC 642, which can be included in both the data plane mirror application layer 640 and the data plane application layer 646.

[0118] In some examples, users or clients of the system can make requests, such as create, read, update, or delete (CRUD) operations, via the public internet 664, which can transmit requests to the metadata management service 662. The metadata management service 662 can transmit the request to the control plane VCN 616 via internet gateway 634. The request can be received by one or more LB subnets 622 contained in the control plane DMZ layer 620. The LB subnets 622 can determine that the request is valid, and in response to this determination, they can transmit the request to one or more application subnets 626 contained in the control plane application layer 624. If the request is validated and requires a call to the public internet 664, the call to the public internet 664 can be transmitted to a NAT gateway 638 that can make calls to the public internet 664. The request may expect the stored metadata to be stored in one or more DB subnets 630.

[0119] In some examples, the data plane mirroring application layer 640 can facilitate direct communication between the control plane VCN 616 and the data plane VCN 618. For example, it may be desirable to apply configuration changes, updates, or other appropriate modifications to resources contained in the data plane VCN 618. Through VNIC 642, the control plane VCN 616 can communicate directly with the resources contained in the data plane VCN 618, and thus can perform configuration changes, updates, or other appropriate modifications.

[0120] In some embodiments, the control plane VCN 616 and data plane VCN 618 may be included in a service lease 619. In this case, the system's users or customers may not own or operate the control plane VCN 616 or data plane VCN 618. Alternatively, the IaaS provider may own or operate both the control plane VCN 616 and data plane VCN 618, and both planes may be included in the service lease 619. This embodiment can enable the isolation of networks that might prevent users or customers from interacting with resources from other users or customers. Furthermore, this embodiment can allow users or customers of the system to privately store databases without relying on the public Internet 664, which may not have the desired level of threat prevention for storage.

[0121] In other embodiments, one or more LB subnets 622 included in the control plane VCN 616 may be configured to receive signals from the service gateway 636. In this embodiment, the control plane VCN 616 and the data plane VCN 618 may be configured to be invoked by the IaaS provider's customers without invoking the public internet 664. The IaaS provider's customers may expect this embodiment because the database(s) used by the customer can be controlled by the IaaS provider and can be stored on a service lease 619, which may be isolated from the public internet 664.

[0122] Figure 7 This is a block diagram 700 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 702 (e.g., Figure 6 The service provider (602) can communicatively couple to the secure host lease (704) (e.g., Figure 6 Secure hosting lease 604), the secure hosting lease 704 may include a Virtual Cloud Network (VCN) 706 (e.g., Figure 6 VCN606) and Secure Host Subnet 708 (e.g., Figure 6 The secure host subnet 608). VCN 606 may include a local peering gateway (LPG) 710 (e.g., Figure 6 The LPG 610), which can be communicatively coupled to the Secure Shell (SSH) VCN 712 (e.g., via the LPG 710 contained in the SSH VCN 712) Figure 6 SSH VCN 612). SSH VCN 712 can include SSH subnet 714 (e.g., Figure 6 SSH subnet 614), and SSH VCN 712 can be communicatively coupled to control plane VCN 716 via LPG 710 included in control plane VCN 716 (e.g., Figure 6 Control plane VCN 616). Control plane VCN 716 may be included in service lease 719 (e.g., Figure 6 In the service lease 619), and the data plane VCN 718 (e.g., Figure 6 The data plane VCN 618 may be included in a customer lease 721 that may be owned or operated by a user or customer of the system.

[0123] The control plane VCN 716 may include one or more LB subnets 722 (e.g., Figure 6 The control plane DMZ layer 720 of (one or more) LB subnets 622) (e.g., Figure 6The control plane DMZ layer 620) may contain one or more application subnets 726 (e.g., Figure 6 The control plane application layer 724 of (one or more) application subnets 626 (e.g., Figure 6 The control plane application layer 624) may contain one or more database (DB) subnets 730 (e.g., similar to...). Figure 6 The control plane data layer 728 of (one or more) DB subnets 630 (e.g., Figure 6 The control plane data layer 628). One or more LB subnets 722 contained in the control plane DMZ layer 720 can be communicatively coupled to one or more application subnets 726 contained in the control plane application layer 724 and an Internet gateway 734 that can be contained in the control plane VCN 716 (e.g., Figure 6 Internet gateway 634), and application subnet(s) 726 can communicatively couple to DB subnet(s) 730 contained in control plane data layer 728 and service gateway 736 (e.g., Figure 6 Service gateway 636) and Network Address Translation (NAT) gateway 738 (e.g., Figure 6 (NAT gateway 638). The control plane VCN 716 may include the service gateway 736 and the NAT gateway 738.

[0124] The control plane VCN 716 may include a data plane mirror application layer 740 that may contain one or more application subnets 726 (e.g., Figure 6 The data plane mirror application layer 740). One or more application subnets 726 contained in the data plane mirror application layer 740 may include computational instances 744 (e.g., similar to...). Figure 6 The virtual network interface controller (VNIC) 742 (e.g., the VNIC of 642) of the computing instance 644. The computing instance 744 may facilitate the mirroring of the application subnet(s) 726 of the data plane application layer 740 and may be included in the data plane application layer 746 (e.g., Figure 6 Communication between one or more application subnets 726 in the data plane application layer 646 via VNIC 742 contained in the data plane mirror application layer 740 and VNIC 742 contained in the data plane application layer 746.

[0125] The Internet gateway 734 included in the control plane VCN 716 can be communicatively coupled to the metadata management service 752 (e.g., Figure 6 Metadata management service 762), which can communicatively couple to the public Internet 754 (e.g., Figure 6 The public internet 754 can communicatively couple to a NAT gateway 738 contained in a control plane VCN 716. The service gateway 736 contained in the control plane VCN 716 can communicatively couple to a cloud service 756 (e.g., ...). Figure 6 Cloud services 667).

[0126] In some examples, data plane VCN 718 may be included in customer lease 721. In this case, the IaaS provider may provide control plane VCN 716 for each customer, and the IaaS provider may set up a unique compute instance 744 for each customer, included in service lease 719. Each compute instance 744 may allow communication between control plane VCN 716 included in service lease 719 and data plane VCN 718 included in customer lease 721. Compute instance 744 may allow resources provisioned in control plane VCN 716 included in service lease 719 to be deployed or otherwise used in data plane VCN 718 included in customer lease 721.

[0127] In other examples, an IaaS provider's customer may have a database residing in customer lease 721. In this example, control plane VCN 716 may include a data plane mirror application layer 740, which may include one or more application subnets 726. Data plane mirror application layer 740 may reside in data plane VCN 718, but it may not reside in data plane VCN 718. That is, data plane mirror application layer 740 may have access to customer lease 721, but it may not reside in data plane VCN 718 or be owned or operated by an IaaS provider's customer. Data plane mirror application layer 740 may be configured to invoke data plane VCN 718, but it may not be configured to invoke any entity contained in control plane VCN 716. Customers may expect to deploy or otherwise use resources provisioned in the control plane VCN 716 in the data plane VCN 718, and the data plane mirroring application layer 740 can facilitate the customer's desired deployment or other use of resources.

[0128] In some embodiments, an IaaS provider's customer can apply filters to data plane VCN 718. In this embodiment, the customer can determine what data plane VCN 718 can access, and the customer can restrict access from data plane VCN 718 to the public Internet 754. The IaaS provider may not be able to apply filters or otherwise control data plane VCN 718's access to any external networks or databases. Applying filters and controls to data plane VCN 718 contained in customer lease 721 helps isolate data plane VCN 718 from other customers and the public Internet 754.

[0129] In some embodiments, cloud service 756 may be invoked by service gateway 736 to access services that may not exist on public internet 754, control plane VCN 716, or data plane VCN 718. The connection between cloud service 756 and control plane VCN 716 or data plane VCN 718 may not be real-time or continuous. Cloud service 756 may reside on different networks owned or operated by an IaaS provider. Cloud service 756 may be configured to receive calls from service gateway 736 and may be configured not to receive calls from public internet 754. Some cloud services 756 may be isolated from other cloud services 756, and control plane VCN 716 may be isolated from cloud services 756 that may not be in the same region as control plane VCN 716. For example, control plane VCN 716 may be located in "Region 1," and cloud service "Deployment 8" may be located in both "Region 1" and "Region 2." If the service gateway 736, contained in the control plane VCN 716 located in region 1, makes a call to deployment 8, then that call can be transmitted to deployment 8 in region 1. In this example, the control plane VCN 716 or deployment 8 in region 1 may not be communicatively coupled to or otherwise communicate with deployment 8 in region 2.

[0130] Figure 8 This is a block diagram 800 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 802 (e.g., Figure 6 The service provider (602) can communicatively couple to the secure host lease (804) (e.g., Figure 6 Secure hosting lease 604), the secure hosting lease 804 may include a Virtual Cloud Network (VCN) 806 (e.g., Figure 6 VCN606) and Secure Host Subnet 808 (e.g., Figure 6 The secure host subnet 608). VCN 806 can include LPG 810 (e.g., Figure 6The LPG 610), which can be communicatively coupled to the SSH VCN 812 via the LPG 810 included in the SSH VCN 812 (e.g., Figure 6 SSH VCN 612). SSH VCN 812 can include SSH subnet 814 (e.g., Figure 6 SSH subnet 614), and SSH VCN 812 can be communicatively coupled to control plane VCN 816 via LPG 810 included in control plane VCN 816 (e.g., Figure 6 The control plane VCN 616) and coupled to the data plane VCN 818 via the LPG 810 contained in the data plane VCN 818 (e.g., Figure 6 Data plane VCN 618). Control plane VCN 816 and data plane VCN 818 can be included in service lease 819 (e.g., Figure 6 In the service rental (619).

[0131] The control plane VCN 816 may include a load balancer (LB) subnet 822 (e.g., Figure 6 The control plane DMZ layer 820 of (one or more) LB subnets 622) (e.g., Figure 6 The control plane DMZ layer 620 may include one or more application subnets 826 (e.g., similar to...). Figure 6 The control plane application layer 824 of (one or more) application subnets 626 (e.g., Figure 6 The control plane application layer 624), may include (one or more) DB subnets 830, and the control plane data layer 828 (e.g., Figure 6 The control plane data layer 628). One or more LB subnets 822 contained in the control plane DMZ layer 820 can be communicatively coupled to one or more application subnets 826 contained in the control plane application layer 824 and an Internet gateway 834 that can be contained in the control plane VCN 816 (e.g., Figure 6 Internet gateway 634), and application subnet 826 can communicatively couple to DB subnet 830 contained in control plane data layer 828 and service gateway 836 (e.g., Figure 6 The service gateway) and the Network Address Translation (NAT) gateway 838 (e.g., Figure 6 (NAT gateway 638). The control plane VCN 816 may include the service gateway 836 and the NAT gateway 838.

[0132] The data plane VCN 818 may include the data plane application layer 846 (e.g., Figure 6 Data plane application layer 646), data plane DMZ layer 848 (e.g., Figure 6 Data plane DMZ layer 648), and data plane data layer 850 (e.g., Figure 6 The data plane data layer 660. The data plane DMZ layer 848 may include one or more trusted application subnets 860 and one or more untrusted application subnets 862 communicatively coupled to the data plane application layer 846, and one or more LB subnets 822 of the Internet gateway 834 contained in the data plane VCN 818. The one or more trusted application subnets 860 may be communicatively coupled to the service gateway 836 contained in the data plane VCN 818, the NAT gateway 838 contained in the data plane VCN 818, and one or more DB subnets 830 contained in the data plane data layer 850. The one or more untrusted application subnets 862 may be communicatively coupled to the service gateway 836 contained in the data plane VCN 818 and one or more DB subnets 830 contained in the data plane data layer 850. The data plane data layer 850 may include one or more DB subnets 830 communicatively coupled to the service gateway 836 contained in the data plane VCN 818.

[0133] One or more untrusted application subnets 862 may include one or more primary VNICs 864(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 866(1)-(N). Each tenant VM 866(1)-(N) may be communicatively coupled to a corresponding application subnet 867(1)-(N) that may be contained in a corresponding container egress VCN 868(1)-(N), which may be contained in a corresponding customer lease 880(1)-(N). A corresponding secondary VNIC 882(1)-(N) may facilitate communication between one or more untrusted application subnets 862 contained in a data plane VCN 818 and the application subnets contained in the container egress VCN 868(1)-(N). Each container egress VCN 868(1)-(N) may include a NAT gateway 838 that can be communicatively coupled to the public Internet 854 (e.g., Figure 6 The public internet (664).

[0134] An Internet gateway 834, contained in the control plane VCN 816 and the data plane VCN 818, can be communicatively coupled to a metadata management service 852 (e.g., Figure 6Metadata management service 852 can be communicatively coupled to the public internet 854. The public internet 854 can be communicatively coupled to a NAT gateway 838 contained in a control plane VCN 816 and a data plane VCN 818. Service gateway 836 contained in the control plane VCN 816 and the data plane VCN 818 can be communicatively coupled to a cloud service 856.

[0135] In some embodiments, the data plane VCN 818 may be integrated with the customer lease 880. Such integration may be useful or desired by the IaaS provider's customers in certain situations, such as when support may be expected during code execution. Customers may provide code that could be destructive, might communicate with other customer resources, or might otherwise cause undesirable effects. In response, the IaaS provider may determine whether to run the code provided by the customer to the IaaS provider.

[0136] In some examples, an IaaS provider's customer may grant the IaaS provider temporary network access and request functionality attached to the data plane application layer 846. The code running this functionality may execute in VMs 866(1)-(N) and may not be configured to run anywhere else on the data plane VCN 818. Each VM 866(1)-(N) may be connected to a customer lease 880. The corresponding container 881(1)-(N) contained in VMs 866(1)-(N) may be configured to run the code. In this case, dual isolation may exist (e.g., container 881(1)-(N) runs the code, where container 881(1)-(N) may be contained in at least one VM 866(1)-(N) contained in untrusted application subnet 862), which can help prevent incorrect or otherwise unintended code from corrupting the IaaS provider's network or the networks of different customers. Container 881(1)-(N) may be communicatively coupled to customer lease 880 and may be configured to transmit or receive data from customer lease 880. Container 881(1)-(N) may not be configured to transmit or receive data from any other entity in data plane VCN 818. After the code execution is complete, the IaaS provider may terminate or otherwise dispose of container 881(1)-(N).

[0137] In some embodiments, one or more trusted application subnets 860 may run code that can be owned or operated by an IaaS provider. In this embodiment, one or more trusted application subnets 860 may be communicatively coupled to one or more database subnets 830 and configured to perform CRUD operations in one or more database subnets 830. One or more untrusted application subnets 862 may be communicatively coupled to one or more database subnets 830, but in this embodiment, one or more untrusted application subnets may be configured to perform read operations in one or more database subnets 830. Containers 881(1)-(N) that may be contained in each customer's VM 866(1)-(N) and may run code from the customer may not be communicatively coupled to one or more database subnets 830.

[0138] In other embodiments, the control plane VCN 816 and the data plane VCN 818 may be coupled without direct communication. In this embodiment, there may be no direct communication between the control plane VCN 816 and the data plane VCN 818. However, communication can occur indirectly through at least one method. The LPG 810 may be established by an IaaS provider, which can facilitate communication between the control plane VCN 816 and the data plane VCN 818. In another example, either the control plane VCN 816 or the data plane VCN 818 may invoke the cloud service 856 via the service gateway 836. For example, an invocation of the cloud service 856 from the control plane VCN 816 may include a request for a service that can communicate with the data plane VCN 818.

[0139] Figure 9 This is a block diagram 900 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 902 (e.g., Figure 6 The service provider (602) can communicatively couple to the secure host lease (904) (e.g., Figure 6 Secure hosting lease 604), the secure hosting lease 904 may include a Virtual Cloud Network (VCN) 906 (e.g., Figure 6 VCN606) and Secure Host Subnet 908 (e.g., Figure 6 The secure host subnet 608). VCN 906 can include LPG 910 (e.g., Figure 6 The LPG 610), the LPG 910 can be accessed via SSH VCN 912 (e.g., LPG 610), Figure 6 The LPG 910 in SSH VCN 912 is communicatively coupled to SSH VCN 912. SSH VCN 912 may include SSH subnet 914 (e.g., Figure 6SSH subnet 614), and SSH VCN 912 can be communicatively coupled to control plane VCN 916 via LPG 910 contained in control plane VCN 916 (e.g., Figure 6 The control plane VCN 616) and coupled to the data plane VCN 918 via the LPG910 contained in the data plane VCN 918 (e.g., Figure 6 Data plane VCN 618). Control plane VCN 916 and data plane VCN 918 may be included in service lease 919 (e.g., Figure 6 In the service rental (619).

[0140] The control plane VCN 916 may include one or more LB subnets 922 (e.g., Figure 6 The control plane DMZ layer 920 of (one or more) LB subnets 622) (e.g., Figure 6 The control plane DMZ layer 620), may include (one or more) application subnets 926 (e.g., Figure 6 The control plane application layer 924 of (one or more) application subnets 626 (e.g., Figure 6 The control plane application layer 624) may include (one or more) DB subnets 930 (e.g., Figure 7 The control plane data layer 928 of (one or more) DB subnets 730 (e.g., Figure 6 The control plane data layer 628). One or more LB subnets 922 contained in the control plane DMZ layer 920 can be communicatively coupled to one or more application subnets 926 contained in the control plane application layer 924 and an Internet gateway 934 that can be contained in the control plane VCN 916 (e.g., Figure 6 Internet gateway 634), and application subnet(s) 926 can communicatively couple to DB subnet(s) 930 contained in control plane data layer 928 and service gateway 936 (e.g., Figure 6 The service gateway) and Network Address Translation (NAT) gateway 938 (e.g., Figure 6 (NAT gateway 638). The control plane VCN 916 may include the service gateway 936 and the NAT gateway 938.

[0141] The data plane VCN 918 may include the data plane application layer 946 (e.g., Figure 6 Data plane application layer 646), data plane DMZ layer 948 (e.g., Figure 6 Data plane DMZ layer 648), and data plane data layer 950 (e.g., Figure 6The data plane data layer 948 may include one or more trusted application subnets 960 that can be communicatively coupled to the data plane application layer 946 (e.g., data plane data layer 660). Figure 7 (one or more) trusted application subnets 770 and (one or more) untrusted application subnets 962 (e.g., Figure 7 The data plane includes one or more untrusted application subnets 772 and one or more LB subnets 922 of an Internet gateway 934 contained in the data plane VCN 918. One or more trusted application subnets 960 may communicatively couple to a service gateway 936 contained in the data plane VCN 918, a NAT gateway 938 contained in the data plane VCN 918, and one or more DB subnets 930 contained in the data plane data layer 950. One or more untrusted application subnets 962 may communicatively couple to a service gateway 936 contained in the data plane VCN 918 and one or more DB subnets 930 contained in the data plane data layer 950. The data plane data layer 950 may include one or more DB subnets 930 that may communicatively couple to a service gateway 936 contained in the data plane VCN 918.

[0142] One or more untrusted application subnets 962 may include a primary VNIC 964(1)-(N) communicatively coupled to tenant virtual machines (VMs) 966(1)-(N) residing within one or more untrusted application subnets 962. Each tenant VM 966(1)-(N) may run code in a corresponding container 967(1)-(N) and is communicatively coupled to an application subnet 926 that may be contained in a data plane application layer 946 contained in a container egress VCN 968. A corresponding secondary VNIC 972(1)-(N) may facilitate communication between one or more untrusted application subnets 962 contained in a data plane VCN 918 and the application subnets contained in a container egress VCN 968. The container egress VCN may include a public internet 954 (e.g., Figure 6 The public internet (664) uses NAT gateway 938.

[0143] The Internet gateway 934, contained in the control plane VCN 916 and the data plane VCN 918, can be communicatively coupled to the metadata management service 952 (e.g., Figure 6Metadata management service 952 (662) can be communicatively coupled to the public Internet 954. The public Internet 954 can be communicatively coupled to a NAT gateway 938 contained in a control plane VCN 916 and a data plane VCN 918. A service gateway 936 contained in a control plane VCN 916 and a data plane VCN 918 can be communicatively coupled to a cloud service 956.

[0144] In some examples, Figure 9 The architecture shown in the block diagram 900 can be considered as... Figure 7 This is an exception to the pattern shown in the architecture of block diagram 700, and this pattern may be what the IaaS provider's customers would expect if the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected region). The customer can access in real time the corresponding container 967(1)-(N) contained in each customer's VM 966(1)-(N). Container 967(1)-(N) can be configured to invoke the corresponding auxiliary VNIC 972(1)-(N) contained in one or more application subnets 926 of the data plane application layer 946, which may be contained in the container egress VCN 968. The auxiliary VNIC 972(1)-(N) can transmit the call to the NAT gateway 938, which can transmit the call to the public Internet 954. In this example, the container 967(1)-(N) that can be accessed by the customer in real time can be isolated from the control plane VCN 916 and from other entities contained in the data plane VCN 918. Container 967(1)-(N) can also be isolated from resources from other customers.

[0145] In other examples, a client can use container 967(1)-(N) to invoke cloud service 956. In this example, the client can run code within container 967(1)-(N) requesting a service from cloud service 956. Container 967(1)-(N) can then transmit the request to auxiliary VNIC 972(1)-(N), which can then transmit the request to a NAT gateway, which can then transmit the request to public internet 954. Public internet 954 can then transmit the request via internet gateway 934 to one or more LB subnets 922 contained in control plane VCN 916. In response to determining that the request is valid, one or more LB subnets can then transmit the request to one or more application subnets 926, which can then transmit the request to cloud service 956 via service gateway 936.

[0146] It should be recognized that the IaaS architectures 600, 700, 800, and 900 depicted in the figures may have other components besides those depicted. Furthermore, the embodiments shown in the figures are merely some examples of cloud infrastructure systems that can be incorporated into embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the figures, may combine two or more components, or may have different configurations or component arrangements.

[0147] In some embodiments, the IaaS system described herein may include application suites, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is the Oracle Cloud Infrastructure (OCI) provided by this assignee.

[0148] Figure 10 An example computer system 1000 in which various embodiments can be implemented is illustrated. System 1000 can be used to implement any of the computer systems described above. As shown, computer system 1000 includes a processing unit 1004 that communicates with a plurality of peripheral subsystems via a bus subsystem 1002. These peripheral subsystems may include a processing acceleration unit 1006, an I / O subsystem 1008, a storage subsystem 1018, and a communication subsystem 1024. Storage subsystem 1018 includes a tangible computer-readable storage medium 1022 and system memory 1010.

[0149] Bus subsystem 1002 provides a mechanism for enabling various components and subsystems of computer system 1000 to communicate with each other as intended. While bus subsystem 1002 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1002 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of the various bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0150] A processing unit 1004, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 1000. One or more processors may be included in the processing unit 1004. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 1004 may be implemented as one or more independent processing units 1032 and / or 1034, each including a single-core or multi-core processor. In other embodiments, the processing unit 1004 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0151] In various embodiments, processing unit 1004 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processing unit 1004 and / or storage subsystem 1018. With appropriate programming, processing unit 1004 can provide the various functions described above. Computer system 1000 may additionally include processing acceleration unit 1006, which may include digital signal processor (DSP), dedicated processor, etc.

[0152] I / O subsystem 1008 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, keyboards, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the Microsoft Kinect® motion sensor, which enables users to control and interact with input devices such as the Microsoft Xbox® 360 game controller via a natural user interface using gestures and voice commands. User interface input devices may also include eye posture recognition devices, such as the Google Glass® blink detector, which detects eye activity from the user (e.g., “blinking” when taking a photo and / or making menu selections) and translates the eye posture into input in an input device (e.g., Google Glass®). Furthermore, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0153] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointing sticks, game panels and drawing tablets, as well as audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0154] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1000 to a user or other computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0155] Computer system 1000 may include storage subsystem 1018, which provides a tangible, non-transitory, computer-readable storage medium for storing software and data constructs that provide the functionality of the embodiments described in this disclosure. The software may include programs, code modules, instructions, scripts, etc., which, when executed by one or more cores or processors of processing unit 1004, provide the aforementioned functionality. Storage subsystem 1018 may also provide a repository for storing data used according to this disclosure.

[0156] like Figure 10 As illustrated in the example, the storage subsystem 1018 may include various components, including system memory 1010, computer-readable storage medium 1022, and computer-readable storage medium reader 1020. System memory 1010 may store program instructions 1012 that can be loaded and executed by processing unit 1004. System memory 1010 may also store data 1014 used during the execution of instructions and / or data generated during the execution of program instructions. Various types of programs may be loaded into system memory 1010, including but not limited to client applications, web browsers, middleware applications, relational database management systems (RDBMS), virtual machines, containers, etc.

[0157] System memory 1010 may also store operating system 1016. Examples of operating system 1016 may include various versions of Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercial UNIX® or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google Chrome® OS, etc.) and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS. In some implementations of computer system 1000 that execute one or more virtual machines, the virtual machine and its guest operating system (GOS) may be loaded into system memory 1010 and executed by one or more processors or cores of processing unit 1004.

[0158] System memory 1010 may employ different configurations depending on the type of computer system 1000. For example, system memory 1010 may be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, etc.). Different types of RAM configurations may be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some implementations, system memory 1010 may include a basic input / output system (BIOS) containing basic routines that facilitate, for example, the transfer of information between elements within computer system 1000 during startup.

[0159] Computer-readable storage medium 1022 may represent remote, local, fixed and / or removable storage devices and storage media for temporarily and / or more permanently containing and storing computer-readable information (including instructions executed by the processing unit 1004 of the computer system 1000) for use by the computer system 1000.

[0160] Computer-readable storage medium 1022 may include any suitable medium known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or other tangible computer-readable media.

[0161] For example, computer-readable storage medium 1022 may include hard disk drives that read from or write to non-removable non-volatile magnetic media, disk drives that read from or write to removable non-volatile magnetic disks, and optical disc drives that read from or write to removable non-volatile optical discs (such as CD ROMs, DVDs, and Blu-ray® discs or other optical media). Computer-readable storage medium 1022 may include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, digital audio tapes, and so on. Computer-readable storage medium 1022 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program modules and other data for computer system 1000.

[0162] Machine-readable instructions executed by one or more processors or cores of processing unit 1004 may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile memory devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard disk drives, floppy disk drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0163] The communication subsystem 1024 provides interfaces to other computer systems and networks. The communication subsystem 1024 serves as an interface for receiving data from other systems and sending data from computer system 1000 to other systems. For example, the communication subsystem 1024 enables computer system 1000 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1024 may include radio frequency (RF) transceiver components (e.g., advanced data network technologies using cellular telephone technology, such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 602.10 series standards), or other mobile communication technologies, or any combination thereof), global positioning system (GPS) receiver components, and / or other components for accessing wireless voice and / or data networks. In some embodiments, as an addition to or alternative to the wireless interface, the communication subsystem 1024 may provide a wired network connection (e.g., Ethernet).

[0164] In some embodiments, the communication subsystem 1024 may also represent one or more users who can use the computer system 1000 to receive input communications in the form of structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc.

[0165] For example, the communication subsystem 1024 can be configured to receive data feeds 1026 in real time from users of social networks and / or other communication services, such as Twitter® feeds, Facebook® updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0166] Furthermore, the communication subsystem 1024 can also be configured to receive data in the form of a continuous data stream, which may include an event stream 1028 and / or event updates 1030 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quotation machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and so on.

[0167] The communication subsystem 1024 can also be configured to output structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1000.

[0168] The computer system 1000 can be one of a variety of types, including handheld portable devices (e.g., iPhone® cellular phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google® Glass head-mounted displays), PCs, workstations, mainframes, information stations, server racks, or any other data processing system.

[0169] Due to the constantly evolving nature of computers and networks, the description of the computer system 1000 depicted in the figures is merely a concrete example. Many other configurations with more or fewer components than the system depicted in the figures are possible. For example, custom hardware may be used and / or specific elements may be implemented using hardware, firmware, software (including applets), or a combination thereof. Additionally, connections to other computing devices, such as network input / output devices, may also be employed. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and / or methods for implementing the various embodiments.

[0170] While specific embodiments have been described, various modifications, alterations, alternative constructions, and equivalents are also included within the scope of this disclosure. The embodiments are not limited to operation within certain specific data processing environments, but can be freely operated within multiple data processing environments. Furthermore, although the embodiments have been described using a specific series of transactions and steps, those skilled in the art will understand that the scope of this disclosure is not limited to the described series of transactions and steps. Various features and aspects of the above embodiments can be used individually or in combination.

[0171] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments may be implemented using only hardware, or only software, or a combination thereof. The various processes described herein can be implemented in any combination on the same processor or on different processors. Accordingly, where a component or service is described as being configured to perform certain operations, such configuration can be accomplished, for example, by designing electronic circuits to perform operations, by programming programmable electronic circuits (such as microprocessors), or any combination thereof. Processes may communicate using a variety of technologies, including but not limited to conventional technologies for inter-process communication, and different pairs of processes may use different technologies, or the same pair of processes may use different technologies at different times.

[0172] Accordingly, the specification and drawings are intended to be illustrative rather than restrictive. However, it will be apparent that additions, omissions, deletions, and other modifications and alterations may be made thereto without departing from the broader spirit and scope set forth in the claims. Thus, while specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0173] In the context of describing the disclosed embodiments (particularly in the context of the following claims), the terms “a,” “an,” and “the,” and similar designations, are to be interpreted as encompassing both singular and plural, unless otherwise indicated herein or clearly contradicted by the context. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” are to be interpreted as open-ended terms (i.e., meaning “including but not limited to”). The term “connected” should be interpreted as partially or wholly contained in, attached to, or joined together, even if something exists in between. Unless otherwise indicated herein, the enumeration of value ranges herein is intended only as a shorthand method for individually referencing each individual value falling within that range, and each individual value is incorporated into the specification as if it were individually enumerated herein. Unless otherwise indicated herein or clearly contradicted by the context, all methods described herein can be performed in any suitable order. The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate the embodiments and does not constitute a limitation on the scope of this disclosure, unless otherwise stated. Nothing in the specification should be construed as indicating that any unclaimed element is essential to the practice of this disclosure.

[0174] As used herein, when an action is “based on” something, it means that the action is at least partially based on at least a portion of that thing. As used herein, the terms “substantially,” “about,” and “approximately” are defined as primarily but not necessarily exactly as specified (and include exactly as specified), as understood by one of ordinary skill in the art. In any disclosed embodiment, the terms “substantially,” “about,” or “approximately” may be replaced with “within [percentage],” where percentages include 0.1, 1, 6, and 8%.

[0175] Disjunctive language, such as the phrase “at least one of X, Y, or Z”, unless otherwise explicitly stated, is intended to be understood in the context generally used to represent items, terms, etc., and may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Therefore, such disjunctive language is generally not intended to, and should not, imply that some embodiments require the presence of at least one of X, at least one of Y, or at least one of Z, each individually.

[0176] This document describes preferred embodiments of the present disclosure, including the best modes known for carrying out the present disclosure. Variations of those preferred embodiments will become apparent to those skilled in the art upon reading the foregoing description. Those skilled in the art should be able to suitably employ such variations and may practice the present disclosure in ways other than those specifically described herein. Accordingly, the present disclosure includes all modifications and equivalents to the subject matter recited in the appended claims, where permitted by applicable law. Furthermore, unless otherwise indicated herein, the present disclosure includes any combination of the foregoing elements in all its possible variations.

[0177] All references cited in this article, including publications, patent applications and patents, are incorporated into this article by reference to the same extent as if each reference individually and specifically indicated to be incorporated by reference and elaborated in full in this article.

[0178] In the foregoing specification, various aspects of this disclosure have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that this disclosure is not limited thereto. The various features and aspects of the foregoing disclosure may be used individually or in combination. Furthermore, embodiments may be used in any number of settings and applications other than those described herein without departing from the broader spirit and scope of this specification. Accordingly, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A computer-implemented method, comprising: Access text transcription, which corresponds to the interaction between a first entity and a second entity; The transcribed text is segmented into multiple parts; For each of the plurality of parts: The first machine learning model prompts the identification of one or more entities contained in the corresponding section. Using a second machine learning model, prompts are made to extract one or more facts from the corresponding portion, at least in part, based on the one or more entities. Add one or more of the facts to the fact set; Multiple third-party machine learning models are used to prompt the generation of a set of note segments based at least in part on the set of facts, wherein each note segment in the set of note segments corresponds to a segment of subjective, objective, evaluative, and planning (SOAP) notes; The SOAP note is generated by combining the note segments in the set of note segments; and The SOAP notes are stored in a database associated with at least one of the first entity and the second entity.

2. The computer-implemented method of claim 1, wherein the text transcription includes a first number of tokens, wherein each of the plurality of portions includes a second number of tokens less than the first number of tokens, and wherein segmenting the text transcription into the plurality of portions includes selecting a corresponding portion of the text transcription having a number of tokens corresponding to the second number of tokens, and determining whether the corresponding portion of the text transcription meets a predetermined criterion.

3. The computer-implemented method of any one of claims 1 and 2, wherein using the first machine learning model prompt to identify the one or more entities included in the corresponding portion comprises: The named entity recognition model is used to extract at least one entity from the corresponding part, and the first machine learning model prompt is generated based at least in part on the corresponding part and the at least one entity.

4. The computer-implemented method of any one of claims 1-3, wherein the second machine learning model hint includes the corresponding portion, the one or more entities, and a query for querying one or more machine learning models to extract the one or more facts from the corresponding portion.

5. The computer-implemented method according to any one of claims 1-4, further comprising: Using multiple fourth machine learning model prompts to generate a set of categorized facts from the fact set, wherein each categorized fact in the set of categorized facts corresponds to a fact in the fact set and is associated with a specific category label selected from a set of category labels, wherein generating the set of note segments using the multiple third machine learning model prompts includes generating the set of note segments based at least in part on the set of categorized facts using the multiple third machine learning model prompts.

6. The computer-implemented method of claim 5, wherein each category label in the set of category labels is associated with a corresponding SOAP note segment in a plurality of SOAP note segments, and wherein generating the set of categorized facts from the fact set using the plurality of fourth machine learning model prompts includes classifying each corresponding fact in the fact set as corresponding to a specific SOAP note in the plurality of SOAP note segments using the plurality of fourth machine learning model prompts.

7. The computer-implemented method of claim 5, wherein generating the set of note segments using the plurality of third machine learning model prompts based at least in part on the set of categorized facts comprises: Extract information associated with the second entity from the electronic records associated with the second entity; Divide the set of classified facts into subsets of classified facts; Based on the subset of classified facts and the information, a machine learning model prompt is generated; as well as The machine learning model is used to prompt the generation of the corresponding note segments in the set of note segments.

8. The computer-implemented method of any one of claims 1-7, wherein the first entity is a healthcare provider, wherein the second entity is a patient associated with the healthcare provider, and wherein storing the SOAP notes in the database comprises storing the SOAP notes in an electronic health record associated with the patient.

9. A system comprising: One or more processing systems; as well as A computer-readable medium storing one or more instructions, which, when executed by the one or more processing systems, cause the systems to perform operations including: Access text transcription, which corresponds to the interaction between a first entity and a second entity; The transcribed text is segmented into multiple parts; For each of the plurality of parts: The first machine learning model prompts the identification of one or more entities contained in the corresponding section. Using a second machine learning model, prompts are made to extract one or more facts from the corresponding portion, at least in part, based on the one or more entities. Add one or more of the facts to the fact set; Multiple third-party machine learning models are used to prompt the generation of a set of note segments based at least in part on the set of facts, wherein each note segment in the set of note segments corresponds to a segment of subjective, objective, evaluative, and planning (SOAP) notes; The SOAP note is generated by combining the note segments in the set of note segments; and The SOAP notes are stored in a database associated with at least one of the first entity and the second entity.

10. The system of claim 9, wherein the text transcription includes a first number of tokens, wherein each of the plurality of portions includes a second number of tokens less than the first number of tokens, and wherein segmenting the text transcription into the plurality of portions includes selecting a corresponding portion of the text transcription having a number of tokens corresponding to the second number of tokens, and determining whether the corresponding portion of the text transcription meets a predetermined criterion.

11. The system of any one of claims 9 and 10, wherein using the first machine learning model prompt to identify the one or more entities included in the corresponding portion comprises: The named entity recognition model is used to extract at least one entity from the corresponding part, and the first machine learning model prompt is generated based at least in part on the corresponding part and the at least one entity.

12. The system of any one of claims 9-11, wherein the second machine learning model prompt includes the corresponding portion, the one or more entities, and a query for querying one or more machine learning models to extract the one or more facts from the corresponding portion.

13. The system according to any one of claims 9-12, wherein the operation further comprises: Using multiple fourth machine learning model prompts to generate a set of categorized facts from the fact set, wherein each categorized fact in the set of categorized facts corresponds to a fact in the fact set and is associated with a specific category label selected from a set of category labels, wherein generating the set of note segments using the multiple third machine learning model prompts includes generating the set of note segments based at least in part on the set of categorized facts using the multiple third machine learning model prompts.

14. The system of claim 13, wherein each category label in the set of category labels is associated with a corresponding SOAP note segment in a plurality of SOAP note segments, and wherein generating the set of categorized facts from the fact set using the plurality of fourth machine learning model prompts includes classifying each corresponding fact in the fact set as corresponding to a specific SOAP note in the plurality of SOAP note segments using the plurality of fourth machine learning model prompts.

15. The system of claim 13, wherein generating the set of note segments using the plurality of third machine learning model prompts based at least in part on the set of categorized facts comprises: Extract information associated with the second entity from the electronic records associated with the second entity; Divide the set of classified facts into subsets of classified facts; Based on the subset of classified facts and the information, a machine learning model prompt is generated; as well as The machine learning model is used to prompt the generation of the corresponding note segments in the set of note segments.

16. The system of any one of claims 9-15, wherein the first entity is a healthcare provider, wherein the second entity is a patient associated with the healthcare provider, and wherein storing the SOAP notes in the database comprises storing the SOAP notes in an electronic health record associated with the patient.

17. A non-transitory computer-readable medium storing one or more instructions, said instructions, when executed by one or more processors, causing a system to perform operations including: Access text transcription, which corresponds to the interaction between a first entity and a second entity; The transcribed text is segmented into multiple parts; For each of the plurality of parts: The first machine learning model prompts the identification of one or more entities contained in the corresponding section. Using a second machine learning model, prompts are made to extract one or more facts from the corresponding portion, at least in part, based on the one or more entities. Add one or more of the facts to the fact set; Multiple third-party machine learning models are used to prompt the generation of a set of note segments based at least in part on the set of facts, wherein each note segment in the set of note segments corresponds to a segment of subjective, objective, evaluative, and planning (SOAP) notes; The SOAP note is generated by combining the note segments in the set of note segments; and The SOAP notes are stored in a database associated with at least one of the first entity and the second entity.

18. One or more non-transitory computer-readable media as claimed in claim 17, wherein the text transcription includes a first number of tokens, wherein each of the plurality of portions includes a second number of tokens less than the first number of tokens, and wherein segmenting the text transcription into the plurality of portions includes selecting a corresponding portion of the text transcription having a number of tokens corresponding to the second number of tokens, and determining whether the corresponding portion of the text transcription meets a predetermined criterion.

19. One or more non-transitory computer-readable media as claimed in any one of claims 17 and 18, wherein using the first machine learning model prompt to identify the one or more entities contained in the corresponding portion comprises: The named entity recognition model is used to extract at least one entity from the corresponding part, and the first machine learning model prompt is generated based at least in part on the corresponding part and the at least one entity.

20. One or more non-transitory computer-readable media as claimed in any one of claims 17-19, wherein the second machine learning model hint includes the corresponding portion, the one or more entities, and a query for querying one or more machine learning models to extract the one or more facts from the corresponding portion.

21. The one or more non-transitory computer-readable media as described in any one of claims 17-20, wherein the operation further comprises: Using multiple fourth machine learning model prompts to generate a set of categorized facts from the fact set, wherein each categorized fact in the set of categorized facts corresponds to a fact in the fact set and is associated with a specific category label selected from a set of category labels, wherein generating the set of note segments using the multiple third machine learning model prompts includes generating the set of note segments based at least in part on the set of categorized facts using the multiple third machine learning model prompts.

22. The one or more non-transitory computer-readable media of claim 21, wherein each category label in the set of category labels is associated with a corresponding SOAP note segment in a plurality of SOAP note segments, and wherein generating the set of categorized facts from the fact set using the plurality of fourth machine learning model prompts includes classifying each corresponding fact in the fact set as corresponding to a specific SOAP note in the plurality of SOAP note segments using the plurality of fourth machine learning model prompts.

23. One or more non-transitory computer-readable media as claimed in claim 21, wherein generating the set of note segments using the plurality of third machine learning model prompts based at least in part on the set of categorized facts comprises: Extract information associated with the second entity from the electronic records associated with the second entity; Divide the set of classified facts into subsets of classified facts; Based on the subset of classified facts and the information, a machine learning model prompt is generated; as well as The machine learning model is used to prompt the generation of the corresponding note segments in the set of note segments.

24. One or more non-transitory computer-readable media as claimed in any one of claims 17-23, wherein the first entity is a healthcare provider, wherein the second entity is a patient associated with the healthcare provider, and wherein storing the SOAP notes in the database comprises storing the SOAP notes in an electronic health record associated with the patient.

25. An apparatus comprising: Components for implementing the operation of the method as described in any one of claims 1-8.

26. A computer program product comprising computer instructions that, when executed by a processor, implement the operation of the method as described in any one of claims 1-8.

Images