Medical treatment intervention procedure tool, resource and duration determination and associated devices, systems and methods

Abstract

An intervention planning system receives first user input identifying a medical professional and second user input identifying a patient. The system retrieves first data representing the medical professional in response to the first user input, retrieves second data representing the patient in response to the second user input, and retrieves third data associated with a medical facility at which the medical professional is capable of performing a treatment procedure on the patient. The system pre-processes the first data, the second data, and the third data to form a structured data set, and provides the structured data set as input to a predictive network. The system generates a procedure plan for the treatment procedure using the predictive network. The system outputs a screen display to a display including at least one of the medical tool or the medical resource.

Description

Technical Field

[0001] The topics described in this article involve using information about patients and / or about one or more physicians to optimize and plan medical interventions for patients. For example, predictive networks analyze medical information (e.g., medical images, medical image reports, medical records, etc.), interventional specialist preferences and characteristics, and available medical resources for a given patient case to generate an intervention plan. Background Technology

[0002] Any given medical intervention can be performed in a variety of ways, such as surgery, cancer treatment, medical imaging, etc. The choice of interventional methods and tools required to perform the intervention depends on the interventional specialist's knowledge, preferences, and experience, but also on external requirements such as the availability of guidelines or resources (e.g., operating room space and availability, equipment availability, other medical personnel with the required skills, etc.). Therefore, interventional planning can prove difficult. Furthermore, interventional planning may require information that is stored in a fragmented manner, often unavailable or inaccessible to the user seeking to plan the intervention.

[0003] The information included in the background section of this specification (including any references cited herein and any descriptions or discussions thereof) is included for technical reference purposes only and should not be considered as the subject matter to which the scope of this disclosure is to be limited. Summary of the Invention

[0004] An automated intervention planning tool is disclosed. An intervention planning request is received from a user (i.e., a medical professional / interventional specialist). One or more intervention plans and durations are generated by a predictive network, such as a neural network. The predictive network can also generate numerous probabilities and indices reflecting the suitability and importance of various tools and resources for the intervention. The user can then select a specific method for the intervention, and based on that selection, the intervention planning tool can automatically generate a checklist. For example, a physician might seek an intervention plan for inserting a stent into an artery. The intervention planning tool will generate various methods indicating preferred catheters and other tools for the stent placement procedure. Furthermore, the intervention planning system will also provide an estimate of the time required to complete the stent placement procedure.

[0005] The intervention planning system disclosed herein has specific, but non-exclusive, utility for planning interventions and predicting the likely duration of said interventions. The intervention planning system receives intervention planning requests and automatically generates predictions for intervention methods and likely durations. Advantageously, the intervention planning system allows users to select subsets of datasets based on similarity or difference with the user. This allows experienced intervention experts to receive predictions from the intervention planning system that conform to their standard intervention methods, and allows less experienced intervention experts to receive intervention methods informed by other intervention experts, such as those trained into the intervention planning system.

[0006] A system of one or more computers can be configured to perform specific operations or actions by installing software, firmware, hardware, or combinations thereof on the system, which in operation cause the system to perform actions. One or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform actions.

[0007] In one general aspect, this disclosure relates to a computer-implemented method. The computer-implemented method further includes: receiving first user input identifying a medical professional from a user input device; retrieving first data representing the medical professional from one or more databases in response to the first user input; receiving second user input identifying a patient from the user input device; retrieving second data representing the patient from the one or more databases in response to the second user input; retrieving third data associated with a medical facility at which the medical professional can perform a treatment procedure on the patient; preprocessing the first data, the second data, and the third data to form a structured dataset; providing the structured dataset as input to a prediction network; generating a procedure plan for the treatment procedure as output of the prediction network, wherein the procedure plan may include at least one of medical tools or medical resources to be used by the medical professional to perform the treatment procedure; and outputting a screen display to a display, the screen display including at least one of the medical tools or medical resources.

[0008] In some aspects, the implementation may include one or more of the following features. The method may include generating a predicted duration of the treatment process as the output of the prediction network. The process plan may include a series of sub-steps, and wherein the predicted duration may include multiple sub-step durations, with one sub-step duration for each sub-step in the process plan. The process plan may include multiple probabilities, each of which is associated with the medical tool or the medical resource, wherein each probability represents the likelihood of using the medical tool or the medical resource in the treatment process. The process plan may include one or more key indices, each key index associated with the medical tool or the medical resource, wherein each key index represents the importance of the medical tool or the medical resource in the outcome of the treatment process. Receiving first user input identifying a medical professional from a user input device may further include: receiving a selection of a subset of medical professionals from the user input device, wherein the structured data is restricted to data associated with the subset of medical professionals. A large language model preprocesses the first data, the second data, and the third data to form a structured dataset. The first data may include biographical information about the medical professional, the second data may include the patient’s medical records, and the third data may include at least one of the available medical tools or resources at the medical facility.

[0009] In one general aspect, this disclosure relates to a system and processor circuitry configured to: receive first user input identifying a medical professional from a user input device; retrieve first data representing the medical professional from one or more databases in response to the first user input; receive second user input identifying a patient from the user input device; retrieve second data representing the patient from the one or more databases in response to the second user input; retrieve third data associated with a medical facility at which the medical professional can perform a treatment procedure on the patient; preprocess the first data, the second data, and the third data to form a structured dataset; provide the structured dataset as input to a prediction network; generate a procedure plan for the treatment procedure as output of the prediction network, wherein the procedure plan may include at least one of medical tools or medical resources to be used by the medical professional to perform the treatment procedure; and output a screen display to a display, the screen display including at least one of the medical tools or medical resources.

[0010] In some aspects, the implementation may include one or more of the following features. The system, wherein the processor circuitry is further configured to generate a predicted duration of the treatment process as the output of the prediction network. The process plan may include a series of sub-steps, and wherein the predicted duration may include multiple sub-step durations, with one sub-step duration for each sub-step in the process plan. The process plan may include multiple probabilities, each of which is associated with the medical tool or the medical resource, wherein each probability represents the likelihood of using the medical tool or the medical resource in the treatment process. The process plan may include one or more key indices, each key index associated with the medical tool or the medical resource, wherein each key index represents the importance of the medical tool or the medical resource in the outcome of the treatment process. The processor circuitry is further configured to receive a selection of a subset of medical professionals from a user input device, wherein the structured data is restricted to data associated with the subset of medical professionals. A large language model preprocesses the first data, the second data, and the third data to form a structured dataset. The first data may include biographical information about the medical professional, the second data may include the patient’s medical records, and the third data may include at least one of the available medical tools or resources at the medical facility.

[0011] In one general aspect, this disclosure relates to a non-transient machine-readable medium that may include a plurality of machine-executable instructions. The non-transient machine-readable medium further includes: receiving first user input from a user input device identifying a medical professional; retrieving first data representing the medical professional from one or more databases in response to the first user input; receiving second user input from the user input device identifying a patient; retrieving second data representing the patient from the one or more databases in response to the second user input; retrieving third data associated with a medical facility at which the medical professional may perform a treatment procedure on the patient; preprocessing the first data, the second data, and the third data to form a structured dataset; providing the structured dataset as input to a predictive network; generating a procedure plan for the treatment procedure as output of the predictive network, wherein the procedure plan may include at least one of medical tools or medical resources to be used by the medical professional to perform the treatment procedure; and outputting a screen display to a display, the screen display including at least one of the medical tools or medical resources.

[0012] In some aspects, the implementation may include one or more of the following features: a non-transient machine-readable medium, in which one or more processors are also caused to perform operations that may include: generating a predicted duration of the treatment procedure as the output of the prediction network. The procedure plan may include a series of sub-steps, and wherein the predicted duration may include multiple sub-step durations, with one sub-step duration existing for each sub-step in the procedure plan. The procedure plan may include multiple probabilities, each of which is associated with the medical tool or the medical resource, wherein each probability represents the likelihood of using the medical tool or the medical resource in the treatment procedure.

[0013] The present invention is provided to present, in a simplified form, the selection of concepts further described below in the detailed description. The present invention is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. A broader presentation of the features, details, utility, and advantages of the automated measurement point detection system as defined in the claims is provided in the following written description of various aspects of this disclosure and is illustrated in the accompanying drawings. Attached Figure Description

[0014] The illustrative aspects of this disclosure will be described with reference to the accompanying drawings, in which:

[0015] Figure 1 This is a schematic diagram of a networked system for intervention planning based on aspects of this disclosure.

[0016] Figure 2 This is a schematic diagram of a processor circuit according to aspects of this disclosure.

[0017] Figure 3 This is a schematic diagram of a deep learning network configuration according to aspects of this disclosure.

[0018] Figure 4 It is a schematic diagram of at least a portion of the intervention planning system according to aspects of this disclosure.

[0019] Figure 5 It is a schematic sequence diagram of the intervention planning process according to aspects of this disclosure.

[0020] Figure 6 This is a schematic diagram of at least a portion of a system for training a prediction network in an intervention planning tool, according to aspects of this disclosure.

[0021] Figure 7 This is a schematic sequence diagram of a process for training a predictive network in an intervention planning tool, according to aspects of this disclosure.

[0022] Figure 8 This is a schematic diagram of a patient database based on aspects of this disclosure.

[0023] Figure 9 This is a schematic diagram of the intervention expert database based on aspects of this disclosure.

[0024] Figure 10 This is a schematic diagram of a resource database based on aspects of this disclosure.

[0025] Figure 11 This is an example shown in accordance with aspects of this disclosure for use in intervention planning requests.

[0026] Figure 12 It is a schematic diagram of a sequence of interfaces based on the prediction of an interventional planning system according to aspects of this disclosure.

[0027] Figure 13 This is a schematic diagram of a time slot planning support system based on aspects of this disclosure.

[0028] Figure 14 This is a schematic flowchart of a method for intervention planning based on aspects of this disclosure.

[0029] Figure 15 This is a schematic flowchart of a method for training a prediction network in an intervention planning system according to aspects of this disclosure. Detailed Implementation

[0030] According to at least one aspect of this disclosure, an intervention planning system for automatically generating intervention plans is provided. This allows, for example, users (including medical professionals and other interventional specialists) to choose among a variety of possible methods for the intervention. Furthermore, the system provides an automated manner for implementing the user's preference for an optimal level of experience used to generate the intervention plan.

[0031] These descriptions are provided for illustrative purposes only and should not be construed as limiting the scope of the intervention planning system. Certain features may be added, removed, or modified without departing from the spirit of the claimed subject matter.

[0032] To facilitate an understanding of the principles of this disclosure, reference will now be made to the aspects illustrated in the accompanying drawings, and these aspects will be described using specific language. However, it should be understood that this is not intended to limit the scope of this disclosure. Any changes and further modifications to the described apparatus, systems, and methods, as well as any further application of the principles of this disclosure, are fully contemplated and included within this disclosure, as would normally occur to those skilled in the art to which this disclosure pertains. In particular, it is fully contemplated that features, components, and / or steps described with respect to one aspect can be combined with features, components, and / or steps described with respect to other aspects of this disclosure. However, for the sake of brevity, multiple iterations of these combinations will not be described separately. As used herein, access may include querying, retrieving, sorting, etc.

[0033] The systems and methods disclosed in this paper offer numerous benefits. For a given intervention procedure to be scheduled, experienced interventional specialists may want to adhere to their workflow, while inexperienced specialists may benefit from receiving guidance from more experienced colleagues. Therefore, the systems and methods disclosed in this paper allow interventional specialists to select proposal preferences for their preferred level of experience for intervention prediction.

[0034] Choosing the best methods and tools for a process and estimating its duration is not always easy; therefore, the systems and methods described in this paper provide intervention planning, including selecting the necessary methods, tools, and resources based on historical data and profiles of intervention experts.

[0035] As used in this article, “treatment plan,” “process plan,” and “intervention prediction” can be used synonymously.

[0036] Figure 1 This is a schematic diagram of a networked system 100 for intervention planning according to aspects of this disclosure. The networked system 100 can, for example, be used to receive intervention planning requests from a user. Intervention planning requests can be transmitted to different components of the networked system 100 to facilitate intervention planning. The networked system 100 can provide the user with predictions in the form of treatment plans output from a prediction network.

[0037] Networked system 100 is used to generate treatment plans. Networked system 100 may include network / cloud computers 110 and 140, user computer 160, medical imaging console 170, and medical imaging equipment 180. Network / cloud computers 110 and 140 can communicate with each other, each transmitting and / or receiving data and information from each other. User computer 160... Figure 1The user computer 160 is depicted as communicating with network / cloud computer 110. However, in some instances, the user computer 160 may communicate with either or both of the network / cloud computers 110 and 140. The user computer 160 may communicate with medical imaging console 170, which in turn communicates with medical imaging equipment 180. As described herein, communication between different components can be achieved through any number of connections (e.g., wired and / or wireless).

[0038] The network / cloud computer 110 may include a processor 112, an input device 114, a display 116, a communication interface 117, and a memory 118.

[0039] Processor 112 may include a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), controller, field-programmable gate array (FPGA) device, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 112 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 112 is configured to process instructions stored in memory 118. Processor 112 is connected to communication interface 117.

[0040] Input device 114 allows users to make selections or provide instructions to network / cloud computer 110. Input device can be a mouse, touch screen, touchpad, etc.

[0041] Display 116 is coupled to processor 112. Display 116 can be a monitor or any suitable display. Display 116 is configured to display information from intervention planner 150, data structured network 155, time slot planning support 126, databases 120, 122, 124, network performance, or any other system diagnostic information.

[0042] Communication interface 117 is coupled to processor 112. Communication interface 117 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. Communication interface 117 may include hardware and / or software components implementing a specific communication protocol suitable for transporting signals to other devices in networking system 100. Communication interface 117 may be referred to as a communication device or communication interface module.

[0043] Memory 118 is coupled to processor 112. Memory 118 can be any suitable storage device, such as cache memory (e.g., cache memory of processor 112), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory device, hard disk drive, solid-state drive, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0044] Memory 118 may be configured to store subject information, measurement results, data, or files, including the subject's medical history, the history of procedures performed, anatomical or biological characteristics, traits, or medical conditions associated with the subject; computer-readable instructions, such as code, software, or other applications; and any other suitable information or data. Subject information may include measurement results, data, files, other forms of medical history, such as, but not limited to, ultrasound images, ultrasound videos, and / or any imaging information related to the subject's anatomy. Subject information may include parameters related to the imaging procedure, such as the anatomical scanning window, probe orientation, and / or subject position during the imaging procedure. Memory 118 may also be configured to store information related to the training and implementation of machine learning algorithms (e.g., neural networks) and / or information related to the implementation of image recognition algorithms, image quantization algorithms, and / or image acquisition guidance algorithms (including those described herein) for detecting / segmenting anatomical structures.

[0045] In some instances, memory 118 includes data and stored instructions for modules, including time-slot planning support 126. Time-slot planning support 126 can utilize predicted durations and other data included in a user intervention planning request to plan and reserve medical facility resources for scheduling interventions. For example, time-slot planning support 126 can reserve rooms for specific time periods based on predicted intervention durations and also allocate staff and other medical professionals required for the intervention. Time-slot planning support 126 can create instructions and commands to provide rooms (e.g., operating rooms) with the necessary tools and resources for the intervention, as selected by interventional specialists from the output of intervention planner 150.

[0046] The network / cloud computer 140 may include a processor 142, an input device 144, a display 146, a communication interface 147, and a memory 148.

[0047] Processor 142 may include a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), controller, field-programmable gate array (FPGA) device, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 142 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 142 is configured to process instructions stored in memory 148. Processor 142 is connected to communication interface 147.

[0048] Input device 144 allows users to make selections or provide instructions to network / cloud computer 140. Input device can be a mouse, touch screen, touchpad, etc.

[0049] Display 146 is coupled to processor 142. Display 146 can be a monitor or any suitable display. Display 146 is configured to display information from intervention planner 150, data structured network 155, time slot planning support 126, databases 120, 122, 124, network performance, or any other system diagnostic information.

[0050] Communication interface 147 is coupled to processor 142. Communication interface 147 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. Communication interface 147 may include hardware and / or software components implementing a specific communication protocol suitable for transporting signals to other devices in networking system 100. Communication interface 147 may be referred to as a communication device or communication interface module.

[0051] Memory 148 is coupled to processor 142. Memory 148 can be any suitable storage device, such as cache memory (e.g., cache memory of processor 142), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory device, hard disk drive, solid-state drive, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0052] Memory 148 may be configured to store subject information, measurement results, data, or files, including the subject's medical history, history of procedures performed, anatomical or biometric features, characteristics, or medical conditions associated with the subject; computer-readable instructions, such as code, software, or other applications; and any other suitable information or data. Subject information may include measurement results, data, files, other forms of medical history, such as, but not limited to, ultrasound images, ultrasound videos, and / or any imaging information related to the subject's anatomy. Subject information may include parameters related to the imaging procedure, such as the anatomical scanning window, probe orientation, and / or subject position during the imaging procedure. Memory 148 may also be configured to store information related to the training and implementation of machine learning algorithms (e.g., neural networks) and / or information related to the implementation of image recognition algorithms, image quantization algorithms, and / or image acquisition guidance algorithms (including those described herein) for detecting anatomical structures, segmenting anatomical structures (e.g., defining the contours of anatomical structures).

[0053] In some instances, memory 148 includes data and instructions for storage in a module that includes an intervention planner 150 (including a prediction network 152) and a data structuring network 155. For example... Figure 1 As shown, intervention planner 150 and data structured network 155 reside on a network / cloud computer separate from databases 120, 122, 124 and time-slot planning support 126. During intervention planning, data from databases 120, 122, 124 can be sent to network / cloud computer 140. Intervention planner 150 can generate intervention predictions from the received data and then send them back to one or more computers and devices in networked system 100.

[0054] The prediction network 152 and the data structuring network 155 can be any neural network-based model or rule-based programming model, for example, as per [reference to...]. Figure 3 As described and as Figure 3 As described. For example, networks 152 and 155 can be convolutional neural networks (CNN), decision tree models, support vector machines (SVM), generative image-to-image transformers, generative image-to-text transformers (GIT), or other transformer-based models.

[0055] User computer 160 may include processor 162, input device 164, display 166, communication interface 167, and memory 168.

[0056] Processor 162 may include a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), controller, field-programmable gate array (FPGA) device, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 162 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 162 is configured to process instructions stored in memory 168. Processor 162 is connected to communication interface 167.

[0057] Input device 164 allows the user to make selections or provide instructions to the user computer 160. Input devices may include mice, touchscreens, touchpads, etc.

[0058] Display 166 is coupled to processor 162. Display 166 can be a monitor or any suitable display. Display 166 is configured to display intervention predictions from intervention planner 150, as described herein. Furthermore, display 166 can be configured to display images from data generated by medical imaging device 180.

[0059] Communication interface 167 is coupled to processor 162. Communication interface 167 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. Communication interface 167 may include hardware and / or software components implementing a specific communication protocol suitable for transporting signals to other devices in networking system 100. Communication interface 167 may be referred to as a communication device or communication interface module.

[0060] Memory 168 is coupled to processor 162. Memory 168 can be any suitable storage device, such as cache memory (e.g., cache memory of processor 162), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory device, hard disk drive, solid-state drive, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0061] Memory 168 may be configured to store subject information, measurement results, data, or files, including the subject's medical history, history of procedures performed, anatomical or biological characteristics, traits, or medical conditions associated with the subject; computer-readable instructions, such as code, software, or other applications; and any other suitable information or data. Subject information may include measurement results, data, files, other forms of medical history, such as, but not limited to, ultrasound images, ultrasound videos, and / or any imaging information related to the subject's anatomy. Subject information may include parameters related to the imaging procedure, such as the anatomical scanning window, probe orientation, and / or subject position during the imaging procedure. Memory 168 may also be configured to store information related to the training and implementation of machine learning algorithms (e.g., neural networks) and / or information related to the implementation of image recognition algorithms, image quantization algorithms, and / or image acquisition guidance algorithms (including those described herein) for detecting anatomical structures, segmenting anatomical structures.

[0062] The medical imaging console 170 may include a processor 172, an input device 174, a display 176, a communication interface 177, and a memory 178.

[0063] Processor 172 may include a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), controller, field-programmable gate array (FPGA) device, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 172 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 172 is configured to process instructions stored in memory 178. Processor 172 is connected to communication interface 177.

[0064] Input device 174 allows the user to make selections or provide commands to medical imaging console 170. Input device may be a mouse, touch screen, touchpad, etc.

[0065] Display 176 is coupled to processor 172. Display 176 can be a monitor or any suitable display. Display 166 is configured to display intervention predictions from intervention planner 150, as described herein. Furthermore, display 176 can be configured to display images from data generated by medical imaging device 180.

[0066] Communication interface 177 is coupled to processor 172. Communication interface 177 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. Communication interface 177 may include hardware and / or software components implementing a specific communication protocol suitable for transporting signals to other devices in networking system 100. Communication interface 177 may be referred to as a communication device or communication interface module.

[0067] Memory 178 is coupled to processor 172. Memory 178 can be any suitable storage device, such as cache memory (e.g., cache memory of processor 172), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory device, hard disk drive, solid-state drive, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0068] Memory 178 may be configured to store subject information, measurement results, data, or files, including the subject's medical history, the history of procedures performed, anatomical or biological characteristics, traits, or medical conditions associated with the subject; computer-readable instructions, such as code, software, or other applications; and any other suitable information or data. Subject information may include measurement results, data, files, other forms of medical history, such as, but not limited to, ultrasound images, ultrasound videos, and / or any imaging information related to the subject's anatomy. Subject information may include parameters related to the imaging procedure, such as the anatomical scanning window, probe orientation, and / or subject position during the imaging procedure. Memory 178 may also be configured to store information related to the training and implementation of machine learning algorithms (e.g., neural networks) and / or information related to the implementation of image recognition algorithms, image quantization algorithms, and / or image acquisition guidance algorithms (including those described herein) for detecting anatomical structures, segmenting anatomical structures.

[0069] Medical imaging device 180 can generate medical images 185. The medical imaging device communicates with medical imaging console 170, for example, via communication interface 177. Medical images generated from data from medical imaging device 180 can be displayed on display 176. Medical images 185 can also be transmitted to any of the following: computer, network / cloud 110, 140, 160.

[0070] In some aspects, aspects of this disclosure can be implemented using medical images 185 of a subject obtained using any suitable medical imaging device 180 and / or modality. Examples of medical images and medical imaging devices include X-ray images (angiographic images, fluoroscopic images, images with or without angiography) obtained by medical imaging devices such as ultrasound imaging devices and X-ray imaging devices; CT images obtained by computed tomography (CT) imaging devices; PET-CT images obtained by positron emission tomography-computed tomography (PET-CT) imaging devices; MRI images obtained by magnetic resonance imaging (MRI) imaging devices; SPECT images obtained by single-photon emission computed tomography (SPECT) imaging devices; OCT images obtained by optical coherence tomography (OCT) imaging devices; and IVPA images obtained by intravascular photoacoustic (IVPA) imaging devices. The medical imaging device 180 can acquire medical images when positioned outside the subject's body, spaced apart from the subject's body, adjacent to the subject's body, in contact with the subject's body, and / or inside the subject's body.

[0071] Training dataset 195 can be stored on network / cloud computer 110, network / cloud computer 140, or any other network or cloud computing environment. In some embodiments, the system described herein can be provided as a cloud service. Client sites can connect to the cloud service to provide training data and receive intervention planning services. The training dataset includes historical information, which may include past interventions, intervention outcomes, patient details, interventional specialist details, and tools, resources, and methods used for the intervention. Training dataset 195 may include both structured and / or unstructured data. As described herein, unstructured data may be (pre)processed before being used as input to a model or for comparison with the model's output.

[0072] It should be noted that the examples above are provided for illustrative purposes and are not intended to be limiting. Other devices and / or device configurations may be used to perform the operations described herein.

[0073] Figure 2 This is a schematic diagram of processor circuitry 250 according to aspects of this disclosure. Processor circuitry 250 can be implemented as needed in network / cloud computers 110, 140, user computers 160, imaging consoles 170, or other devices or workstations (e.g., third-party workstations, network routers, etc.), or on cloud processors or other remote processing units to implement the method. As shown, processor circuitry 250 may include processor 260, memory 264, and communication interface 268. These components can communicate directly or indirectly with each other, for example, via one or more buses.

[0074] Processor 260 may include any combination of a central processing unit (CPU), digital signal processor (DSP), ASIC, controller or general-purpose computing device, reduced instruction set computing (RISC) device, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other related logic devices, including mechanical and quantum computers. Processor 260 may also include another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 260 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0075] Memory 264 may include cache memory (e.g., the cache memory of processor 260), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory devices, hard disk drives, other forms of volatile and non-volatile memory, or combinations of different types of memory. In one aspect, memory 264 includes a non-transient computer-readable medium. Memory 264 may store instructions 266. Instructions 266 may include instructions that, when executed by processor 260, cause processor 260 to perform the operations described herein. Instructions 266 may also be referred to as code. The terms “instruction” and “code” should be interpreted broadly to include any type of computer-readable statement(s). For example, the terms “instruction” and “code” may refer to one or more programs, routines, subroutines, functions, flows, etc. “Instruction” and “code” may include a single computer-readable statement or a number of computer-readable statements.

[0076] Communication interface 268 may include any electronic circuitry and / or logic circuitry to facilitate direct or indirect data communication between processor circuitry 250 and other processors or devices. In this regard, communication interface 268 may be an input / output (I / O) device. In some instances, communication interface 268 facilitates direct or indirect communication between processor circuitry 250 and / or various components of the computer in network system 100. Communication interface 268 can communicate within processor circuitry 250 via a variety of methods or protocols. Serial communication protocols may include, but are not limited to, the US Serial Protocol Interface (US SPI), internal integrated circuits (I... 2C) Recommended standards RS-232, RS-485, Controller Area Network (CAN), Ethernet, ARINC 429, MODBUS, MIL-STD-1553, or any other suitable method or protocol. Parallel protocols include, but are not limited to, Industry Standard Architecture (ISA), Advanced Technology Attachment (ATA), Small Computer System Interface (SCSI), Peripheral Component Interconnect (PCI), IEEE 488, IEEE 1284, and other suitable protocols. Where appropriate, serial and parallel communication may be bridged by a Universal Asynchronous Receiver / Transmitter (UART), a Universal Synchronous Receiver / Transmitter (USART), or other suitable subsystems.

[0077] External communication (including, but not limited to, software updates, firmware updates, model sharing between the processor and the central server, or output generated by the model selection and prediction system described herein) can be implemented using any suitable wireless or wired communication technology, such as cable interfaces like Universal Serial Bus (USB), micro USB, Lightning or FireWire, Bluetooth, Wi-Fi, ZigBee, Li-Fi, or cellular data connections like 2G / GSM (Global System for Mobile Communications), 3G / UMTS (Universal Mobile Telecommunications System), 4G, LTE, WiMax, or 5G. For example, Bluetooth Low Energy (BLE) radios can be used to establish connections to cloud services, for data transfer, and for receiving software patches. The controller can be configured to communicate with remote servers or local devices such as laptops, tablets, or handheld devices, or may include a display capable of showing state variables and other information. Information can also be transferred on physical media such as USB flash drives or memory sticks.

[0078] Figure 3 This is a schematic diagram of a deep learning network configuration according to aspects of this disclosure. Configuration 300 can be implemented by a deep learning network. Configuration 300 includes a deep learning network 310, which may include one or more CNNs 312. CNN 312 is an example of a prediction model type (prediction network 152 or data structured network 155). For the sake of simplicity in illustration and discussion, Figure 3 The illustration shows a CNN 312. However, the embodiment can be scaled to include any suitable number of CNN 312s (e.g., about 2, 3 or more). Configuration 300 can be trained to recognize various anatomical structures (organs, tissues, bones) and / or other features (natural and / or artificial) within patient anatomy. Configuration 300 can be further trained for segmenting human anatomy, diagnosing medical conditions, or any number of other diagnostic or medical tasks.

[0079] CNN 312 may include a set of N convolutional layers 320, followed by a set of K fully connected layers 330, where N and K can be any positive integers. The convolutional layers 320 are shown as 320(1) to 320(N). The fully connected layers 330 are shown as 330(1) to 330(K). Each convolutional layer 320 may include a set of filters 322 configured to extract features from input 302 (e.g., an X-ray venogram image or other additional data). The values ​​N and K, as well as the sizes of the filters 322, may vary depending on the embodiment. In some instances, the convolutional layers 320(1) to 320(N) and the fully connected layers 330(1) to 330(K-1) may utilize a leaky rectified nonlinear (ReLU) activation function and / or batch normalization. The fully connected layers 330 may be nonlinear and may progressively reduce the high-dimensional output to the dimension of the prediction result 340 (e.g., the location of an object detection box or other classification output). The fully connected layer 330 may also be referred to as a classifier. In some embodiments, the fully convolutional layer 320 may also be referred to as a perceptual layer.

[0080] When the prediction result 340 is in the form of a classification output, it can indicate a confidence score (e.g., probability) for each category 342 based on the input image 302. Categories 342 are shown as 342a, 342b, ..., 342c. For example, when training CNN 312 on areas of stenosis or general venous compression, category 342 can indicate areas of inguinal ligament category 342a, crossing category 342b, pelvic notch category 342c, blood flow restriction category 342d, or any other suitable category. A category 342 indicating a high confidence score indicates that the input image 302, or a portion or pixel of image 302, may include anatomical objects / features of category 342. Conversely, a category 342 indicating a low confidence score indicates that the input image 302, or a portion or pixel of image 302, is unlikely to include anatomical objects / features of category 342.

[0081] CNN 312 can also output a feature vector 350 at the output of the last convolutional layer 320(N). The feature vector 350 can indicate objects detected from the input medical image 302 or other data. For example, the feature vector 350 can indicate regions associated with diseased tissue, such as those identified from image 302, which could be a patient medical image. The feature vector 350 can indicate pixels in the medical image associated with the location of the diseased tissue, as described herein.

[0082] The deep learning network 310 can be implemented or include any suitable type of learning network. For example, in some embodiments and as per [reference to...] Figure 3As described, the deep learning network 310 may include a convolutional neural network 312. Furthermore, the convolutional neural network 310 may additionally or alternatively be or include any suitable network or module that recognizes features within an image, such as a multi-class classification network, an encoder-decoder type network, a fully connected deep learning network, or a network or module that identifies features within an image.

[0083] In embodiments where the deep learning network 310 includes a fully connected neural network, the fully connected neural network can transform data unrelated to the medical image, or it can transform data derived from images (e.g., detected objects) generated by another network, program, or human annotator. For example, the fully connected neural network can transform information about a patient, such as age, weight, or other low-dimensional data.

[0084] In embodiments where the deep learning network 310 includes an encoder-decoder network, the network may include two paths. One path may be a shrinking path, where a large image, such as image 302, may be convolved by several convolutional layers 320 such that the size of image 302 changes relative to the depth of the network layers. Image 302 can then be represented in a low-dimensional space or a flattened space. From this flattened space, an additional path may expand the flattened space to the original size of image 302. In some embodiments, the implemented encoder-decoder network may also be referred to as a principal component analysis (PCA) method. In some embodiments, the encoder-decoder network may segment image 302 into patches. In additional embodiments of this disclosure, the deep learning network 310 may include a multi-class classification network. In such embodiments, the multi-class classification network may include an encoder path. For example, image 302 may be a high-dimensional image. Image 302 can then be processed with convolutional layers 320 to reduce its size. The resulting low-dimensional representation of image 302 can be used to generate Figure 3 The feature vector 350 is shown. The low-dimensional representation of image 302 can be additionally used by fully connected layer 330 to regress and output one or more categories 342. In some respects, fully connected layer 330 can process the output of encoder or convolutional layer 320. Fully connected layer 330 can also be referred to as task layer or regression layer, among other terms.

[0085] Any suitable combination or variation of the described deep learning network 310 is fully envisioned. For example, the deep learning network may include a fully convolutional network or layers or a fully connected network or layers or a combination of both. Furthermore, the deep learning network may include a multi-class classification network, an encoder-decoder network, or any combination of networks.

[0086] In some embodiments, the data structuring network 155 may include a Large Language Model (LLM). An LLM can be used to generate structured data from unstructured data. For example, the input to an LLM may be a large amount of text contained in a medical report written by a medical professional. The LLM can generate structured data from the unstructured medical report, including diagnoses, frequency of patient interactions, responses to treatments, etc. The LLM can have various network architectures, including autoencoders and transformer-based models.

[0087] Figure 4 This is a schematic diagram of at least a portion of an intervention planning system 400 according to aspects of this disclosure. The intervention planning system 400 generates one or more treatment plans (i.e., intervention predictions 425) and predicts the expected duration 420 of the treatment plans. User input (e.g., an intervention planning request 410) and selections may take the form of medical images, patient information, medical records, interventional goals, interventional specialist experience, medical resource availability (e.g., infrastructure, available scheduling times, staff resources, etc.), and the output may be a treatment plan 425 and the expected duration 420 of the intervention associated with the treatment plan. The user may be any of multiple medical professionals. In some instances, user selections may cause the system 400 to automatically collect additional information based on selections from various databases. The user may also limit the prediction of the treatment plan by interventional specialist experience. For example, the user may prefer to consider interventions from all interventional specialists, interventions from interventional specialists with similar experience to the user, or only interventions previously performed by the user. The intervention planning system 400 may include a data structured network 155, an intervention planner 150, and a database 415.

[0088] The interventional planning system 400 can receive an interventional planning request 410 from a user computer 160 or a medical imaging console 170. The interventional planning request 410 may include an interventional expert identifier and input 412, an interventional objective 414, and patient details 416. In some instances, the interventional expert identifier and input 412 may be used by an interventional expert regarding... Figure 11 A selection interface for description and further description is provided to allow interventional specialists to choose their preferred recommendations. In some instances, interventional goal 414 is a goal set by the interventional specialist, such as a 30% reduction in arterial blockage. In some instances, patient details may include patient identification information, enabling retrieval of patient-related medical records from one or more databases. Interventional planning requests can be structured using a data structuring network 155 and used as queries against database 415 to obtain additional patient, interventional specialist, and / or resource-related information.

[0089] The data structuring network 155 can receive the unstructured portion of an intervention planning request. The data structuring network 155 can include one or more neural networks suitable for structured data. The input to the data structuring network can be natural language, such as an intervention expert description of the intervention objective. The output of the data structuring network is structured data, such as diagnostics conforming to a standardized format. In some embodiments, the structured data output from the data structuring network takes the form of an n-dimensional vector, where the positions of entries in the n-dimensional vector correspond to different portions of the input context. For relatively simple contexts, the n-dimensional vector can have a low dimension, such as 1 to 10, while for more complex contexts (e.g., more diverse types of available input data), the n-dimensional vector can have a high dimension, such as greater than 10. As an example, the data structuring network 155 can be a Large Language Model (LLM). The term "Large Language Model" (LLM) can refer to a neural network-based deep learning system designed to understand and generate natural language. An LLM can have a transformer architecture, which typically includes a large number of parameters (weights in a neural network). For example, LLMs such as Generative Pretrained Transformer (GPT)3 have 17.5 billion parameters, and Text-to-Text Transformer (T5) has approximately 1.1 billion parameters.

[0090] Database 415 may include patient database 120, interventional specialist database 122, or resource database 124. Information included in interventional planning request 410 may be used to retrieve certain information from database 415. For example, patient details 416 may be used to retrieve all or part of a patient's medical history contained in patient database 120; interventional specialist identifiers and input 412 may be used to retrieve information about interventional specialists from interventional specialist database 122; or information from interventional planning request 410 may be used to retrieve desired information from resource database 124, such as specific medical facilities and staff, and their capabilities and experience. If unstructured data is included and retrieved from database 415 based on interventional planning request 410, then data structuring network 155 may be used to generate a structured form of data from database 415. For example, patient database 120 may include medical professional notes and / or medical reports (e.g., radiology reports) containing written language that can be processed by data structuring network 155 into a standard format.

[0091] Intervention planner 150 can receive structured data from intervention planning request 410 and database 415. Using prediction network 152, intervention planner 150 generates intervention prediction 425 and associated prediction duration 420 for the intervention prediction. In some embodiments, prediction network 152 is capable of receiving all inputs received by intervention planner 150 and producing a multi-parameter regression as output. In some embodiments, prediction network 152 may include a deep neural network comprising multiple network layers with full-to-full connections between nodes in sequential layers. In some embodiments, prediction network 152 may include one or more neural network models capable of receiving medical images, such as those described above. Figure 3 The CNN described. Besides neural networks, prediction networks can also be support vector machines, random forest models, or decision tree models.

[0092] Intervention prediction 425 may include one or more different possible interventions, such as different procedures using different equipment or using the same equipment in different ways. In some embodiments, intervention prediction 425 may take the form shown in Table 1 below. Table 1 shows intervention prediction 425 suggesting three possible intervention methods. For each method, the probability of needing a specific tool or resource is shown in the corresponding entry in the table. For example, method 1 has a 20% probability of needing tool 1, an 80% probability of needing tool 2, a 94% probability of needing resource 1, and a 50% probability of needing resource 2. Tools can be various medical devices, such as intravenous catheters, and resources can be various patient rooms (e.g., examination rooms or operating rooms) or various staff requirements (e.g., methods requiring an anesthesiologist). Information about tool requirements can also be enhanced by corresponding usage (e.g., lesion removal) and time (e.g., preparation for removal) or a more detailed timeline. Furthermore, information about the lower-level inferences of the probabilities generated by the prediction network can be provided to the user, thereby increasing acceptance of the suggested methods and tools(s). For example, the suitability of a particular interventional method can be determined primarily by the patient's advanced age, which prescribes a variety of alternative interventional methods.

[0093] Intervention prediction 425 may include any number of methods, tools, and resources suitable for the context and intervention planning request 410. For example, only one method may be available for the intervention, and therefore only one method will be predicted. However, for interventions in which many procedures are possible, a large number of methods may be suggested. In some embodiments, sub-steps of a method may be listed in intervention prediction 425. For example, a stent placement procedure may require post-placement imaging, which may be listed as a sub-step. In some embodiments, a criticality index may be included in intervention prediction 425. A criticality index reflects the degree of criticality of a resource to a method. In other words, a high criticality index for a tool may indicate that the absence of the tool has a significant impact on the outcome of the intervention. In some embodiments, a criticality index may be generated for each tool or resource for different sub-steps / stages of the intervention. The probability of using a resource in conjunction with the criticality index can be used to prioritize or de-prioritize these critical resources in the hospital scheduling system. Table 1

[0094] The predicted duration 420 of the intervention can be an estimate of the time required to complete the intervention, such as minutes or hours and minutes. The time-slot planning support 126 can use the predicted duration 420 to reserve and schedule equipment and resources for interventions at medical facilities, such as hospitals. In some embodiments, the estimate of the intervention duration may also depend on the selected method, and may further include variations depending on the expertise of the intervention specialist. In this case, separate estimates are output, one for each possible method. In some embodiments, the predicted duration 420 may also include an estimate of the variance (e.g., error bars) of the planned intervention duration. In some embodiments, the predicted duration may include an estimate of the time and / or associated variance (e.g., error bars) of the time for sub-steps / stages of the intervention.

[0095] In some embodiments, the proposed tools and resources can be compared with the availability of tools and resources in the hospital. If the required tools or resources are unavailable, the suitability of the corresponding methods can be adjusted accordingly, or the intervention can be rescheduled if the resources are only temporarily unavailable.

[0096] Intervention prediction 425 or prediction duration 420 can be provided to the user computer 160 or medical imaging console 170. (See also: ...) Figure 14Further described, intervention prediction 425 can be provided to intervention experts via a user interface, allowing them to choose among recommended intervention methods. The intervention expert's selection and the predicted duration can be provided to time slot planning support 126. Time slot planning support 126 can be used by the organization to plan and reserve all necessary infrastructure and resources for intervention.

[0097] Interventional specialists can provide feedback 430 regarding interventional prediction 425 via user computer 160 or medical imaging console 170. In some embodiments, feedback 430 may be a quality indicator reflecting the interventional specialist's evaluation of the recommended intervention. Furthermore, feedback 430 may provide the extent to which interventional outcomes are included, to the degree that they can be determined after the intervention is completed. In some embodiments, the feedback loop may be an automated feedback loop that collects information about tools, resources, and duration from an IT system; or the feedback loop may be a manual feedback loop that obtains feedback about the intervention from the interventional specialist after the procedure; or it may be a combination of both. In some embodiments, for example, such as Figure 12 As shown, the checklist can include "Ready" and "Not Required" options for each element. The selections in the checklist can be logged when the suggested tools and resources are actually used during the intervention / process. The checklist can also include options for manually adding needed tools or resources that the system has not yet predicted. In some instances, tool and / or resource usage can be automatically logged using known device / motion tracking techniques, including object detection using CNNs.

[0098] Figure 5 This is a schematic sequence diagram of the process for an interventional planning system 400 according to aspects of this disclosure. The interventional planning system may include a time-slot planning support 126, a user computer 160 or a medical imaging console 170, a data structured network 155, an interventional planner 150, and a database 415.

[0099] At step 502, the data structured network 155 may receive an intervention planning request from the user computer 160 or the medical imaging console 170, and may query the database 415 for information associated with the intervention planning request. Database 415 may include a patient database 120, an interventional expert database 122, or a resource database 124 as described herein.

[0100] At step 504, data associated with the intervention planning request is retrieved from database 415 and provided to data structured network 155.

[0101] In step 506, the data structuring network 155 may (pre)process unstructured data contained in the intervention planning request and associated with the intervention planning request from the database 415. In some embodiments, the data from the database 415 may be structured and therefore does not need to be structured by the data structuring network 155.

[0102] At step 508, the structured intervention planning request and structured data from the database are provided to the intervention planner 150.

[0103] At step 510, intervention planner 150 generates a treatment plan and predicts the duration. In some embodiments, request 410 may be received at intervention planner 150 without any intermediate structuring by data structuring network 155.

[0104] At step 512, the treatment plan and predicted duration are provided to the user computer 160 or medical imaging console 170 and time slot planning support 126.

[0105] In step 514, the treatment plan is output to the display for user evaluation and selection.

[0106] In step 516, user selection is provided to slot planning support 126.

[0107] In step 518, time slot planning supports scheduling interventions. As described herein, scheduling interventions may include reserving various infrastructure (e.g., operating rooms), resources (e.g., surgical instruments), and staff.

[0108] At step 520, the interventional specialist can provide feedback to the interventional planner 150 through interaction with the user computer 160 or the medical imaging console 170.

[0109] Those skilled in the art will understand that the steps in the process can be performed individually or simultaneously and in other arrangements.

[0110] Figure 6This is a schematic diagram of at least a portion of a training system 600 for predicting network 152 according to aspects of this disclosure. Training system 600 trains predictive network 152 to accurately predict interventions and the duration of those interventions from the context provided at the time of inference. Predictive network 152 is trained by providing historical information on the selection of methods, tools, and resources, as well as the duration of interventions and corresponding input features described in intervention planning system 400 (e.g., the content of intervention planning request 410 and data retrieved from database 415). Information regarding tools and methods can be extracted from report documents or can be manually recorded after each intervention. Information regarding the duration of interventions can be extracted from the hospital IT system or can be manually recorded after each intervention. Training system 600 includes a data structured network 155, an intervention planner 150, and a model objective / function 615.

[0111] Training data 602 may be included in training dataset 195. Training data 602 may include unstructured training data 604, structured training data 606, and baseline ground truth labels and target predictions 608. Unstructured data 604 may take the form of natural language from medical reports, interventional objectives, and other sources as described herein. Furthermore, unstructured data may include images with or without annotations or classifiers. Structured data may include patient details such as gender, weight, height, medications, etc. Baseline ground truth labels and predictions 608 may include the preferred method for intervention in a given context, the outcome of the preferred intervention, and / or the actual duration of the intervention.

[0112] The data structuring network 155 can receive training data 602 and generate structured training data. The data structuring network 155 may include one or more neural networks suitable for structured and unstructured data, for example, as per [reference to...]. Figure 3The input to a data structured network can be natural language, such as an interventional expert description of the intervention target. The output of the data structured network is structured data, such as a diagnosis conforming to a standardized format. In some embodiments, the structured data output from the data structured network takes the form of an n-dimensional vector, where the positions of entries in the n-dimensional vector correspond to different parts of the input context. For relatively simple contexts, the n-dimensional vector can have a low dimension, such as 1 to 10, while for more complex contexts (e.g., more types of available input data), the n-dimensional vector can have a high dimension, such as greater than 10. As an example, the data structured network 155 can be a Large Language Model (LLM), the term "Large Language Model" (LLM) can refer to a neural network-based deep learning system designed to understand and generate natural language. LLMs can have a transformer architecture, which typically includes a large number of parameters (weights in a neural network). For example, an LLM such as the Generative Pre-trained Transformer (GPT) 3 has 17.5 billion parameters, and a Text-to-Text Transformer (T5) has approximately 1.1 billion parameters.

[0113] Intervention planner 150 can receive structured training data from data structured network 155. Intervention planner 150 includes prediction network 152, which is similar to the prediction network 155. Figure 3 and Figure 4 The prediction network is described. Using the prediction network 152, the intervention planner 150 generates intervention predictions and duration predictions 610, which are similar in structure and content to those described in [the previous section]. Figure 4 The predicted duration 420 and the intervention prediction 425 are described.

[0114] Using model objective / function 615, the training system 600 compares the intervention predictions and duration predictions 610 with the associated baseline ground truth labels and predictions 608. Model objective / function 615 may include an objective / function that penalizes the predicted durations to a greater or lesser extent, as these predicted durations are further or more closely approximated to the baseline ground truth labels and predictions, respectively. Similarly, model objective / function 615 may include an objective / function that penalizes the selection of an incorrect tool or resource with a higher probability than the baseline ground truth predictions. Model objective / function 615 may include an objective / function that penalizes predictions based on incorrect ranking of the method. Losses may be selected based on the task and / or data type. Some examples of losses may be L1 and L2 losses, mean squared error losses, cross-entropy losses, Kullback Leibler divergence losses, etc.

[0115] The comparison from the model objective / function 615 can update the parameters 620 of the prediction network 152. In some instances, the update can be accomplished using the gradient of the objective function and backpropagation to update the parameters of the prediction network 152.

[0116] In some embodiments, training and / or fine-tuning can be performed on newly introduced or experimental devices (e.g., within clinical trials). For example, a provider of a newly approved device may make data from testing and trials of the device available for model training, thereby introducing the device into a (potential) workflow.

[0117] Figure 7 This is a schematic sequence diagram of the process for training a prediction network 600 according to aspects of this disclosure. The system for training the prediction network may include a training dataset 195, a data structured network 155, an intervention planner 150, and a model objective / function 615.

[0118] In step 702, training data from training dataset 195 is provided to data structured network 155. In some embodiments, the training data may include benchmark ground truth labels or target predictions.

[0119] At step 704, the data structuring network 155 (pre)processes the unstructured training data. In some instances, the training data can be structured and does not need to be processed by the data structuring network.

[0120] In step 706, structured training data 706 is provided to the intervention planner 150.

[0121] At step 708, the intervention planner 150 generates a prediction. In some embodiments, the prediction may include a treatment plan and the expected duration of the treatment plan.

[0122] At step 710, the predicted and baseline true labels or target predictions are provided to the model objective / function 615.

[0123] At step 712, the model objective / function 615 is used to compare the prediction with the baseline true label or the target prediction.

[0124] At step 714, a comparison between the prediction and the baseline true value label or the target prediction is provided to the intervention planner 150.

[0125] At step 716, the intervention planner 150 may update the network parameters based on comparisons. In some embodiments, the update involves modifying the parameters of the prediction network included in the intervention planner 150 using backpropagation of the model objective / function gradient.

[0126] Those skilled in the art will understand that the steps in the process can be performed individually or simultaneously and in other arrangements.

[0127] Figure 8-10This is a schematic diagram of various databases. Each database can share common data with each other. For example, an interventional specialist may have performed interventions on a patient in the past, and this information can be reflected in both the patient database 120 and the interventional specialist database 122. Each item in these databases can be associated with a scheduling system to allow for the efficient allocation of infrastructure, equipment, staff, and resources, thereby avoiding double booking.

[0128] Figure 8 This is a schematic diagram of a patient database 120 according to aspects of this disclosure. The patient database 120 may include patient details 810, medical images 820, and medical reports 830. Patient details 810 may be information influencing the methods or tools used for the intervention, and may include gender, weight, allergies, age, health status, special requirements, medications, and results from previous interventions. Medical images 820 and medical reports 830 may include laboratory reports, medical device data (e.g., ECG, EEG), and any previous medical images or data containing information that may influence how the intervention was performed. In some instances, such information may be found in electronic medical records (EMR) and electronic health records (HER). Images may also be enhanced by radiological reports of the medical images.

[0129] Figure 9 This is a schematic diagram of an interventional expert database 122 according to aspects of this disclosure. The interventional expert database 122 may include one or more interventional experts (including those who can be placed as described above). Figure 4 The described intervention planning request is a collection of past procedures 910 and intervention expert details 920 performed by the intervention experts. Past procedures 910 can be a structured or unstructured list of historical interventions performed by the intervention experts, including outcome resources used and other information associated with the intervention. Intervention expert details 920 can include information such as each intervention expert's education, experience, expertise, and preferences.

[0130] Figure 10This is a schematic diagram of a resource database 124 according to aspects of this disclosure. Resource database 124 may include facility infrastructure 1010, equipment 1020, and staff resources 1030. Facility infrastructure 1010 may include a list of rooms and associated capabilities of one or more medical facilities. For example, an operating room will have different capabilities compared to an examination room. Many other room types reflecting different specialized capabilities may exist. Equipment 1020 may include any number of devices, such as catheters, EKG, CT, MRI, etc. Equipment 1020 includes tools that may include portions of the output of the interventional planning system 400. Staff resources 1030 may include a list of staff with different specialties and experiences. For example, some interventions may require minimal staff support beyond interventional specialists, while surgeries may require multiple staff with different specialties and experiences.

[0131] Figure 11 This is an example display 1100 for an intervention planning request according to aspects of this disclosure. Example display 1100 may be a user interface provided to an interventional specialist via a display at a user computer 160 or medical imaging console 170. Display 1100 may include prompts for the interventional specialist to identify the patient 1105, the intervention target 1110, and the interventional specialist 1115. Patient slot 1105 may receive a patient identifier (e.g., by a unique identifier or name). Intervention target 1110 may be a natural language description provided by the interventional specialist, describing the patient's diagnosis and the goal of the intervention. In some embodiments, this will be used by the intervention planning system to set the context for the planned intervention. Interventional specialist slot 1115 may receive an interventional specialist identifier (e.g., a unique alphanumeric identifier or name). Interventional specialist slot 1115 may be automatically populated based on credentials used by the interventional specialist to log in to the user computer 160 or medical imaging console 170.

[0132] In addition, the interventional planner can be prompted to select an experience domain, within which the intervention planning system can generate interventions, as reflected in the "Select Suggestion Preference" sub-window 1120. The probabilities generated by the intervention planning system 400 can depend on the interventional expert profiles, as they may have different preferences or expertise. The intervention planning system 400 can output separate sets of probabilities for: (1) the interventional expert scheduled to perform the procedure (i.e., based on status or training data from the interventional expert); (2) the interventional expert's colleagues scheduled to perform the procedure (i.e., based on status or training data from a group of interventional experts with similar backgrounds, experience, and preferences); or (3) all interventional experts (i.e., based on the complete training dataset).

[0133] Figure 12This is a schematic diagram of a sequence of interfaces based on the prediction of an intervention planning system according to aspects of this disclosure. Intervention prediction 425 is used, as shown in Table 1 and... Figure 12 The example reproduced in the image can generate a user selection interface 1205 presented to the interventional specialist on the display of the user computer 160 or medical imaging console 170. The user selection interface 1205 includes a structured presentation of the intervention prediction 425. In addition to the percentages shown for each tool and resource, the user selection interface 1205 also includes suitability percentages 1207-1209. Suitability percentages 1207-1209 indicate how suitable the method is for the planned intervention. For example, method 1 has a maximum suitability percentage of 90%. In some embodiments, a criticality index may also be included in the probability for each tool and resource.

[0134] After the interventional expert selects one of the presented methods in the user selection interface 1205, a readiness checklist 1210 can be generated. The readiness checklist 1210 includes a list of tools and resources associated with the selected method. The checklist 1210 may have boxes for marking whether a particular tool or resource is ready. In some embodiments, the checklist 1210 may be used to mark whether a tool or resource was used during the intervention. The completed checklist can be used to provide feedback to the intervention planning system 400 and update the predictive network, as described herein.

[0135] Figure 13 This is a schematic diagram of a time-slot planning support system 1300 according to aspects of this disclosure. The time-slot planning support system 1300 allows medical facilities or other healthcare organizations to plan interventions and reserve facility space, such as operating rooms. The time-slot planning support system 1300 includes a time-slot planning support 126. The time-slot planning support 126 can receive an estimated duration 420 and generate infrastructure and staff reservations 1305. Infrastructure and staff reservations 1305 can include using the medical facility's IT system to reserve one or more rooms and assign one or more staff members to assist in performing an intervention in that room at the scheduled time. In some embodiments, the time-slot planning support 126 can also receive predicted interventions. The time-slot planning support 126 can utilize predictions of the likelihood of using various tools and resources to ensure that tools are available when an intervention occurs in the correct room.

[0136] Figure 14 This is a schematic flowchart of method 1400 for intervention planning according to aspects of this disclosure. It should be understood that the steps of method 1400 can be... Figure 14The different sequences of execution shown may provide additional steps before, during, and after the steps, and / or may otherwise replace or eliminate some of the described steps. One or more of the steps of method 1400 may be performed by one or more devices and / or systems described herein, such as components of networking system 100 and / or processor circuitry 250.

[0137] In step 1402, method 1400 includes receiving first user input from a user input device that identifies a medical professional. For example, a user enters an interventional expert identifier via user computer 160 in response to prompt 1115. In some embodiments, method 1400 also includes receiving a selection of a subset of medical professionals from the user input device (e.g., Figure 11 (1120 in the middle).

[0138] In step 1404, method 1400 includes retrieving first data representing a medical professional from one or more databases in response to first user input. For example, interventional expert details from interventional expert database 122. In some embodiments, the first data includes resume information about the medical professional (e.g., stored in interventional expert database 122).

[0139] In step 1406, method 1400 includes receiving second user input identifying the patient from a user input device. For example, the user inputs prompt 1105 via user computer 160.

[0140] In step 1408, method 1400 includes retrieving second data representing a patient from one or more databases in response to second user input. For example, patient information from patient database 120. In some embodiments, the second data includes the patient's medical records (e.g., stored in patient database 120).

[0141] In step 1410, method 1400 includes retrieving third data associated with a medical facility at which medical professionals can perform treatment procedures on patients from one or more databases. For example, resource information from resource database 124. In some embodiments, the third data includes at least one of the available medical tools or resources at the medical facility (e.g., stored in resource database 124).

[0142] In step 1412, method 1400 includes preprocessing the first data, the second data, and the third data to form a structured dataset. For example, the intervention planning request 410 and the data from database 415 are preprocessed using a data structuring network 155.

[0143] In step 1414, method 1400 includes providing a structured dataset as input to a prediction network. For example, the output of data structuring network 155 is provided to intervention planner 150, which includes prediction network 152. In some embodiments, method 1400 further includes restricting the structured data to data associated with a subset of medical professionals (e.g., based on the user's choices in 1120).

[0144] In step 1416, method 1400 includes generating a process plan for a treatment procedure as the output of a prediction network, wherein the process plan includes at least one of medical tools or medical resources to be used by a medical professional to perform the treatment procedure. For example, intervention planner 150 outputs intervention prediction 425. In some embodiments, method 1400 may further include generating a predicted duration 420 of the treatment procedure using prediction network 152. In some instances, process plan 425 may include a series of sub-steps, and predicted duration 420 may include multiple sub-step durations, each sub-step duration being associated with each sub-step in the process plan. In some embodiments, process plan 425 includes multiple probabilities, each of which is associated with a medical tool or medical resource (e.g., probabilities depicted in Table 1), wherein each probability represents the likelihood of using a medical tool or medical resource in the treatment procedure. In some embodiments, process plan 425 includes one or more key indices, each key index being associated with a medical tool or medical resource, wherein each key index represents the importance of the medical tool or medical resource in the outcome of the treatment procedure.

[0145] In step 1418, method 1400 includes outputting a screen display to a monitor that includes at least one of a medical tool or medical resource. For example, monitor 166 on user computer 160 displays a user selection interface 1205.

[0146] Figure 15 This is a schematic flowchart of a method 1500 for training a prediction network 152 in an intervention planning system, according to aspects of this disclosure. It should be understood that the steps of method 1500 can be... Figure 15 The different sequences of execution shown may provide additional steps before, during, and after the steps, and / or may otherwise replace or eliminate some of the described steps. One or more of the steps of method 1500 may be performed by one or more devices and / or systems described herein, such as components of networking system 100 and / or processor circuitry 250.

[0147] In step 1502, method 1500 includes retrieving training data from one or more databases, the training data including the inputs to the prediction network and the baseline ground truth outputs (intervention plans and / or durations). For example, training data 602 is retrieved from training dataset 195.

[0148] In step 1504, method 1500 includes preprocessing the training data into structured training data. For example, training data 602 is preprocessed using a data structured network 155.

[0149] In step 1506, method 1500 includes providing structured training data as input to a prediction network. For example, the output of data structured network 155 is provided to intervention planner 150, which includes prediction network 152.

[0150] In step 1508, method 1500 includes generating a procedure plan and expected duration for a treatment process as outputs of a prediction network, wherein the procedure plan includes at least one of medical tools or medical resources to be used by a medical professional to perform the treatment process. For example, intervention planner 150, including prediction network 152, generates intervention prediction and duration prediction 610.

[0151] In step 1510, method 1500 includes comparing the process schedule and expected duration with corresponding baseline truth outputs using an objective function and / or loss. For example, model objective / function 615 compares intervention predictions and duration predictions 610 with associated baseline truth labels and predictions 608.

[0152] In step 1512, method 1500 includes updating the prediction network based on comparisons. For example, the parameters 620 of the prediction network 152 are updated using comparisons from the model objective / function 615.

[0153] Furthermore, the techniques disclosed herein are also applicable to other medical imaging modalities in which 3D data is available from medical imaging equipment, such as other ultrasound applications, camera-based video, X-ray video, and 3D volumetric images, such as computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, optical coherence tomography (OCT) scans, or intravenous ultrasound (IVUS) pull-back sequences. The techniques described herein can be used in a variety of settings, including emergency rooms, intensive care units, inpatient settings, and outpatient settings.

[0154] Therefore, the logical operations constituting the various aspects of the technology described herein are referred to differently as operations, steps, objects, layers, elements, components, algorithms, or modules. Furthermore, it should be understood that these can occur, be performed, or be arranged in any order unless otherwise expressly required or the language of the claims inherently necessitates a specific order.

[0155] All directional references (e.g., up, down, inside, outside, upward, downward, left, right, side, front, back, top, bottom, above, below, vertical, horizontal, clockwise, counterclockwise, near, and far) are used for identification purposes only to aid the reader's understanding of the claimed subject matter and do not impose limitations. Unless otherwise stated, connection references (e.g., attachment, coupling, connection, combination, or "communication") should be interpreted broadly and may include intermediate members between sets of elements and relative movement between elements. Therefore, a connection reference does not necessarily imply that two elements are directly connected and fixed to each other. The term "or" should be interpreted as "and / or" rather than "exclusive OR". The word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude multiple. Unless otherwise stated in the claims, the values ​​stated should be interpreted as illustrative only and should not be considered restrictive.

[0156] The foregoing specification, examples, and data provide a complete description of the structure and use of exemplary aspects of the intervention planning system disclosed herein. Although various aspects of the claimed subject matter have been described above with a degree of specificity or reference to one or more individual aspects, those skilled in the art can make many changes to the disclosed aspects without departing from the spirit or scope of the claimed subject matter.

[0157] Other aspects are also envisioned. It is intended that all content contained in the above description and shown in the accompanying drawings be interpreted as illustrative of specific aspects and not as limiting. Changes in detail or structure may be made without departing from the essential elements of the subject matter as defined in the appended claims.

Claims

1. A computer-implemented method, comprising: Receive (502) first user input from the user input device (164) that identifies the medical professional (1115); In response to the first user input, retrieve (504) first data representing the medical professional from one or more databases (415); Receive (502) a second user input identifying the patient (1105) from the user input device (164); In response to the second user input, second data representing the patient is retrieved (504) from the one or more databases (415); Retrieve (504) third data associated with a medical facility from one or more databases (415), at which the medical professional can perform treatment procedures on the patient; The first data, the second data, and the third data are preprocessed (506) to form a structured dataset; The structured dataset is provided as input (508) to the prediction network (152); Generate (510) a process plan (425) for the treatment procedure as the output of the prediction network, wherein the process plan includes at least one of medical tools or medical resources to be used by the medical professional to perform the treatment procedure; as well as Output (514) to the display (166) a screen display (1205) including at least one of the medical tools or the medical resources.

2. The method according to claim 1, further comprising: Generate (510) the predicted duration (420) of the treatment process as the output of the prediction network (152).

3. The method according to claim 2, in, The process plan (425) includes a series of sub-steps, and The predicted duration (420) includes multiple sub-step durations, with one sub-step duration for each sub-step in the process plan.

4. The method according to claim 1, wherein, The process plan (425) includes a plurality of probabilities, each of which is associated with the medical tool or the medical resource, wherein each probability represents the likelihood of using the medical tool or the medical resource in the treatment process.

5. The method according to claim 4, wherein, The process plan (425) includes one or more key indices, each key index being associated with the medical tool or the medical resource, wherein each key index represents the importance of the medical tool or the medical resource in the outcome of the treatment process.

6. The method according to claim 1, wherein, Receiving (502) the first user input from the user input device (164) to identify the medical professional (1120) also includes: Receive (502) a selection of a subset of medical professionals (1120) from the user input device (164), and The structured data is limited to data associated with a subset of the medical professionals.

7. The method according to claim 1, wherein, The large language model (155) preprocesses the first data, the second data and the third data to form a structured dataset.

8. The method of claim 1, in, The first data includes resume information (920) regarding the medical professional. The second data includes the patient's (12) medical records, and The third data includes at least one of the available medical tools or available medical resources (124) at the medical facility.

9. A system (400) comprising: The processor circuit (250) is configured as follows: Receive (502) first user input from the user input device (164) that identifies the medical professional (1115); In response to the first user input, retrieve (504) first data representing the medical professional from one or more databases (415); Receive (502) a second user input identifying the patient (1105) from the user input device (164); In response to the second user input, second data representing the patient is retrieved (504) from the one or more databases (415); Retrieve (504) third data associated with a medical facility from one or more databases (415), at which the medical professional can perform treatment procedures on the patient; The first data, the second data, and the third data are preprocessed (506) to form a structured dataset; The structured dataset is provided as input (508) to the prediction network (152); Generate (510) a process plan (425) for the treatment procedure as the output of the prediction network, wherein the process plan includes at least one of medical tools or medical resources to be used by the medical professional to perform the treatment procedure; as well as Output (514) to the display (166) a screen display (1205) including at least one of the medical tools or the medical resources.

10. The system according to claim 9, wherein, The processor circuit is also configured to: Generate (510) the predicted duration (420) of the treatment process as the output of the prediction network (152).

11. The system according to claim 10, in, The process plan (425) includes a series of sub-steps, and The predicted duration (420) includes multiple sub-step durations, with one sub-step duration for each sub-step in the process plan.

12. The system according to claim 9, wherein, The process plan (425) includes a plurality of probabilities, each of which is associated with the medical tool or the medical resource, wherein each probability represents the likelihood of using the medical tool or the medical resource in the treatment process.

13. The system according to claim 12, wherein, The process plan (425) includes one or more key indices, each key index being associated with the medical tool or the medical resource, wherein each key index represents the importance of the medical tool or the medical resource in the outcome of the treatment process.

14. The system according to claim 9, wherein, The processor circuit is also configured to: Receive (502) a selection of a subset of medical professionals (1120) from the user input device (164), and The structured data is limited to data associated with a subset of the medical professionals.

15. The system according to claim 9, wherein, The large language model (155) preprocesses the first data, the second data and the third data to form a structured dataset.

16. The system according to claim 9, in, The first data includes resume information (920) regarding the medical professional. The second data includes the patient's medical records (120), and The third data includes at least one of the available medical tools or available medical resources at the medical facility.

17. A non-transient machine-readable medium comprising a plurality of machine-executable instructions, said plurality of machine-executable instructions, when executed by one or more processors, being adapted to cause said one or more processors to perform operations including: Receive (502) first user input from the user input device (164) that identifies the medical professional (1115); In response to the first user input, retrieve (504) first data representing the medical professional from one or more databases (415); Receive (502) a second user input identifying the patient (1105) from the user input device (164); In response to the second user input, second data representing the patient is retrieved (504) from the one or more databases (415); Retrieve (504) third data associated with a medical facility from one or more databases (415), at which the medical professional can perform treatment procedures on the patient; The first data, the second data, and the third data are preprocessed (506) to form a structured dataset; The structured dataset is provided as input (508) to the prediction network (152); Generate (510) a process plan (425) for the treatment procedure as the output of the prediction network, wherein the process plan includes at least one of medical tools or medical resources to be used by the medical professional to perform the treatment procedure; as well as Output (514) to the display (166) a screen display (1205) including at least one of the medical tools or the medical resources.

18. The non-transient machine-readable medium according to claim 17, wherein, It also causes the one or more processors to perform operations including the following: Generate (510) the predicted duration (420) of the treatment process as the output of the prediction network (152).

19. The non-transient machine-readable medium according to claim 18, in, The process plan (425) includes a series of sub-steps, and The predicted duration (420) includes multiple sub-step durations, with one sub-step duration for each sub-step in the process plan.

20. The non-transient machine-readable medium according to claim 17, wherein, The process plan (425) includes a plurality of probabilities, each of which is associated with the medical tool or the medical resource, wherein each probability represents the likelihood of using the medical tool or the medical resource in the treatment process.

Images