System, method, and computer program product for machine learning based medical procedure planning

Abstract

Systems for machine learning based medical procedure planning may include at least one processor to: receive patient scan data associated with a body of a patient, wherein the patient scan data includes data associated with a plurality of medical images of the body of the patient, wherein the data associated with the plurality of medical images includes one or more locations on the body of the patient marked as a target site; automatically generate a map of a surface of the body of the patient; determine one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site; and generate data associated with a medical procedure plan based on the one or more locations on the surface map of the body. Methods and computer program products are also disclosed.

Description

SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT FOR MACHINE LEARNING BASED MEDICAL PROCEDURE PLANNINGBACKGROUND1. Field

[0001] This disclosure relates generally to systems, methods, and / or devices that are used in robot-assisted medical procedures and, in some non-limiting embodiments, to systems, methods, and computer program products for machine learning based medical procedure planning, including patient location identification.2. Technical Considerations

[0002] Robotically-assisted surgery (e.g., robotic surgery, robot-assisted surgery, etc.) may refer to types of surgical procedures that are performed using robotic systems during a medical procedure. Robotically-assisted surgery may have been developed to overcome limitations of pre-existing minimally-invasive surgical procedures and to enhance the capabilities of surgeons performing open surgery. In examples of robotically-assisted minimally-invasive surgery, instead of a surgeon directly moving surgical instruments, the surgeon may use one of two methods to perform dissection, hemostasis, and / or resection, using a direct tele-manipulator and / or other form of computer control.

[0003] Artificial intelligence (Al) may be used in healthcare as a way to mimic human cognition in analysis, presentation, and / or comprehension of healthcare data. Al may describe the ability of computer programs, such as computer algorithms, to approximate conclusions based on input data, which may be medical data. In some instances, computer algorithms can be used to recognize patterns in data and create logic for identifying such patterns. Such computer algorithms may include machine learning models that are trained to perform certain tasks using extensive amounts of input data.

[0004] However, during robotically-assisted surgery procedures, an operator of the robotic system, such as a surgeon, may not be able to recognize the complexity of having to locate an internal structure (e.g., a target structure, a structure to avoid, etc.) that is internal to the body of the patient, either from personal experience or from external landmarks on the body of the patient. Furthermore, there is an opportunity for the operator to mistake a location for a surgical procedure on the body of a patient. For example, a surgeon may fail to precisely account for a situation where the internalorgans and / or other internal structures in the body of a patient are not in the same location for all patients and can move based on a position of the patient.SUMMARY

[0005] Accordingly, provided are systems, methods, and computer program products for machine learning based medical procedure planning.

[0006] Further non-limiting embodiments or aspects are set forth in the following numbered clauses:

[0007] Clause 1 : A system for machine learning based medical procedure planning, comprising: at least one processor configured to: receive patient scan data associated with a body of a patient, wherein the patient scan data comprises data associated with a plurality of medical images of the body of the patient, wherein the data associated with the plurality of medical images of the body of the patient comprises one or more locations on the body of the patient marked as a target site; automatically generate a surface map of the body of the patient based on the patient scan data; determine one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site; and generate data associated with a medical procedure plan for a medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient.

[0008] Clause 2: The system of clause 1 , wherein the at least one processor is further configured to: perform the medical procedure on the patient using a robotic surgical tool based on the data associated with the medical procedure plan.

[0009] Clause 3: The system of clauses 1 or 2, wherein, when performing the medical procedure on the patient using the robotic surgical tool, the at least one processor is configured to: perform the medical procedure on the patient using a robotically controlled surgical arm.

[0010] Clause 4: The system of any of clauses 1 -3, wherein the at least one processor is further configured to: determine a tissue type of the one or more locations on the surface map of the body of the patient; and wherein, when generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient, the at least one processor is configured to: generate the data associated with the medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body ofthe patient and the tissue type associated with the one or more locations on the surface map of the body of the patient.

[0011] Clause 5: The system of any of clauses 1 -4, wherein the at least one processor is further configured to: identify one or more locations of landmarks on the body of the patient in relation to the one or more locations on the body of the patient marked as a target site; wherein, when generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient, the at least one processor is configured to: generate the data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient, the tissue type associated with the one or more locations on the surface map of the body of the patient, and the one or more locations of landmarks on the body of the patient.

[0012] Clause 6: The system of any of clauses 1 -5, wherein, when receiving the patient scan data associated with the body of the patient, the at least one processor is configured to: receive the patient scan data from a computerized tomography (CT) scan device or a magnetic resonance imaging (MRI) device.

[0013] Clause 7: The system of any of clauses 1-6, wherein, when automatically generating the surface map of the body of the patient, the at least one processor is configured to: automatically generate a three-dimensional (3D) surface map of the body of the patient; and wherein, when determining the one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site, the at least one processor is configured to: determine 3D coordinates for each of the one or more locations on the 3D surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site.

[0014] Clause 8: The system of any of clauses 1-7, further comprising: a robotic surgical tool; and wherein the at least one processor is coupled to the robotic surgical tool.

[0015] Clause 9: A method for machine learning based medical procedure planning comprising: receiving, with at least one processor, patient scan data associated with a body of a patient, wherein the patient scan data comprises data associated with a plurality of medical images of the body of the patient, wherein the data associated with the plurality of medical images of the body of the patient comprises one or more locations on the body of the patient marked as a target site; automatically generating,with at least one processor, a surface map of the body of the patient based on the patient scan data; determining, with at least one processor, one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site; and generating, with at least one processor, data associated with a medical procedure plan for a medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient.

[0016] Clause 10: The method of clause 9, further comprising: performing the medical procedure on the patient using a robotic surgical tool based on the data associated with the medical procedure plan.

[0017] Clause 11 : The method of clauses 9 or 10, wherein performing the medical procedure on the patient using the robotic surgical tool comprises: performing the medical procedure on the patient using a robotically controlled surgical arm.

[0018] Clause 12: The method of any of clauses 9-11 , further comprising: determining a tissue type of the one or more locations on the surface map of the body of the patient; and wherein generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient comprises: generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient and the tissue type associated with the one or more locations on the surface map of the body of the patient.

[0019] Clause 13: The method of any of clauses 9-12, further comprising: identifying one or more locations of landmarks on the body of the patient in relation to the one or more locations on the body of the patient marked as a target site; wherein generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient comprises: generating the data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient, the tissue type associated with the one or more locations on the surface map of the body of the patient, and the one or more locations of landmarks on the body of the patient.

[0020] Clause 14: The method of any of clauses 9-13, wherein receiving the patient scan data associated with the body of the patient comprises: receiving the patient scan data from a computerized tomography (CT) scan device or a magnetic resonance imaging (MRI) device.

[0021] Clause 15: The method of any of clauses 9-14, wherein automatically generating the surface map of the body of the patient comprises: automatically generating a three-dimensional (3D) surface map of the body of the patient; and wherein determining the one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site comprises: determining 3D coordinates for each of the one or more locations on the 3D surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site.

[0022] Clause 16: A computer program product for machine learning based medical procedure planning, the computer program product comprising at least one non- transitory computer-readable medium including one or more instructions that, when executed by at least one processor, cause the at least one processor to: receive patient scan data associated with a body of a patient, wherein the patient scan data comprises data associated with a plurality of medical images of the body of the patient, wherein the data associated with the plurality of medical images of the body of the patient comprises one or more locations on the body of the patient marked as a target site; automatically generate a surface map of the body of the patient based on the patient scan data; determine one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site; and generate data associated with a medical procedure plan for a medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient.

[0023] Clause 17: The computer program product of clause 16, wherein the one or more instructions further cause the at least one processor to: perform the medical procedure on the patient using a robotic surgical tool based on the data associated with the medical procedure plan.

[0024] Clause 18: The computer program product of clauses 16 or 17, wherein the one or more instructions that cause the at least one processor to perform the medical procedure on the patient using the robotic surgical tool, cause the at least one processor to: perform the medical procedure on the patient using a robotically controlled surgical arm.

[0025] Clause 19: The computer program product of any of clauses 16-18, wherein the one or more instructions further cause the at least one processor to: determine a tissue type of the one or more locations on the surface map of the body of the patient;and wherein the one or more instructions that cause the at least one processor to generate the data associated with the medical procedure plan for the medical procedure to be performed on the patient, cause the at least one processor to: generate the data associated with the medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient and the tissue type associated with the one or more locations on the surface map of the body of the patient.

[0026] Clause 20: The computer program product of any of clauses 16-19, wherein the one or more instructions further cause the at least one processor to: identify one or more locations of landmarks on the body of the patient in relation to the one or more locations on the body of the patient marked as a target site; wherein the one or more instructions that cause the at least one processor to generate the data associated with the medical procedure plan for the medical procedure to be performed on the patient, cause the at least one processor to: generate the data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient, the tissue type associated with the one or more locations on the surface map of the body of the patient, and the one or more locations of landmarks on the body of the patient.

[0027] Clause 21 : The computer program product of any of clauses 16-20, wherein the one or more instructions that cause the at least one processor to receive the patient scan data associated with the body of the patient, cause the at least one processor to: receive the patient scan data from a computerized tomography (CT) scan device or a magnetic resonance imaging (MRI) device.

[0028] Clause 22: The computer program product of any of clauses 16-21 , wherein the one or more instructions that cause the at least one processor to automatically generate the surface map of the body of the patient cause the at least one processor to: automatically generate a three-dimensional (3D) surface map of the body of the patient; and wherein the one or more instructions that cause the at least one processor to determine the one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site cause the at least one processor to: determine 3D coordinates for each of the one or more locations on the 3D surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site.

[0029] These and other features and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structures and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the present disclosure. As used in the specification and the claims, the singular form of “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise.BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Additional advantages and details of non-limiting embodiments or aspects are explained in greater detail below with reference to the exemplary embodiments that are illustrated in the accompanying schematic figures, in which:

[0031] FIG. 1 is a diagram of a non-limiting embodiment of an environment in which systems, devices, products, apparatus, and / or methods, described herein, may be implemented, according to the principles of the present disclosure;

[0032] FIG. 2 is a diagram of a non-limiting embodiment of a system for providing machine learning based medical procedure planning;

[0033] FIG. 3 is a diagram of a non-limiting embodiment of components of one or more systems or one or more devices of FIGS. 1 and 2;

[0034] FIG. 4 is a flowchart of a non-limiting embodiment of a process for providing machine learning based medical procedure planning;

[0035] FIG. 5 is a flowchart of a non-limiting embodiment of a robotic surgical system;

[0036] FIGS. 6A-6E are diagrams of an implementation of a non-limiting embodiment or aspect of a process for providing machine learning based medical procedure planning.DETAILED DESCRIPTION

[0037] For purposes of the description hereinafter, the terms “end,” “upper,” “lower,” “right,” “left,” “vertical,” “horizontal,” “top,” “bottom,” “lateral,” “longitudinal,” and derivatives thereof shall relate to the present disclosure as it is oriented in the drawingfigures. However, it is to be understood that the present disclosure may assume various alternative variations and step sequences, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary embodiments of the present disclosure. Hence, specific dimensions and other physical characteristics related to the embodiments or aspects of the embodiments disclosed herein are not to be considered as limiting unless otherwise indicated.

[0038] No aspect, component, element, structure, act, step, function, instruction, and / or the like used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items, and may be used interchangeably with “one or more” and “at least one.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, etc.) and may be used interchangeably with “one or more” or “at least one.” Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based at least partially on” unless explicitly stated otherwise. The phrase “based on” may also mean “in response to” and be indicative of a condition for automatically triggering a specified operation of an electronic device (e.g., a processor, a computing device, etc.) as appropriately referred to herein.

[0039] As used herein, the terms “communication” and “communicate” may refer to the reception, receipt, transmission, transfer, provision, and / or the like of information (e.g., data, signals, messages, instructions, commands, and / or the like). For one unit (e.g., a device, a system, a component of a device or system, combinations thereof, and / or the like) to be in communication with another unit means that the one unit is able to directly or indirectly receive information from and / or transmit information to the other unit. This may refer to a direct or indirect connection that is wired and / or wireless in nature. Additionally, two units may be in communication with each other even though the information transmitted may be modified, processed, relayed, and / or routed between the first and second unit. For example, a first unit may be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit. As another example, a first unitmay be in communication with a second unit if at least one intermediary unit (e.g., a third unit located between the first unit and the second unit) processes information received from the first unit and communicates the processed information to the second unit. In some non-limiting embodiments, a message may refer to a network packet (e.g., a data packet and / or the like) that includes data. It will be appreciated that numerous other arrangements are possible.

[0040] As used herein, the term “system” may refer to one or more computing devices or combinations of computing devices such as, but not limited to, processors, servers, client devices, software applications, and / or other like components. In addition, reference to “a server” or “a processor,” as used herein, may refer to a previously-recited server and / or processor that is recited as performing a previous step or function, a different server and / or processor, and / or a combination of servers and / or processors. For example, as used in the specification and the claims, a first server and / or a first processor that is recited as performing a first step or function may refer to the same or different server and / or a processor recited as performing a second step or function.

[0041] Non-limiting embodiments of the present disclosure are directed to systems, methods, and computer program products for machine learning based medical procedure planning. In some non-limiting embodiments or aspects, an Artificial Intelligence (Al) medical procedure planning system may include a robotic surgical tool and at least one processor coupled to the robotic surgical tool and configured to receive patient scan data associated with a body of a patient, wherein the patient scan data includes imaging data associated with one or more medical images of a body of the patient, and the imaging data may include one or more locations on the body of the patient marked as a lesion. In some non-limiting embodiments, the imaging data may include data associated with a series of images of the body of the patient that are produced based on one or more modalities of medical images.

[0042] In some non-limiting embodiments, the Al medical procedure planning system may automatically generate a surface map of the body of the patient based on the patient scan data, determine one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a lesion, and generate data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient.

[0043] In some non-limiting embodiments, the robotic surgical tool includes a robotically controlled surgical arm and a robotically controlled platform with movement along at least two axis. In some non-limiting embodiments, the Al medical procedure planning system may perform the medical procedure on the patient using a robotic surgical tool based on the data associated with the medical procedure plan. In some non-limiting embodiments, performing the medical procedure on the patient using the robotic surgical tool may include performing the medical procedure on the patient using a robotically controlled surgical arm.

[0044] In some non-limiting embodiments, the Al medical procedure planning system may determine a tissue type of the one or more locations on the surface map of the body of the patient, and generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient may include generating the data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient and the tissue type of the one or more locations on the surface map of the body of the patient.

[0045] In some non-limiting embodiments, the Al medical procedure planning system may identify one or more locations of landmarks on the body of the patient in relation to the one or more locations on the body of the patient marked as a target site, and generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient may include generating the data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient, the tissue type of the one or more locations on the surface map, and the one or more locations of landmarks on the body of the patient.

[0046] In some non-limiting embodiments, receiving the patient scan data associated with the body of the patient may include receiving the patient scan data from a computerized tomography (CT) scan device or a magnetic resonance imaging (MRI) device. In some non-limiting embodiments, automatically generating the map of the surface of the body of the patient may include automatically generating a three- dimensional (3D) map of the body of the patient, and determining the one or more locations on the map of the surface of the body of the patient that correspond to the one or more locations on the body of the patient marked as a lesion may include determining 3D coordinates for each of the one or more locations on the 3D surfacemap of the body of the patient that correspond to the one or more locations on the body of the patient marked as a lesion.

[0047] In this way, non-limiting embodiments of the present disclosure provide an Al medical procedure planning system that allows for accurately locating a structure that is internal to the body of a patient and helps to eliminate mistakes with regard to performing a medical procedure at an incorrect location on the body of a patient. Furthermore, an Al medical procedure planning system allows for the generation of a medical procedure plan for a medical procedure to be performed on a patient that minimizes risks to the patient as compared to a situation where an operator, or other individual, contemplates a plan for the medical procedure.

[0048] Referring now to FIG. 1 , FIG. 1 is a diagram of a non-limiting embodiment of an environment 100 in which devices, systems, methods, and / or computer program products, described herein, may be implemented. As shown in FIG. 1 , environment 100 includes Artificial Intelligence (Al) medical procedure planning system 102, robotic surgical system 104, healthcare data source 106, user device 108, fluid injection system 110, medical imaging system 112, and communication network 114. In some non-limiting embodiments, Al medical procedure planning system 102, robotic surgical system 104, healthcare data source 106, user device 108, fluid injection system 110, and / or medical imaging system 112 may interconnect (e.g., establish a connection to communicate) via wired connections, wireless connections, or a combination of wired and wireless connections. Any devices or systems in environment 100 may communicate with each other in a same or different communication network 114 as other devices or systems.

[0049] In some non-limiting embodiments, Al medical procedure planning system 102 may include one or more devices capable of being in communication with robotic surgical system 104, healthcare data source 106, user device 108, fluid injection system 110, and / or medical imaging system 112, via communication network 114. For example, Al medical procedure planning system 102 may include one or more computing devices, such as one or more computers, one or more servers (e.g., a cloud server, a group of servers, etc.), one or more desktop computers, one or more mobile devices (e.g., one or more tablets, one or more smartphones, etc.), and / or the like. In some non-limiting embodiments, Al medical procedure planning system 102 may include one or more (e.g., a plurality of) applications (e.g., software applications) that perform a set of functionalities on an external application programming interface (API)that allows Al medical procedure planning system 102 to send data to an external system associated with the external API and to receive data from the external system associated with the external API. In some non-limiting embodiments, the application may be supported by an application associated with user device 108 that would allow Al medical procedure planning system 102, which may function as workstation (e.g., a workstation of a control room for a site at which medical procedures are performed), to be the only one device that controls other systems and / or devices, and, in such an example, Al medical procedure planning system 102 may provide an authentication function.

[0050] Additionally or alternatively, Al medical procedure planning system 102 may generate (e.g., train, validate, re-train, and / or the like), store, and / or implement (e.g., operate, provide inputs to and / or outputs from, and / or the like) one or more machine learning models. For example, Al medical procedure planning system 102 may generate one or more machine learning models by fitting (e.g., validating) one or more machine learning models against data used for training (e.g., training data). In some non-limiting embodiments or aspects, Al medical procedure planning system 102 may generate, store, and / or implement one or more machine learning models that are provided for a real-time environment (e.g., a runtime environment) used for providing inferences based on data in a live situation. In some non-limiting embodiments or aspects, Al medical procedure planning system 102 may be in communication with a data storage device, which may be local or remote to Al medical procedure planning system 102.

[0051] In some non-limiting embodiments, Al medical procedure planning system 102 may be capable of receiving (e.g., retrieving via a pull) information from, storing information in, transmitting information to, and / or searching information stored in a data source (e.g., healthcare data source 106). In some non-limiting embodiments, Al medical procedure planning system 102 may be a component of robotic surgical system 104, user device 108, fluid injection system 110, and / or medical imaging system 112.

[0052] In some non-limiting embodiments, Al medical procedure planning system 102 may operate based on the use of artificial intelligence. For example, Al medical procedure planning system 102 may include one or more artificial intelligence algorithms (e.g., machine learning models) that may work in collaboration with each other to identify patient structures, reconstruct patient surfaces, plan medicalprocedures, and provide data to a robotic surgical system (e.g., which includes a robotically controlled surgical arm) for execution or support in execution of a medical procedure. In some non-limiting embodiments, one or more of the artificial intelligence algorithms of Al medical procedure planning system 102 may be used to calculate an approach for a surgical procedure.

[0053] In some non-limiting embodiments, robotic surgical system 104 may include one or more devices capable of being in communication with Al medical procedure planning system 102, healthcare data source 106, user device 108, fluid injection system 110, and / or medical imaging system 112 via communication network 114. For example, healthcare data source 106 may include a server, a computing device, such as a desktop computer, a mobile device (e.g., a tablet, a smartphone, a wearable, such as a wearable health sensor, an implantable device, such as a pacemaker, an internal body sensor, etc.), and / or the like. In some non-limiting embodiments, robotic surgical system 104 may include a robot and / or one or more robotic surgical tools, such as one or more robotically controlled surgical arms. In some non-limiting embodiments, a robotically controlled surgical arm may include a manipulator, such as an end effector, that is configured to interact with an environment of robotic surgical system 104 (e.g., a body of a patient). In some non-limiting embodiments, a design of the manipulator may be based on a medical procedure (e.g., a surgical procedure) that is to be performed using robotic surgical system 104. In some non-limiting embodiments, robotic surgical system 104 may be a component of Al medical procedure planning system 102, user device 108, fluid injection system 110, and / or medical imaging system 112.

[0054] In some non-limiting embodiments, healthcare data source 106 may include one or more devices capable of being in communication with Al medical procedure planning system 102, robotic surgical system 104, user device 108, fluid injection system 110, and / or medical imaging system 112 via communication network 114. For example, healthcare data source 106 may include a server, a computing device, such as a desktop computer, a mobile device (e.g., a tablet, a smartphone, a wearable, such as a wearable health sensor, an implantable device, such as a pacemaker, an internal body sensor, etc.), and / or the like. In some non-limiting embodiments, healthcare data source 106 may include an electronic record system (e.g., an electronic medical record (EMR) system and / or an electronic health record (EHR) system), a patient procedure tracking system, a hospital information system (HIS), aradiology information system (RIS), a radiology analytics system (RAS), a laboratory information system (LIS), a pathology system, such as a digital pathology system (DPS), and / or an image archive and communication system, such as a picture archive and communication system (PACS). In some non-limiting embodiments, an HIS may include one or more subsystems, such as a patient procedure tracking system (e.g., a system that operates a modality worklist, a system that provides patient demographic information for fluid injection procedures and / or medical imaging procedures, etc.), a fluid injector management system, an image archive and communication system (e.g., a picture archive and communication system (PACS)), a radiology information system, a radiology analytics system (e.g., the Radimetrics® Enterprise Application marketed and sold by Bayer Healthcare LLC), and / or other like systems or devices.

[0055] Additionally or alternatively, healthcare data source 106 may include a medical imaging system (e.g., an imaging scanner), a fluid injection system (e.g., a fluid injector), a device associated with a facility, such as a communication device associated with a medical device (e.g., a hand-held medical device, a wearable medical device, such as a portable health sensor, etc.), a fluid injection system, and / or a device associated with a patient (e.g., a user device, such as a computing device operated by a patient). Additionally or alternatively, healthcare data source 106 may include additional devices located at a location, such as a hospital, that provides medical care, such as an Internet of Things (IOT) device, and / or other devices at locations relevant to a patient (e.g., a device at a home of a patient, a device located in a vehicle of a patient, etc.).

[0056] In some non-limiting embodiments, user device 108 may include one or more devices capable of being in communication with Al medical procedure planning system 102, robotic surgical system 104, healthcare data source 106, fluid injection system 110, and / or medical imaging system 112 via communication network 114. For example, user device 108 may include a computing device, such as one or more computers, including a desktop computer, a workstation device, a laptop, tablet, and / or the like. In some non-limiting embodiments, at least a portion of the processes executed at the user device 108 may be executed at a remote server (e.g., a cloud computing server). In some non-limiting embodiments, user device 108 may provide a user interface for controlling operation of a system or device, such as Al medical procedure planning system 102 and / or other systems (e.g., robotic surgical system104, fluid injection system 110, medical imaging system 112, etc.), including to generate instructions for and / or provide instructions to Al medical procedure planning system 102. Additionally or alternatively, user device 108 may display operational parameters of Al medical procedure planning system 102 during operation (e.g., during real-time operation) of Al medical procedure planning system 102. In some non-limiting embodiments, user device 108 may provide interconnectivity between Al medical procedure planning system 102 and other devices or systems, such as robotic surgical system 104, fluid injection system 110, and / or medical imaging system 112. In some non-limiting embodiments, user device 108 may include the Certegra® Workstation provided by Bayer Healthcare LLC. In some non-limiting embodiments, user device 108 may include a display unit (e.g., a display device, a display screen, etc.,), such as a computer monitor, a touchscreen, a heads-up display, and / or the like, which may be used to display a user interface (e.g., a graphical user interface (GUI) of a software application), via which a user may interact with user device 108 to view parameters and / or control operation of Al medical procedure planning system 102, robotic surgical system 104, fluid injection system 110, and / or medical imaging system 112. For example, a user of user device 108 may provide inputs to user device 108 using one or more hardware or software components of the user device 108 in connection with a touch screen, a mouse, a trackpad, a keyboard, a stylus, a gesturesensing camera, a microphone for receiving voice commands, and / or the like.

[0057] In some non-limiting embodiments, fluid injection system 110 may include one or more devices capable of being in communication with Al medical procedure planning system 102, robotic surgical system 104, healthcare data source 106, user device 108, and / or medical imaging system 112 via communication network 114. For example, fluid injection system 110 may include one or more computing devices, such as one or more computers, one or more servers (e.g., a cloud server, a group of servers, etc.), one or more desktop computers, one or mobile devices (e.g., one or more tablets, one or more smartphones, etc.), and / or the like. In some non-limiting embodiments, fluid injection system 110 may include one or more injection devices (e.g., one or more fluid injection devices, one or more fluid injectors). In some nonlimiting embodiments, fluid injection system 110 is configured to administer (e.g., inject, deliver, etc.) contrast fluid including a contrast agent to a patient, and / or administer an aqueous fluid, such as saline, to a patient before, during, and / or after administering the contrast fluid. For example, fluid injection system 110 can inject oneor more prescribed dosages of contrast fluid directly into a patient’s blood stream via a hypodermic needle and syringe. In some non-limiting embodiments, fluid injection system 110 may be configured to continually administer the aqueous fluid to a patient through a peripheral intravenous line (PIV) and catheter, and one or more prescribed dosages of contrast fluid may be introduced into the PIV and administered via the catheter to the patient. In some non-limiting embodiments, fluid injection system 110 is configured to inject a dose of contrast fluid along with and / or followed by administration of a particular volume of the aqueous fluid. In some non-limiting embodiments, fluid injection system 110 may include one or more exemplary fluid injection devices that are disclosed in: U.S. Patent Application Serial No. 09 / 715,330, filed on November 17, 2000, issued as U.S. Patent No. 6,643,537; U.S. Patent Application Serial No. 09 / 982,518, filed on October 18, 2001 , issued as U.S. Patent No. 7,094,216; U.S. Patent Application Serial No. 10 / 825,866, filed on April 16, 2004, issued as U.S. Patent No. 7,556,619; U.S. Patent Application Serial No. 12 / 437,011 , filed May 7, 2009, issued as U.S. Patent No. 8,337,456; U.S. Patent Application Serial No. 12 / 476,513, filed June 2, 2009, issued as U.S. Patent No. 8,147,464; and U.S. Patent Application Serial No. 11 / 004,670, filed on December 3, 2004, issued as U.S. 8,540,698, the disclosures of each of which are incorporated herein by reference in their entireties. In some non-limiting embodiments, fluid injection system 110 may include the MEDRAD® Stellant CT Injection System, the MEDRAD® Stellant FLEX CT Injection System, the MEDRAD® MRXperion MR Injection System, the MEDRAD® Mark 7 Arterion Injection System, the MEDRAD® Intego PET Infusion System, or the MEDRAD® Centargo CT Injection System, all of which are provided by Bayer Healthcare LLC.

[0058] In some non-limiting embodiments, medical imaging system 112 may include one or more devices capable of being in communication with Al medical procedure planning system 102, robotic surgical system 104, healthcare data source 106, user device 108, and / or fluid injection system 110 via communication network 114. For example, medical imaging system 112 may include one or more computing devices, such as one or more computers, one or more servers (e.g., a cloud server, a group of servers, etc.), one or more desktop computers, one or mobile devices (e.g., one or more tablets, one or more smartphones, etc.), and / or the like. In some non-limiting embodiments, medical imaging system 112 may include one or more scanning devices, such as a computed tomography (CT) device, a magnetic resonance imaging(MRI) device, a positron emission tomography (PET) device, an ultrasound device, and / or any combination thereof. Additionally or alternatively, medical imaging system 112 may include one or more X-ray machines.

[0059] In some non-limiting embodiments, communication network 114 may include one or more wired and / or wireless networks. For example, communication network 114 may include a cellular network (e.g., a long-term evolution (LTE®) network, a third generation (3G) network, a fourth generation (4G) network, a fifth generation (5G) network, a sixth generation (6G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, a short range wireless communication network (e.g., a Bluetooth® network, a near field communication (NFC) network, etc.) and / or the like, and / or a combination of these or other types of networks.

[0060] The number and arrangement of systems and / or devices shown in FIG. 1 are provided as an example. There may be additional systems and / or devices, fewer systems and / or devices, different systems and / or devices, or differently arranged systems and / or devices than those shown in FIG. 1. Furthermore, two or more systems and / or devices shown in FIG. 1 may be implemented within a single system or a single device, or a single system or a single device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally or alternatively, a set of systems or a set of devices (e.g., one or more systems, one or more devices) of environment 100 may perform one or more functions described as being performed by another set of systems or another set of devices of environment 100.

[0061] Referring now to FIG. 2, FIG. 2 is a diagram of a non-limiting embodiment of system 200 for providing machine learning based analytics of healthcare information. In some non-limiting embodiments, one or more of the functions described herein with respect to system 200 may be performed (e.g., completely, partially, and / or the like) by Al medical procedure planning system 102. In some non-limiting embodiments, one or more of the functions described with respect to system 200 may be performed (e.g., completely, partially, and / or the like) by another device or a group of devices separate from and / or including Al medical procedure planning system 102, such as workstation device 208 (e.g., which includes display unit 208A), fluid injection system110, hospital information system 206, and / or medical imaging system 112.

[0062] As shown in FIG. 2, system 200 includes Al medical procedure planning system 102, fluid injection system 110, workstation device 208, which includes display unit 208A, fluid injection system 110, hospital information system 206, electronic record system 212, digital pathology system 214, medical imaging system 112, and laboratory information system 218. In some non-limiting embodiments, Al medical procedure planning system 102 may interconnect (e.g., establish a connection to communicate with and / or the like) fluid injection system 110, workstation device 208, hospital information system 206, electronic record system 212, digital pathology system 214, medical imaging system 112, and / or laboratory information system 218 via wired connections, wireless connections, or a combination of wired and wireless connections. In some non-limiting embodiments, workstation device 208 may be the same as or similar to user device 108.

[0063] As further shown in FIG. 2, hospital information system 206 may include a plurality of subsystems. The plurality of subsystems may include patient procedure tracking system 206A, image archive and communication system 206B, radiology information system 206C, and radiology analytics system 206D. In some non-limiting embodiments, Al medical procedure planning system 102 may receive healthcare data from hospital information system 206 via a communication network (e.g., communication network 114), according to a communications protocol for communicating the data associated with informatics. For example, Al medical procedure planning system 102 may receive data associated with a patient procedure from hospital information system 206 (e.g., from patient procedure tracking system 206A) via the communication network according to a Digital Imaging and Communications in Medicine (DICOM) communications protocol, data associated with an operation of the fluid injection system 110 from hospital information system 206 via the communication network based on an API call (e.g., an API call from Al medical procedure planning system 102), data associated with a radiology image from hospital information system 206 (e.g., from image archive and communication system 206B) via the communication network according to a DICOM communications protocol, data associated with a patient examination procedure from hospital information system 206 (e.g., from radiology information system 206C) via the communication network according to a Health Level Seven (HL7) standard communications protocol, and / or data associated with radiation dosage during a medical imaging procedure fromhospital information system 206 (e.g., from radiology analytics system 206D) via the communication network based on an API call (e.g., an API call from Al medical procedure planning system 102 to radiology analytics system 206D).

[0064] In some non-limiting embodiments, electronic record system 212 may include one or more devices capable of being in communication with Al medical procedure planning system 102, workstation device 208, fluid injection system 110, hospital information system 206, digital pathology system 214, medical imaging system 112, and / or laboratory information system 218 via a communication network (e.g., communication network 114). In some non-limiting embodiments, electronic record system 212 may include one or more devices that receive, manage, store, and / or transmit electronic medical records (e.g., electronic health records) that include medical record data associated with a medical record of a patient, such as demographics, medical history, medication and allergies, immunization status, laboratory test results, radiology images, vital signs, personal statistics (e.g., age, weight, height, etc.), billing information, and / or the like, associated with specific instances of medical care. Additionally or alternatively, electronic record system 212 may include a patient portal (e.g., a web-based interface) to allow a patient to interact with a respective electronic medical record for the patient.

[0065] In some non-limiting embodiments, digital pathology system 214 may include one or more devices capable of being in communication with Al medical procedure planning system 102, workstation device 208, fluid injection system 110, hospital information system 206, electronic record system 212, medical imaging system 112, and / or laboratory information system 218 via a communication network (e.g., communication network 114 in FIG. 1 ). In some non-limiting embodiments, medical imaging system 112 may include one or more devices that receive, manage, transmit, and / or interpret pathology information including data (e.g., image data, slide data, etc.) analyzed by a microscope, a scanner, and / or other like devices.

[0066] In some non-limiting embodiments, medical imaging system 112 may include one or more devices capable of being in communication with Al medical procedure planning system 102, workstation device 208, fluid injection system 110, hospital information system 206, electronic record system 212, digital pathology system 214, and / or laboratory information system 218 via a communication network (e.g., communication network 114). In some non-limiting embodiments, medical imaging system 112 may include one or more scanning devices, such as a computedtomography (CT) device and / or a magnetic resonance imaging (MRI) device, capable of communicating via a communication network and capable of performing medical imaging procedures involving the use of a radiological contrast material.

[0067] In some non-limiting embodiments, laboratory information system 218 may include one or more devices capable of being in communication with Al medical procedure planning system 102, fluid injection system 110, workstation device 208, hospital information system 206, electronic record system 212, digital pathology system 214, and / or medical imaging system 112 via a communication network (e.g., communication network 114). In some non-limiting embodiments, medical imaging system 112 may include one or more devices that record, manage, update, and / or store patient data and / or testing data for clinical and / or anatomic pathology laboratories, including receiving test orders, transmitting orders to laboratory analyzers, tracking orders, results, and / or quality control information, and / or transmitting results to other systems or devices.

[0068] In some non-limiting embodiments, Al medical procedure planning system 102 may include a plurality of applications, and each of the plurality of applications may be associated with an API associated with a respective application (e.g., a first API associated with a first application, a second API associated with a second application, a third API associated with a third application, etc.) that allows other systems and / or devices to interface (e.g., communicate, establish a communication interface, etc.) with Al medical procedure planning system 102 and / or that allows Al medical procedure planning system 102 to interface with other systems and / or devices (e.g., individual subsystems of hospital information system 206, such as patient procedure tracking system 206A, image archive and communication system 206B, radiology information system 206C, and / or radiology analytics system 206D). In some non-limiting embodiments, Al medical procedure planning system 102 may provide a user interface (e.g., via an application that includes a user interface, such as a webbased user interface) that allows a user to access information such as a medical procedure plan for a patient.

[0069] As further shown in FIG. 2, workstation device 208 may include display unit 208A. In some non-limiting embodiments, display unit 208A may be capable of displaying the user interface (e.g., the web-based user interface) provided by Al medical procedure planning system 102. In some non-limiting embodiments, display unit 208A may include a computing device, such as a smart display unit, a portablecomputer, such as a tablet, a laptop, and / or the like. In some non-limiting embodiments, display unit 208A may include a touchscreen for receiving inputs by a user. In some non-limiting embodiments, display unit 208A may include a display device (e.g., a monitor, a screen, and / or the like) for displaying visual information.

[0070] In some non-limiting embodiments, Al medical procedure planning system 102 may transmit data associated with an image received from medical imaging system 112 to fluid injection system 110 via a communication network. For example, Al medical procedure planning system 102 may transmit data associated with the image received from medical imaging system 112 to fluid injection system 110 via the communication network based on an API call from fluid injection system 110. In some non-limiting embodiments, Al medical procedure planning system 102 may transmit data associated with a fluid injection procedure (e.g., data associated with a volume, a flow rate and / or an amount of time for injecting a radiological contrast material into a patient) received from fluid injection system 110 to medical imaging system 112 via the communication network. For example, Al medical procedure planning system 102 may transmit the data associated with the fluid injection procedure received from fluid injection system 110 to medical imaging system 112 via the communication network based on an API call (e.g., an API call for an imaging system interface (ISI), an API call for an ISI2 interface, an API call for a Connect CT interface, etc.) from medical imaging system 112. In some non-limiting embodiments, medical imaging system 112 may perform a medical imaging procedure on a patient based on the data, inclusive of an injection protocol, associated with the fluid injection procedure. In some nonlimiting embodiments, Al medical procedure planning system 102 may receive data associated with an operation of medical imaging system 112 from medical imaging system 112 via the communication network based on an API call (e.g., an API call from Al medical procedure planning system 102 to medical imaging system 112).

[0071] In some non-limiting embodiments, Al medical procedure planning system 102 may provide a communication interface between hospital information system 206 and fluid injection system 110 such that fluid injection system 110 is able to receive data based on an API call from fluid injection system 110 to Al medical procedure planning system 102. In some non-limiting embodiments, Al medical procedure planning system 102 may transmit data associated with informatics received from hospital information system 206 to fluid injection system 110 via a communication network (e.g., communication network 114). For example, Al medical procedureplanning system 102 may transmit data associated with informatics received from hospital information system 206 to fluid injection system 110 via the communication network based on an API call from fluid injection system 110.

[0072] Referring now to FIG. 3, FIG. 3 is a diagram of example components of device 300. Device 300 may correspond to one or more devices of Al medical procedure planning system 102, one or more devices of robotic surgical system 104, healthcare data source 106, user device 108, workstation device 208, one or more devices of fluid injection system 110, one or more devices of medical imaging system 112, one or more devices of hospital information system 206, one or more devices of electronic record system 212, one or more devices of digital pathology system 214, and / or one or more devices of laboratory information system 218. In some non-limiting embodiments, Al medical procedure planning system 102, robotic surgical system 104, healthcare data source 106, user device 108, workstation device 208, fluid injection system 110, medical imaging system 112, hospital information system 206, electronic record system 212, digital pathology system 214, and / or laboratory information system 218 may include at least one device 300 and / or at least one component of device 300.

[0073] As shown in FIG. 3, device 300 may include bus 302, processor 304, memory 306, storage component 308, input component 310, output component 312, and communication interface 314. Bus 302 may include a component that permits communication among the components of device 300. In some non-limiting embodiments, processor 304 may be implemented in hardware, firmware, or a combination of hardware and software. For example, processor 304 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.) that can be programmed to perform a function. Memory 306 may include random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by processor 304.

[0074] Storage component 308 may store information and / or software related to the operation and use of device 300. For example, storage component 308 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid statedisk, etc.), a compact disc (CD), a digital versatile disc (DVD), a floppy disk, a cartridge, a magnetic tape, and / or another type of computer-readable medium, along with a corresponding drive.

[0075] Input component 310 may include a component that permits device 300 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally, or alternatively, input component 310 may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output component 312 may include a component that provides output information from device 300 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.).

[0076] Communication interface 314 may include a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables device 300 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 314 may permit device 300 to receive information from another device and / or provide information to another device. For example, communication interface 314 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, and / or the like.

[0077] Device 300 may perform one or more processes described herein. Device 300 may perform these processes based on processor 304 executing software instructions stored by a computer-readable medium, such as memory 306 and / or storage component 308. A computer-readable medium (e.g., a non-transitory computer-readable medium) is defined herein as a non-transitory memory device. A memory device may include memory space located inside of a single physical storage device or memory space spread across multiple physical storage devices.

[0078] Software instructions may be read into memory 306 and / or storage component 308 from another computer-readable medium or from another device via communication interface 314. When executed, software instructions stored in memory 306 and / or storage component 308 may cause processor 304 to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments described herein are not limited toany specific combination of hardware circuitry and software.

[0079] The number and arrangement of components shown in FIG. 3 are provided as an example. In some non-limiting embodiments, device 300 may include additional components, fewer components, different components, or differently arranged components than those shown in FIG. 3. Additionally, or alternatively, a set of components (e.g., one or more components) of device 300 may perform one or more functions described as being performed by another set of components of device 300.

[0080] Referring now to FIG. 4, FIG. 4 is a flowchart of a non-limiting embodiment of a process 400 for machine learning based medical procedure planning. In some nonlimiting embodiments, one or more of the steps of process 400 are performed (e.g., completely, partially, etc.) by Al medical procedure planning system 102. In some non-limiting embodiments, one or more of the steps of process 400 are performed (e.g., completely, partially, etc.) by another device or a group of devices separate from or including Al medical procedure planning system 102, such as a data source (e.g., healthcare data source 106), a user device (e.g., user device 108, workstation device 208), a fluid injection system (e.g., fluid injection system 110, such as one or more devices of fluid injection system 110), a medical imaging system (e.g., medical imaging system 112, such as one or more devices of medical imaging system 112), and / or a hospital information system (e.g., hospital information system 206, such as one or more subsystems of hospital information system 206, etc.).

[0081] As shown in FIG. 4, at step 402, process 400 may include receiving data associated with a body of a patient. For example, Al medical procedure planning system 102 may receive data associated with the body of patient. In some non-limiting embodiments, the data associated with a body of a patient may include patient scan data associated with the body of a patient that is undergoing a medical procedure (e.g., a surgical procedure). In some non-limiting embodiments, the patient scan data may include data associated with one or more medical images (e.g., a plurality of medical images) of a body of the patient that includes one or more locations on the body of the patient that are marked as a target site for the medical procedure (e.g., marked as a site for a surgical procedure, such as a site marked as a lesion, a site for a biopsy, etc.).

[0082] Additionally or alternatively, the patient scan data associated with the body of the patient may include medical record data associated with a medical record of the patient (e.g., a medical record that includes prior surgical procedures performed on apatient, such as prior surgical implantation procedures), identification data associated with an identifier of a patient, data associated with a patient examination procedure (e.g., a fluid injection procedure and / or a medical imaging procedure performed on a patient), such as data associated with a contrast fluid provided during a fluid injection procedure, and / or data associated with characteristics of one or more locations on the body of the patient (e.g., data associated with a tissue type of one or more locations on the body of the patient, data associated with devices implanted in the body of the patient, etc.). In some non-limiting embodiments, Al medical procedure planning system 102 may store the patient scan data in a data structure (e.g., a database, such as a database of a healthcare data source). For example, Al medical procedure planning system 102 may store the patient scan data in the data structure with a unique identifier for the patient.

[0083] In some non-limiting embodiments, Al medical procedure planning system 102 may receive the patient scan data from robotic surgical system 104, healthcare data source 106, user device 108, fluid injection system 110, and / or medical imaging system 112. In some non-limiting embodiments, Al medical procedure planning system 102 may retrieve patient scan data from robotic surgical system 104, healthcare data source 106, user device 108, fluid injection system 110, and / or medical imaging system 112 based on an identifier associated with the patient. For example, Al medical procedure planning system 102 may transmit an identifier (e.g., a unique patient identifier, such as a unique patient identifier for a patient medical record) associated with a patient (e.g., a patient undergoing an examination procedure) to robotic surgical system 104, healthcare data source 106, user device 108, fluid injection system 110, and / or medical imaging system 112, and Al medical procedure planning system 102 may receive the patient scan data. In some nonlimiting embodiments, robotic surgical system 104, healthcare data source 106, user device 108, fluid injection system 110, and / or medical imaging system 112 may receive the identifier associated with the patient, retrieve the patient scan data based on the identifier, and transmit the patient scan data to Al medical procedure planning system 102. In some non-limiting embodiments, the identifier may be associated with a patient record of a patient (e.g., a patient record of a patient stored in patient procedure tracking system 206A of hospital information system 206).

[0084] In some non-limiting embodiments, the patient scan data may be based on one or more medical images (e.g., one or more radiological images, one or moreultrasound images, etc.) produced by medical imaging system 112. In one example, medical imaging system 112 may produce a series of images in which a location (e.g., a lesion) is marked that identifies a target site. In some non-limiting embodiments, the series of images may be produced in any single modality or any combination of modalities. In one example, all images of the series may be produced by a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, a positron emission tomography (PET) device, an ultrasound device, or an X-ray machine. In another example, the images of the series may be produced by any combination of a CT scanner, an MRI scanner, PET scanner, an ultrasound scanner, and / or an X-ray machine. In some non-limiting embodiments, Al medical procedure planning system 102 may receive data associated with one or more medical images and Al medical procedure planning system 102 may generate the patient scan data based on the data associated with one or more medical images.

[0085] In some non-limiting embodiments, Al medical procedure planning system 102 may generate the patient scan data. In one example, Al medical procedure planning system 102 may receive data associated with one or more medical images of a body of a patient (e.g., a series of images of a body of a patient, such as a series of sequential images) and Al medical procedure planning system 102 may mark one or more locations on the one or images as target sites to provide the patient scan data. In some non-limiting embodiments, Al medical procedure planning system 102 may receive an input (e.g., from an operator of Al medical procedure planning system 102, user device 108, medical imaging system 112, etc.) that includes an indication of a target site on the body of the patient at one or more locations in the one or more medical images. Additionally or alternatively, Al medical procedure planning system 102 may automatically determine one or more locations in the one or more medical images to mark as a target site. In one example, Al medical procedure planning system 102 may automatically determine the one or more locations to mark as a target site using a first machine learning model that is configured to identify a specific type of location (e.g., a location for a surgical procedure, such as a location of a lesion or a location of a biopsy site) on an image of a body of a patient. In some non-limiting embodiments, Al medical procedure planning system 102 may mark one or more locations on the one or images as a boundary of one or more target sites for a medical procedure.

[0086] As shown in FIG. 4, at step 404, process 400 may include automaticallygenerating a map of a surface of the body of the patient. For example, Al medical procedure planning system 102 may include automatically generating the map of a surface of the body of the patient. In some non-limiting embodiments, Al medical procedure planning system 102 may automatically generate a three-dimensional (3D) map of the body of the patient.

[0087] In some non-limiting embodiments, Al medical procedure planning system 102 may generate the map of a surface of the body of the patient based on the patient scan data. For example, Al medical procedure planning system 102 may provide the patient scan data (e.g., data associated with a plurality of medical images of the body of the patient) as an input to a second machine learning model and the second machine learning model may provide, as an output, internal structure data with regard to one or more medical images that identifies internal structures of the body, such as organs, vasculature, nerves, lymph nodes, and / or the like. In some non-limiting embodiments, Al medical procedure planning system 102 may provide the internal structure data as an input to a third machine learning model and the second third learning model may provide, as an output, a surface map (e.g., a 3D surface map) of the surface of the body of the patient. The surface map may be reconstructed from a plurality of medical images of the body of the patient. Additionally or alternatively, the third machine learning model may provide, as an output, a 3D image of the target site.

[0088] As shown in FIG. 4, at step 406, process 400 may include determining one or more locations on the surface map of the body of the patient that correspond to one or more marked locations. For example, Al medical procedure planning system 102 may determine one or more locations on the surface map of the body of the patient that correspond to (e.g., that match) one or more locations marked as a target site on one or more images of the body of the patient. In some non-limiting embodiments, Al medical procedure planning system 102 may determine coordinates (e.g., position coordinates, such as coordinates in a Cartesian coordinate system) for each of one or more locations on the surface map of the body of the patient that correspond to one or more locations marked as a target site in one or more medical images. For example, Al medical procedure planning system 102 may determine 3D coordinates for each of one or more locations on a 3D surface map of the body of the patient that correspond to one or more locations marked as a target site.

[0089] As shown in FIG. 4, at step 408, process 400 may include generating data associated with a medical procedure plan for a medical procedure. For example, Almedical procedure planning system 102 may generate data associated with a medical procedure plan for a medical procedure. In some non-limiting embodiments, Al medical procedure planning system 102 may generate the data associated with a medical procedure plan for a medical procedure to be performed on the patient based on the one or more locations on the surface map (e.g., the 3D surface map) of the body of the patient and / or a tissue type associated with the one or more locations on the surface map of the body of the patient. In some non-limiting embodiments, Al medical procedure planning system 102 may determine a tissue type of the one or more locations on the surface map of the body of the patient. For example, Al medical procedure planning system 102 may determine the tissue type of the one or more locations based on the patient scan data.

[0090] In some non-limiting embodiments, the medical procedure plan may include a route in the body of the patient to one or more locations marked as one or more target sites and / or series of actions to be carried out by robotic surgical system 104 to accomplish the medical procedure at the target site. In some non-limiting embodiments, the route of the medical procedure plan may be configured to avoid internal structures in the body of the patient. In some non-limiting embodiments, the medical procedure plan may be based on feedback (e.g., provided in real-time by robotic surgical system 104) during the medical procedure. For example, a feedback may be provided via an ultrasound component of robotic surgical system 104. In such an example, robotic surgical system 104 may use the feedback in combination with patient scan data (e.g., data associated with one or more medical images of the body of the patient and / or data associated with one or more marked locations) for accessing multiple entry points to the body of the patient.

[0091] In some non-limiting embodiments, Al medical procedure planning system 102 may generate the medical procedure plan based on the surface map (e.g., the 3D surface map) of the body of the patient to (e.g., via a landmarking process) determine the route for a medical procedure plan. In one example, Al medical procedure planning system 102 may mark relevant locations (e.g., landmarks, such as surface markers on the body of the patient that may be known (e.g., visible) to an operator of Al medical procedure planning system 102 and / or recognizable by Al medical procedure planning system 102) on the surface map of the body of the patient and use the relevant locations to determine the route for a medical procedure plan. In some non-limiting embodiments, Al medical procedure planning system 102 may determinea relevant location by comparing a location on the surface map of the body of the patient with data associated with a marked location on the body of the patient (e.g., of the patient scan data associated with the body of patient) and determining whether the location is within a threshold distance of the marked location. If Al medical procedure planning system 102 determines that the location is within the threshold distance, Al medical procedure planning system 102 may determine that the location is a relevant location. If Al medical procedure planning system 102 determines that the location is not within the threshold distance, Al medical procedure planning system 102 may determine that the location is not a relevant location.

[0092] In some non-limiting embodiments, Al medical procedure planning system 102 may generate the medical procedure based on an input. For example, Al medical procedure planning system 102 may receive an input (e.g., from an operator of Al medical procedure planning system 102, such as a clinician) that includes data that identifies a type of surgical procedure desired (e.g., a biopsy, such as fine needle aspiration (FNA), a core needle procedure, a vacuum assisted procedure, etc.) and Al medical procedure planning system 102 may generate the medical procedure based on the data that identifies the type of surgical procedure desired. Additionally or alternatively, Al medical procedure planning system 102 may receive an input that includes data that identifies a number of samples are needed to be taken from each location of the one or more locations that are marked and Al medical procedure planning system 102 may generate the medical procedure based on the data that identifies the number of samples. In some non-limiting embodiments, such inputs may also indicate if internal structures (e.g., lymph nodes) are to be biopsied.

[0093] In some non-limiting embodiments, Al medical procedure planning system 102 may provide the inputs (e.g., data that identifies the type of surgical procedure desired, data that identifies a number of samples are needed to be taken from each location of the one or more locations that are marked as a target site), identification data associated with an identifier of a patient, data associated with one or more medical images of the body of the patient that includes one or more locations marked as a target site, and / or data associated with the surface map as an input to a fourth machine learning model and the fourth machine learning model may provide output data associated with a medical procedure plan (e.g., an optimized route with regard to the body of the patient) for a medical procedure. In some non-limiting embodiments, the medical procedure plan may identify a surface location for needle insertion as wellas an approach angle and / or route (e.g., path) to the location that is marked as the target site.

[0094] In some non-limiting embodiments, Al medical procedure planning system 102 may provide data associated with the medical procedure plan to robotic surgical system 104, healthcare data source 106, user device 108, fluid injection system 110, and / or medical imaging system 112 and / or other systems and devices. In some nonlimiting embodiments, Al medical procedure planning system 102 may provide the data associated with the medical procedure plan to a system or device based on receiving a request for the medical procedure plan. For example, Al medical procedure planning system 102 may receive a request for the medical procedure plan that includes a unique patient identifier associated with a patient. Al medical procedure planning system 102 may retrieve the medical procedure plan from a data structure based on the unique patient identifier associated with the patient and Al medical procedure planning system 102 may transmit the medical procedure plan to a system or device that provided the request for the medical procedure plan. In some non-limiting embodiments, the data associated with a medical procedure plan may be provided to an operator and / or robotic surgical system 104, which may use the data associated with a medical procedure plan to position the needed surgical tools to assist the operator.

[0095] Referring now to FIG. 5, FIG. 5 is a diagram of a non-limiting embodiment of robotic surgical system 504. In some non-limiting embodiments, robotic surgical system 504 may be the same as or similar to robotic surgical system 104. In some non-limiting embodiments, robotic surgical system 504 may include more components or fewer components than those shown in FIG. 5. As shown in FIG. 5, robotic surgical system 504 may include robotic surgical tools 522a, 522b, 522c, robotically controlled platform 526, and instrument table 528. As further shown in FIG. 5, each of robotic surgical tools 522a, 522b, 522c may include (respectively) robotically controlled surgical arms 522aa, 522ba, 522ca, which are connected to (e.g., proximal ends of robotically controlled surgical arms 522aa, 522ba, 522ca are connected to) moveable base sections 522ad, 522bd, 522cd, respectively. As further shown in FIG. 5, moveable base sections 522ad, 522bd, 522cd are mounted to stationary rail sections 522ac, 522bc, 522cc, which allows for movement of moveable base sections 522ad, 522bd, 522cd in at least two directions along a length of stationary rail sections 522ac, 522bc, 522cc.

[0096] As further shown in FIG. 5, stationary rail section 522ac may be mounted to a ceiling of a room in which robotic surgical system 504 is positioned. In addition, stationary rail sections 522bc and 522cc may be mounted to robotically controlled platform 526. Furthermore, each of robotic surgical tools 522a, 522b, 522c may include (respectively) end effectors 522ab, 522bb, 522cb that are attached to (e.g., attached to a distal end of) robotically controlled surgical arms 522aa, 522ba, 522ca. In some non-limiting embodiments, each end effectors 522ab, 522bb, 522cb may include appropriate surgical instruments and / or components for coupling end effectors 522ab, 522bb, 522cb to instruments 530 on instrument table 528. Each of robotically controlled surgical arms 522aa, 522ba, 522ca may include one or more articulating sections that allow end effectors 522ab, 522bb, 522cb to reach any desirable location during a procedure performed with robotic surgical system 504. In this way, robotically controlled surgical arms 522aa, 522ba, 522ca may be able to hold surgical instruments in a correct position to allow for medical procedures to be performed at optimal locations and angles with minimal impact to internal structures of a patient. In some non-limiting embodiments, robotically controlled surgical arms 522aa, 522ba, 522ca may use a centering tool to make sure that a respective robotically controlled surgical arm 522aa, 522ba, 522ca is correctly oriented and / or positioned on robotically controlled platform 526 and subject to correlate with an expected location of a target site based on a surface map of a body of a patient. In some non-limiting embodiments, robotically controlled surgical arms 522aa, 522ba, 522ca may use orientation data (e.g., coordinate and / or position data) to mark the location of the target site for the medical procedure.

[0097] In some non-limiting embodiments, robotically controlled surgical arms 522aa, 522ba, 522ca may hold surgical instruments for a medical procedure in the correct location and / or orientation to allow for an operator to perform the medical procedure in a manual or semi-manual fashion. In some non-limiting embodiments, robotic surgical system 504 may receive data (e.g., data associated with a medical procedure plan, data associated with one or more medical images, data associated with feedback from a component, such as an ultrasound component, etc.) and robotically controlled surgical arms 522aa, 522ba, 522ca may be controlled to perform a medical procedure automatically.

[0098] As further shown in FIG. 5, robotically controlled platform 526 includes top surface 526a and base 526b. In some non-limiting embodiments, top surface 526amay be mounted to base 526b so that top surface 526a may configured to be able to move along at least two axis. In one example, top surface 526a may configured to be able to move with six degrees of freedom in 3D space.

[0099] Referring now to FIGS. 6A-6E, FIGS 6A-6E are diagrams of a non-limiting embodiment or aspect of implementation 600 relating to a process (e.g., process 400) for machine learning based medical procedure planning. In some non-limiting embodiments or aspects, one or more of the steps of the process may be performed (e.g., completely, partially, etc.) by Al medical procedure planning system 102 (e.g., by one or more devices of Al medical procedure planning system 102). In some nonlimiting embodiments or aspects, one or more of the steps of the process may be performed (e.g., completely, partially, etc.) by another device or a group of devices separate from or including Al medical procedure planning system 102, such as a data source (e.g., healthcare data source 106), a user device (e.g., user device 108, workstation device 208), a fluid injection system (e.g., fluid injection system 110, such as one or more devices of fluid injection system 110), and / or a hospital information system (e.g., hospital information system 206, such as one or more subsystems of hospital information system 206, etc.).

[0100] As shown by reference number 605 in FIG. 6A, Al medical procedure planning system 102 may receive data associated with a plurality of medical images of a patient undergoing a medical procedure from a medical imaging system 112. In some nonlimiting embodiments, the plurality of medical images may include radiology images in a CT series of images and / or MRI series of images.

[0101] As shown by reference number 610 in FIG. 6B, Al medical procedure planning system 102 may generate patient scan data associated with the body of the patient. For example, Al medical procedure planning system 102 may generate patient scan data associated with the body of the patient based on the plurality of medical images. In some non-limiting embodiments, Al medical procedure planning system 102 may provide the data associated with the plurality of medical images as an input to a target site location machine learning model and the target site location machine learning model may mark a location on one or more of the medical images as a target site for a medical procedure to provide the patient scan data.

[0102] As shown by reference number 615 in FIG. 6C, Al medical procedure planning system 102 may generate a 3D surface map of the body of the patient. For example, Al medical procedure planning system 102 may generate a 3D surface map of thebody of the patient based on the patient scan data associated with the body of the patient. In some non-limiting embodiments, Al medical procedure planning system 102 may provide the plurality of medical images and the location on one or more of the medical images marked as a target site as an input to a surface map generation machine learning model and the surface map generation machine learning model may provide the 3D surface map as an output. In some non-limiting embodiments, the surface map generation machine learning model may reconstruct layers of radiology images as the 3D surface map. In some non-limiting embodiments, Al medical procedure planning system 102 may identify the one or more locations on the surface of the body of the patient marked as a target site using the 3D surface map. In some non-limiting embodiments, Al medical procedure planning system 102 may identify the one or more locations based on data associated with the plurality of medical images and / or data associated with a tissue type (e.g., a tissue type of an area associated with the location marked as a target site, such as an area in which or around where the marked location is located).

[0103] As shown by reference number 620 in FIG. 6D, Al medical procedure planning system 102 may identify one or more landmarks on the body of the patient. For example, Al medical procedure planning system 102 may identify one or more landmarks on the body of the patient based on the 3D surface map of the body of the patient and data associated with one or more medical images of the body of the patient. In some non-limiting embodiments, Al medical procedure planning system 102 may provide the 3D surface map and data associated with one or more medical images of the body of the patient as inputs to a medical procedure landmark identification machine learning model and the medical procedure landmark identification machine learning model may provide a location of one or more landmarks on the body of the patient in proximity to the marked location as an output.

[0104] As shown by reference number 625 in FIG. 6E, Al medical procedure planning system 102 may generate the medical procedure plan for the medical procedure. For example, Al medical procedure planning system 102 may generate the medical procedure plan based on data that identifies a type of medical procedure, data that identifies a number of samples are needed to be taken from each location of the one or more locations that are marked as a target site, identification data associated with an identifier of a patient, data associated with one or more medical images of the body of the patient that includes one or more locations marked as a target site, dataassociated with the 3D surface map, and / or any combination thereof. In some nonlimiting embodiments, Al medical procedure planning system 102 may provide data that identifies a type of medical procedure, data that identifies a number of samples are needed to be taken from each location of the one or more locations that are marked as a target site, identification data associated with an identifier of a patient, data associated with one or more medical images of the body of the patient that includes one or more locations marked as a target site, and / or data associated with the 3D surface map as inputs to a medical procedure planning machine learning model and the medical procedure planning machine learning model may provide one or more plans to approach the marked location as an output. In some non-limiting embodiments, the one or more plans may include one or more routes along the 3D surface map that provides data regarding movements (e.g., movements with regard to distance, angles, velocity, etc.) that a robotic surgical tool may perform to avoid one or more structures of the body of the patient (e.g., nerves, organs, vasculature, etc.).

[0105] As shown by reference number 630 in FIG. 6F, Al medical procedure planning system 102 may perform the medical procedure on the patient using a robotic surgical tool. For example, Al medical procedure planning system 102 may perform the medical procedure on the patient based on the data associated with the medical procedure plan using a robotically controlled surgical arm of robotic surgical system 104. In some non-limiting embodiments, Al medical procedure planning system 102 may perform the medical procedure on the patient by providing commands to robotic surgical system 104 to perform the medical procedure according to the medical procedure plan.

[0106] Although the above systems, methods, and computer program products have been described in detail for the purpose of illustration based on what is currently considered to be the most practical and preferred embodiments, it is to be understood that such detail is solely for that purpose and that the present disclosure is not limited to the described embodiments or aspects but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, at least one feature of any embodiment or aspect can be combined with at least one feature of any other embodiment.

Claims

WHAT IS CLAIMED IS:

1. A system for machine learning based medical procedure planning, comprising: at least one processor configured to: receive patient scan data associated with a body of a patient, wherein the patient scan data comprises data associated with a plurality of medical images of the body of the patient, wherein the data associated with the plurality of medical images of the body of the patient comprises one or more locations on the body of the patient marked as a target site; automatically generate a surface map of the body of the patient based on the patient scan data; determine one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site; and generate data associated with a medical procedure plan for a medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient.

2. The system of claim 1 , wherein the at least one processor is further configured to: perform the medical procedure on the patient using a robotic surgical tool based on the data associated with the medical procedure plan.

3. The system of claim 2, wherein, when performing the medical procedure on the patient using the robotic surgical tool, the at least one processor is configured to: perform the medical procedure on the patient using a robotically controlled surgical arm.

4. The system of any of claims 1 -3, wherein the at least one processor is further configured to: determine a tissue type of the one or more locations on the surface map of the body of the patient; andwherein, when generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient, the at least one processor is configured to: generate the data associated with the medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient and the tissue type associated with the one or more locations on the surface map of the body of the patient.

5. The system of claim 4, wherein the at least one processor is further configured to: identify one or more locations of landmarks on the body of the patient in relation to the one or more locations on the body of the patient marked as a target site; wherein, when generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient, the at least one processor is configured to: generate the data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient, the tissue type associated with the one or more locations on the surface map of the body of the patient, and the one or more locations of landmarks on the body of the patient.

6. The system of any of claims 1-5, wherein, when receiving the patient scan data associated with the body of the patient, the at least one processor is configured to: receive the patient scan data from a computerized tomography (CT) scan device or a magnetic resonance imaging (MRI) device.

7. The system of any of claims 1-6, wherein, when automatically generating the surface map of the body of the patient, the at least one processor is configured to:automatically generate a three-dimensional (3D) surface map of the body of the patient; and wherein, when determining the one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site, the at least one processor is configured to: determine 3D coordinates for each of the one or more locations on the 3D surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site.

8. The system of any of claims 1 -7, further comprising: a robotic surgical tool; and wherein the at least one processor is coupled to the robotic surgical tool.

9. A method for machine learning based medical procedure planning comprising: receiving, with at least one processor, patient scan data associated with a body of a patient, wherein the patient scan data comprises data associated with a plurality of medical images of the body of the patient, wherein the data associated with the plurality of medical images of the body of the patient comprises one or more locations on the body of the patient marked as a target site; automatically generating, with at least one processor, a surface map of the body of the patient based on the patient scan data; determining, with at least one processor, one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site; and generating, with at least one processor, data associated with a medical procedure plan for a medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient.

10. The method of claim 9, further comprising: performing the medical procedure on the patient using a robotic surgical tool based on the data associated with the medical procedure plan.

11. The method of claim 10, wherein performing the medical procedure on the patient using the robotic surgical tool comprises: performing the medical procedure on the patient using a robotically controlled surgical arm.

12. The method of claim 10, further comprising: determining a tissue type of the one or more locations on the surface map of the body of the patient; and wherein generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient comprises: generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient and the tissue type associated with the one or more locations on the surface map of the body of the patient.

13. The method of claim 12, further comprising: identifying one or more locations of landmarks on the body of the patient in relation to the one or more locations on the body of the patient marked as a target site; wherein generating the data associated with the medical procedure plan for the medical procedure to be performed on the patient comprises: generating the data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient, the tissue type associated with the one or more locations on the surface map of the body of the patient, and the one or more locations of landmarks on the body of the patient.

14. The method of any of claims 9-13, wherein receiving the patient scan data associated with the body of the patient comprises: receiving the patient scan data from a computerized tomography (CT) scan device or a magnetic resonance imaging (MRI) device.

15. The method of any of claims 9-14, wherein automatically generating the surface map of the body of the patient comprises: automatically generating a three-dimensional (3D) surface map of the body of the patient; and wherein determining the one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site comprises: determining 3D coordinates for each of the one or more locations on the 3D surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site.

16. A computer program product for machine learning based medical procedure planning, the computer program product comprising at least one non- transitory computer-readable medium including one or more instructions that, when executed by at least one processor, cause the at least one processor to: receive patient scan data associated with a body of a patient, wherein the patient scan data comprises data associated with a plurality of medical images of the body of the patient, wherein the data associated with the plurality of medical images of the body of the patient comprises one or more locations on the body of the patient marked as a target site; automatically generate a surface map of the body of the patient based on the patient scan data; determine one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site; and generate data associated with a medical procedure plan for a medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient.

17. The computer program product of claim 16, wherein the one or more instructions further cause the at least one processor to: perform the medical procedure on the patient using a robotic surgical tool based on the data associated with the medical procedure plan.

18. The computer program product of claim 17, wherein the one or more instructions that cause the at least one processor to perform the medical procedure on the patient using the robotic surgical tool, cause the at least one processor to: perform the medical procedure on the patient using a robotically controlled surgical arm.

19. The computer program product of any of claims 16-18, wherein the one or more instructions further cause the at least one processor to: determine a tissue type of the one or more locations on the surface map of the body of the patient; and wherein the one or more instructions that cause the at least one processor to generate the data associated with the medical procedure plan for the medical procedure to be performed on the patient, cause the at least one processor to: generate the data associated with the medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient and the tissue type associated with the one or more locations on the surface map of the body of the patient.

20. The computer program product of claim 19, wherein the one or more instructions further cause the at least one processor to: identify one or more locations of landmarks on the body of the patient in relation to the one or more locations on the body of the patient marked as a target site; wherein the one or more instructions that cause the at least one processor to generate the data associated with the medical procedure plan for the medical procedure to be performed on the patient, cause the at least one processor to: generate the data associated with a medical procedure plan for the medical procedure to be performed on the patient based on the one or more locations on the surface map of the body of the patient, the tissue type associated with the one or more locations on the surface map of the body ofthe patient, and the one or more locations of landmarks on the body of the patient.

21. The computer program product of any of claims 16-20, wherein the one or more instructions that cause the at least one processor to receive the patient scan data associated with the body of the patient, cause the at least one processor to: receive the patient scan data from a computerized tomography (CT) scan device or a magnetic resonance imaging (MRI) device.

22. The computer program product of any of claims 16-21 , wherein the one or more instructions that cause the at least one processor to automatically generate the surface map of the body of the patient cause the at least one processor to: automatically generate a three-dimensional (3D) surface map of the body of the patient; and wherein the one or more instructions that cause the at least one processor to determine the one or more locations on the surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site cause the at least one processor to: determine 3D coordinates for each of the one or more locations on the 3D surface map of the body of the patient that correspond to the one or more locations on the body of the patient marked as a target site.

Images