Reconstructing a Super-Second-Order Neural Network using an Anatomical Structure Mapping Engine

Abstract

A method is provided. This method is performed by a mapping engine. The mapping engine includes processor-executable code stored in memory and executed by a processor. The method includes acquiring the catheter trajectory in real time during an ablation procedure and training a pre-trained neural network based on a dataset and the catheter trajectory to provide a trained neural network. The method includes approximating the atrial shape using the trained neural network and portions of the catheter transverse path and generating a three-dimensional model output from the trained neural network and the atrial shape. The method also includes displaying the three-dimensional model output as early visualization during an ablation procedure.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This application claims priority to U.S. Provisional Application No. 63 / 495,665, filed on 12 April 2023, the contents of which are incorporated herein by reference in their entirety.

[0002] (Field of invention) This invention relates to signal processing. More specifically, this invention relates to anatomical mapping for signal processing, such as the reconstruction of a super-second-order neural network using an anatomical structure mapping engine. [Background technology]

[0003] Currently, catheter-based radiofrequency (RF) ablation for pulmonary vein isolation is the first-line treatment for atrial fibrillation (AF). RF ablation requires a very precise map of the subendocardial surface of the left atrium, including the pulmonary vein orifice.

[0004] For example, an electroanatomical mapping (EAM) system tracks the movement of a catheter through the body to create an anatomical surface. To evaluate the efficiency of the EAM system, it was applied to 25 patients undergoing pulmonary vein isolation (PVI) procedures. Fast anatomical mapping (FAM) points were defined as the average catheter position within a 1-second breathing gate window, and the surface was reconstructed using the alpha-shape algorithm. The mapping process took an average of 9 minutes, with a variation of plus or minus 3 minutes (±3). Surface accuracy was compared to MRI scans, yielding an average distance of 3.46 ± 0.02 mm and a vein isolation success rate of 96%. However, the quality of the FAM after 10 minutes was often insufficient, sometimes requiring physicians to take more than 20 minutes to complete the mapping and potentially necessitating manual shape editing steps. [Overview of the project] [Problems that the invention aims to solve]

[0005] There is a great need for solutions for faster and more accurate mapping procedures. [Means for solving the problem]

[0006] A method is provided by exemplary embodiments. The method is carried out by a mapping engine. The mapping engine includes processor-executable code stored in memory and executed by at least one processor. The method includes acquiring one or more catheter trajectories in real time during an ablation procedure and training a pre-trained neural network based on a dataset and one or more catheter trajectories to provide a trained neural network. The method includes approximating the atrial shape using the trained neural network and one or more portions of the transverse catheter trajectory and generating a three-dimensional model output from the trained neural network and the atrial shape. The method also includes displaying the three-dimensional model output as early visualization during an ablation procedure.

[0007] According to one or more embodiments, the exemplary methods described above can be implemented as devices, systems, and / or computer program products. [Brief explanation of the drawing]

[0008] A more detailed understanding can be obtained from the following explanation, which is given as an example in conjunction with the attached drawings, where similar reference numbers in the drawings indicate similar elements. [Figure 1] This describes exemplary catheter-based electrophysiological mapping and ablation systems in one or more embodiments. [Figure 2] This is a block diagram of an exemplary system for remotely monitoring and communicating patient biometric data, according to one or more embodiments. [Figure 3]A system diagram of an example of a computing environment that communicates with a network according to one or more embodiments. [Figure 4A] A block diagram of an exemplary device that can implement one or more features of the present disclosure according to one or more embodiments. [Figure 4B] A graphic depiction of an artificial intelligence system incorporating the exemplary device of FIG. 4A according to one or more embodiments. [Figure 5] A method implemented in the artificial intelligence system of FIG. 4B is depicted based on one or more embodiments. [Figure 6] An exemplary neural network according to one or more embodiments is depicted. [Figure 7] A method according to one or more embodiments is depicted. [Figure 8] A system according to one or more embodiments is depicted. [Figure 9] A transformation according to one or more embodiments is depicted. [Figure 10] A diagram showing a model for shape configuration induction according to one or more embodiments. [Figure 11] A table according to one or more embodiments is depicted. [Figure 12] A model according to one or more embodiments is depicted. [Figure 13] A model according to one or more embodiments is depicted. [Figure 14] A graph according to one or more embodiments is depicted. [Figure 15] A table according to one or more embodiments is depicted. [Figure 16] A reconstructed surface according to one or more embodiments is depicted. [Figure 17] A reconstructed surface according to one or more embodiments is depicted. [Figure 18A] Clinical cases according to one or more embodiments are depicted. [Figure 18B] Clinical cases according to one or more embodiments are depicted. [Figure 19]Depicts a 3D reconstruction using one or more embodiments. [Figure 20] Depicts a 3D reconstruction using one or more embodiments. [Figure 21] Describes a reconstruction using one or more embodiments. [Modes for carrying out the invention]

[0009] This specification discloses methods and / or systems for anatomical mapping. More specifically, the methods and systems relate to a mapping engine that can perform anatomical mapping using machine learning / artificial intelligence (ML / AI), such as by utilizing the reconstruction of anatomical structures using a super-secondary neural network.

[0010] A mapping engine (including any AI / ML algorithms) is processor-executable code or software that can be executed by the processing hardware of any technically feasible device, such as a medical device. For simplicity of explanation, the mapping engine is described herein in relation to mapping the heart, however, any anatomical structure, body part, organ, or part thereof can be targeted for mapping by the mapping engine described herein. According to an exemplary embodiment, the mapping engine generates a map of the endocardial surface of the left atrium (LA) using a portion of the catheter transverse route (e.g., the initial directional route). The map may include one or more three-dimensional (3D) models. For example, in contrast to conventional mapping techniques, the mapping engine focuses on using rapidly acquired catheter routes that traverse only anatomical landmarks, the majority of which are in the blood pool and do not require contact with the surface.

[0011] One or more advantages, technical effects, and / or benefits of a mapping engine that generates maps include providing early visualization, particularly for pulmonary veins and important anatomical parts. Early visualization can guide catheters, such as single-shot catheters, that have surfaces that are difficult to sample. Thus, a mapping engine can reduce cognitive load, simplify procedures, and enable less experienced surgeons to achieve better results. A mapping engine can provide accurate imaging of hard-to-reach areas of the atrium, such as the ridge between the left superior vein and the left atrial appendage, and can improve catheter navigation and visualization of tissue contact during ablation procedures.

[0012] For example, atrial fibrillation (AF) is a common form of arrhythmia in humans, affecting millions of people worldwide each year. AF is associated with an increased risk of embolic stroke and a reduced quality of life. Catheter-based electroanatomical mapping (EAM) with 3D-guided radiofrequency ablation for pulmonary vein isolation (PVI) is rapidly becoming the first-line treatment for AF. The EAM system records the catheter's position and electrical signals as the catheter moves through the cardiac chambers. The EAM system uses the position and electrical signal data to reconstruct the endocardial surface and approximate the electrical waves that cause cardiac chamber contraction. The EAM system generates a visualization of the endocardial surface anatomical structures of the left atrium ("LA"), including the pulmonary veins (PV) and their anatomical parts (e.g., LS - upper left, RS - upper right, LI - lower left, RI - lower right, LAA - left atrial appendage). Accurate mapping of the LA surface by the EAM system requires extensive catheter manipulation, which is time-consuming and requires a skilled physician. The boundary extraction process, known as fast anatomical mapping (FAM), involves a catheter making contact with a large portion of the boundary to extract many surface points. In some cases, local electrophoresis, force displays, and other indicators are used to ensure the catheter is in the correct position.

[0013] Furthermore, anatomical imaging methods such as magnetic resonance imaging (MRI) and intracardiac ultrasound catheterization can capture the LA surface by segmenting it from the acquired image data. Anatomical imaging methods must consider acquisition time, radiation exposure, limited field of view, noise and contrast, as well as deformation of cardiac shape due to respiration, heartbeat, and posture. The segmentation approach involves converting the acquired image into a 3D map showing the probability that each point belongs to a tissue or blood pool. The segmentation must be smooth and adhere to prior knowledge of various anatomical details.

[0014] EAM systems and other anatomical imaging methods are not only insufficient to generate sufficiently accurate maps, but there are currently no techniques to reduce mapping time while maintaining anatomical accuracy. In addition, because physicians can use different workflows, catheters, and imaging guidance methods depending on their preferences and skills, as well as on arrhythmias, available systems and catheters, proficiency, and site constraints, EAM systems and other anatomical imaging methods are limited in their ability to accommodate the many variations of real-world ablation procedures for AF. The mapping engine herein provides a solution by including a dense encoder-decoder network with a regularization term to reconstruct the shape of the left atrium from partial data derived from catheter manipulation.

[0015] Refer to Figure 1, which shows an exemplary system shown as System 10 (e.g., a medical device and / or catheter-based electrophysiological mapping and ablation system) in which one or more features of the subject matter of this specification can be implemented according to one or more embodiments. All or part of System 100 may be used to collect information (e.g., biometric data and / or training datasets) and / or to implement machine learning and / or artificial intelligence algorithms (e.g., mapping engines) as described herein. The mapping engine 101 (e.g., including any internal AI / ML algorithms) is processor-executable code or software stored in the memory of System 10 and necessarily rooted in the process operation by System 10 and the processing hardware of System 10. In various examples, the mapping engine 101 may be or include such software, or hardware (e.g., a fixed-function or programmable processor), or a combination thereof.

[0016] System 10, as illustrated, includes a recorder 11, a heart 12, a catheter 14, a model or anatomical map 20, an electrograph 21, a spline 22, a patient 23, a physician 24 (representing any medical professional, technician, or clinician), a location pad 25, one or more electrodes 26, a display device 27, a distal tip 28, a sensor 29, a coil 32, a patient interface unit (PIU) 30, an electrode skin patch 38, an ablation energy generator 50, and a workstation 55. It should be further noted that each element and / or item of System 10 represents one or more of its elements and / or items. The embodiment disclosed herein can be carried out by modifying the example of System 10 shown in Figure 1. Embodiments of this disclosure can also be applied similarly using other system components and settings. Additionally, System 10 may include additional components such as elements for sensing electrical activity, wired or wireless connectors, processing devices, and display devices.

[0017] System 10 includes a plurality of catheters 14 that are percutaneously inserted by a physician 24 into the cardiac chambers or vascular structures of the heart 12 through the patient's vascular system. Typically, a delivery sheath catheter is inserted into the left or right atrium near a desired location within the heart 12. The plurality of catheters can then be inserted into the delivery sheath catheter to reach the desired location. The plurality of catheters 14 may include a catheter dedicated to sensing intracardiac electrogram (IEGM) signals, a catheter dedicated to ablation, and / or a catheter dedicated to both sensing and ablation. An exemplary catheter 14 configured to sense IEGM is illustrated herein. To sense a target site in the heart 12, the physician 24 brings the distal end 28 of the catheter 14 into contact with the heart wall. For ablation, the physician 24 similarly brings the distal end of the ablation catheter to the target site for ablation.

[0018] The catheter 14 is an exemplary catheter comprising at least one, preferably multiple, electrodes 26 optionally distributed across a plurality of splines 22 at the distal tip 28 and configured to sense IEGM signals. The catheter 14 may additionally include a sensor 29 embedded in or near the distal tip 28 to track the position and orientation of the distal tip 28. Optionally and preferably, the position sensor 29 is a magnetic-based position sensor comprising three magnetic coils for sensing 3D position and orientation. According to one or more embodiments, the shape and parameters of the catheter 14 vary based on whether the catheter 14 is used for diagnostic or ablation purposes, the type of arrhythmia, the patient's anatomical structure, and other factors affecting the maneuverability of the catheter (e.g., the ability to touch the surface and tracked portion of the catheter 14 without bending). The shape and parameters of the catheter 14 also affect the accuracy of the anatomical map. Large, spherical single-shot catheters that can ablate pulmonary veins within seconds are popular but require guidance from fluoroscopy, CT / MRI, or additional mapping catheters. The mapping engine solves the shortcomings of catheter 14 as described herein.

[0019] A sensor 29 (e.g., a position-based or magnetic-based position sensor) may work in conjunction with a place pad 25 which includes a plurality of magnetic coils 32 configured to generate a magnetic field within a given working volume. The real-time position of the distal tip 28 of the catheter 14 may be tracked based on the magnetic field generated by the place pad 25 and sensed by the sensor 29. Details of magnetic-based position sensing technology are described in U.S. Patents 5,5391,199, 5,443,489, 5,558,091, 6,172,499, 6,239,724, 6,332,089, 6,484,118, 6,618,612, 6,690,963, 6,788,967, and 6,892,091.

[0020] System 10 includes one or more electrode patches 38 positioned on the patient 23 for skin contact to establish location reference of the location pad 25 and impedance-based tracking of the electrodes 26. For impedance-based tracking, a current is directed to the electrodes 26 and sensed in the patch 38 (e.g., an electrode skin patch), thereby allowing the location of each electrode to be triangulated through the patch 38. Details of impedance-based location tracking techniques are described in U.S. Patents 7,536,218, 7,756,576, 7,848,787, 7,869,865, and 8,456,182, which are incorporated herein by reference.

[0021] The recorder 11 displays the electrocardiogram 21 captured by the electrode 18 (e.g., an electrocardiogram (ECG) electrode) and the intracardiac electrocardiogram (IEGM) captured by the electrode 26 of the catheter 14. The recorder 11 may include pacing capabilities for pacing the heart rhythm and / or may be electrically connected to a standalone pacer.

[0022] System 10 may include an ablation energy generator 50 adapted to deliver ablation energy to one or more electrodes 26 located at the distal tip 28 of a catheter 14 configured for ablation. The energy produced by the ablation energy generator 50 may include, but is not limited to, radio frequency (RF) energy or pulsed-field ablation (PFA) energy, or a combination thereof, including unipolar or bipolar high-voltage DC pulses that may be used to induce irreversible electroporation (IRE).

[0023] The PIU 30 is an interface configured to establish electrical communication between the catheter, the electrophysiological equipment, the power supply, and the workstation 55 that controls the operation of the system 10. The electrophysiological equipment of the system 10 may include, for example, multiple catheters 14, a location pad 25, a body surface ECG electrode 18, an electrode patch 38, an ablation energy generator 50, and a recorder 11. Optionally, and preferably, the PIU 30 additionally includes processing capabilities for implementing real-time calculation of the catheter location and performing ECG calculations.

[0024] The workstation 55 includes memory, a processor unit having memory or storage loaded with appropriate operating software, and user interface functions (for example, the memory or storage of the workstation 55 stores the mapping engine 101, and the processor unit of the workstation 55 runs the mapping engine 101). The workstation 55 may optionally provide several functions, including (1) modeling the endocardial anatomical structure in three dimensions (3D) and rendering the model or anatomical map 20 for display on a display device 27; (2) displaying the activation sequence (or other data) compiled from the recorded electrophoresis diagram 21 on the display device 27 as representative visual indicators or images superimposed on the rendered anatomical map 20; (3) displaying the real-time position and orientation of multiple catheters within the cardiac chambers; and (5) displaying sites of interest on the display device 27, such as the locations where ablation energy has been applied. One commercially available product that embodies the elements of System 10 is the CARTO® 3 system, available from Biosense Webster, Inc. (31A Technology Drive, Irvine, CA 92618).

[0025] For example, system 10 can be part of a surgical system (e.g., the CARTO® system sold by Biosense Webster) configured to acquire biometric data (e.g., anatomical and electrical measurements of a patient's organs, such as the heart 12, as described herein) and perform cardiac ablation procedures. More specifically, in the treatment of cardiac conditions such as cardiac arrhythmias, it is often required to obtain detailed mapping of cardiac tissue, chambers, veins, arteries, and / or electrical pathways. For example, a prerequisite for successfully performing catheter ablation (as described herein) is that the cause of the cardiac arrhythmia is precisely located in the chambers of the heart 12. Such localization can be performed by electrophysiological examination, during which spatially resolved potentials are detected by a mapping catheter (e.g., catheter 14) introduced into the chambers of the heart 12. Thus, this electrophysiological examination, so-called electroanatomical mapping, provides 3D mapping data that can be displayed on a display device 27. In many cases, mapping and therapeutic functions (e.g., ablation) are provided by a single catheter or a group of catheters, and as a result, the mapping catheter also functions as a therapeutic (e.g., ablation) catheter.

[0026] Figure 2 is a block diagram of an exemplary system 100 for remotely monitoring and communicating patient biometrics (i.e., patient data). In the example illustrated in Figure 2, system 100 includes a patient biometric monitoring and processing unit 102 associated with patient 104, a local computing device 106, a remote computing system 108, a first network 110, a patient biometric sensor 112, a processor 114, a user input (UI) sensor 116, a memory 118, a second network 120, and a transmitter-receiver (i.e., transceiver) 122.

[0027] According to one embodiment, the patient biometric measurement and monitoring device 102 may be a device located inside the patient's body (e.g., subcutaneously implantable), such as the catheter 14 in Figure 1. The patient biometric measurement and monitoring device 102 may be inserted into the patient via any applicable method, including oral injection, surgical insertion via vein or artery, endoscopic procedure, or laparoscopic procedure.

[0028] According to one embodiment, the patient biometric monitoring and processing device 102 may be an external device to the patient, such as the electrode patch 38 in Figure 1. For example, as will be described in more detail below, the patient biometric monitoring and processing device 102 may include an attachable patch (e.g., attached to the patient's skin). The monitoring and processing device 102 may also include a catheter, probe, blood pressure cuff, scale, bracelet or smartwatch biometric tracker having one or more electrodes, a glucose monitor, a continuous positive airway pressure (CPAP) machine, or substantially any device capable of providing input regarding the patient's health or biometric indicators.

[0029] According to one embodiment, the patient biometric measurement monitoring and processing device 102 may include both components located inside the patient and components located outside the patient.

[0030] A single patient biometric monitoring and processing unit 102 is shown in Figure 2. However, the exemplary system may include multiple patient biometric monitoring and processing units. A patient biometric monitoring and processing unit may communicate with one or more other patient biometric monitoring and processing units. Additionally or alternatively, a patient biometric monitoring and processing unit may communicate with a network 110.

[0031] One or more patient biometric monitoring and processing devices 102 can acquire patient biometric data (e.g., electrical signals, blood pressure, body temperature, blood glucose levels, or other biometric data) and can receive at least a portion of the patient biometric data representing the acquired patient biometric values, as well as additional information associated with the patient biometric values ​​acquired from one or more other patient biometric monitoring and processing devices 102. The additional information may be, for example, diagnostic information and / or additional information obtained from additional devices such as wearable devices. Each patient biometric monitoring and processing device 102 can process data including its own acquired patient biometric values ​​and data received from one or more other patient biometric monitoring and processing devices 102.

[0032] Biometric data (e.g., patient biometrics, patient data, or patient biometric data) may include one or more of the following: local activation time (LAT), electrical activity, topology, bipolar mapping, baseline activity, ventricular activity, dominant frequency, impedance, etc. LAT may be the time of threshold activity corresponding to local activation, calculated based on a normalized initial start point. Electrical activity may be any applicable electrical signal that can be measured based on one or more thresholds and can be sensed and / or augmented based on the signal-to-noise ratio and / or other filters. Topology may correspond to the physical structure of a body part or a part of a body part, and may correspond to variations in the physical structure of different parts of a body part or different parts of a body part. Dominant frequency may be a frequency or range of frequencies commonly found in a part of a body part and may differ in different parts of the same body part. For example, the dominant frequency of the PV in a heart may differ from the dominant frequency of the right atrium of the same heart. Impedance may be a resistance measurement in a given region of a body part.

[0033] Examples of biometric data include, but are not limited to, patient identification data, intracardiac electrocardiogram (IC ECG) data, bipolar intracardiac reference signals, anatomical and electrical measurements, trajectory information, body surface (BS) ECG data, historical data, brain biometrics, blood pressure data, ultrasound signals, radio signals, voice signals, two-dimensional or three-dimensional image data, blood glucose data, and temperature data. Biometric data can generally be used to monitor, diagnose, and treat any number of different diseases, such as cardiovascular diseases (e.g., arrhythmias, cardiomyopathy, and coronary artery disease) and autoimmune diseases (e.g., type 1 and type 2 diabetes). Note that BS ECG data may include data and signals collected from electrodes on the patient's surface, IC ECG data may include data and signals collected from electrodes inside the patient's body, and ablation data may include data and signals collected from the tissue being ablated. Furthermore, BS ECG data, IC ECG data, and ablation data can be derived from one or more procedure records, along with catheter electrode position data.

[0034] In Figure 2, network 110 is an example of a short-range network (e.g., a local area network (LAN) or a personal area network (PAN)). Information can be transmitted between the patient vital signs monitoring and processing device 102 and the local computing device 106 via network 110 using one of various short-range wireless communication protocols such as Bluetooth, Wi-Fi, Zigbee, Z-Wave, near-field communication (NFC), ultra-wideband wireless, Zigbee, or infrared (IR).

[0035] Network 120 may be a wired network, a wireless network, or may include one or more wired and wireless networks. For example, network 120 may be a long-range network (e.g., a wide area network (WAN), the Internet, or a cellular network). Information may be transmitted over network 120 using any one of various long-range wireless communication protocols (e.g., TCP / IP, HTTP, 3G, 4G / LTE, or 5G / New Radio).

[0036] The patient biometric measurement monitoring and processing device 102 may include a patient biometric sensor 112, a processor 114, a UI sensor 116, a memory 118, and a transceiver 122. The patient biometric measurement monitoring and processing device 102 may continuously or periodically monitor, store, process, and communicate any number of different patient biometric measurements via the network 110. Examples of patient biometric measurements include electrical signals (e.g., ECG signals and brain biometric measurements), blood pressure data, blood glucose data, and body temperature data. Patient biometric measurements may be monitored and communicated for therapeutic purposes across any number of different diseases, such as cardiovascular diseases (e.g., arrhythmias, cardiomyopathy, and coronary artery disease), and autoimmune diseases (e.g., type 1 and type 2 diabetes).

[0037] The patient biometric sensor 112 may include, for example, one or more sensors configured to sense the type of biometric value of a biometric patient. For example, the patient biometric sensor 112 may include electrodes configured to acquire electrical signals (e.g., cardiac signals, brain signals, or other bioelectrical signals), a body temperature sensor, a blood pressure sensor, a blood glucose sensor, a blood oxygen sensor, a pH sensor, an accelerometer, and a microphone.

[0038] As will be described in more detail below, the patient biometric monitoring and processing device 102 may be an ECG monitor for monitoring the ECG signal of the heart (e.g., heart 12). The patient biometric sensor 112 of the ECG monitor may include one or more electrodes for acquiring the ECG signal. The ECG signal can be used for the treatment of various cardiovascular diseases.

[0039] In another example, the patient biometric monitoring and processing device 102 may be a continuous glucose monitor (CGM) for continuously monitoring a patient's blood glucose levels to treat various diseases such as type 1 and type 2 diabetes. The CGM may include subcutaneously placed electrodes that can monitor blood glucose levels from the patient's interstitial fluid. The CGM may be a component of a closed-loop system in which blood glucose data is sent to an insulin pump for, for example, calculated insulin delivery without user intervention.

[0040] The transceiver 122 may include separate transmitters and receivers. Alternatively, the transceiver 122 may include transmitters and receivers integrated into a single device.

[0041] The processor 114 may be configured to store patient data, such as patient biometric data acquired by the patient biometric sensor 112, in the memory 118 and to communicate the patient data over the network 110 via the transmitter of the transceiver 122. Data from one or more other patient biometric monitoring and processing devices 102 may also be received by the receiver of the transceiver 122, as will be described in more detail below.

[0042] According to one embodiment, the patient biometric measurement monitoring and processing device 102 includes a UI sensor 116, which may be a piezoelectric or capacitive sensor configured to receive user input such as a tap or touch. For example, the UI sensor 116 may be controlled to implement capacitive coupling in response to a patient 104 tapping or touching the surface of the patient biometric measurement monitoring and processing device 102. Gesture recognition may be performed via any one of various capacitive types, such as resistive capacitive, surface capacitive, projected capacitive, surface acoustic wave, piezoelectric, and infrared touch. The capacitive sensor may be positioned over a small area or length on the surface so that a tap or touch on the surface activates the monitoring device.

[0043] As will be described in more detail below, the processor 114 may be configured to selectively respond to different tapping patterns (e.g., single tap or double tap) of a capacitive sensor, which may be a UI sensor 116, and as a result, different tasks of the patch (e.g., data acquisition, storage, or transmission) may be triggered based on the detected pattern. In some embodiments, when a gesture is detected, audible feedback may be provided to the user from the patient biometric monitoring and processing device 102.

[0044] The local computing device 106 of system 100 may be configured to communicate with the patient biometrics monitoring and processing device 102 and to function as a gateway to the remote computing system 108 via the second network 120. The local computing device 106 may be, for example, a smartphone, smartwatch, tablet, or other portable smart device configured to communicate with other devices via the network 120. Alternatively, the local computing device 106 may be a fixed or standalone device, such as a fixed base station including modem and / or router capabilities, a desktop or laptop computer using an executable program to communicate information between the patient biometrics monitoring and processing device 102 and the remote computing system 108 via a wireless module of a PC, or a USB dongle. Patient biometrics may be communicated between the local computing device 106 and the patient biometrics monitoring and processing device 102 via a short-range wireless network 110, such as a local area network (LAN) (e.g., a personal area network (PAN)), using short-range wireless technology standards (e.g., Bluetooth, Wi-Fi, ZigBee, Z-wave, and other short-range wireless standards). In some embodiments, the local computing device 106 may also be configured to display acquired patient electrical signals and information associated with those signals, as will be described in more detail below.

[0045] In some embodiments, the remote computing system 108 may be configured to receive at least one of monitored patient biometrics and information associated with the monitored patient via a long-range network, which is a network 120. For example, if the local computing device 106 is a mobile phone, the network 120 may be a wireless cellular network, and information may be communicated between the local computing device 106 and the remote computing system 108 via a wireless technology standard, such as one of the wireless technologies described above. As will be described in more detail below, the remote computing system 108 may be configured to provide (e.g., visually and / or audibly) at least one of the patient's biometrics and associated information to a medical professional (e.g., a physician).

[0046] Figure 3 is a system diagram of an example computing environment 200 communicating with network 120. In some examples, the computing environment 200 is integrated into a public cloud computing platform (such as Amazon Web Services or Microsoft Azure), a hybrid cloud computing platform (such as HP Enterprise OneSphere), or a private cloud computing platform.

[0047] As shown in Figure 3, the computing environment 200 includes a remote computing system 108 (hereinafter referred to as the computer system), which is one example of a computing system in which embodiments described herein may be implemented.

[0048] The remote computing system 108 can perform a variety of functions via a processor 220 which may include one or more processors. These functions may include analyzing biometric indicators and associated information of a monitored patient, and providing warnings, additional information, or instructions (e.g., via a display 266) according to thresholds and parameters determined by a physician or algorithmically driven. As will be described in more detail below, the remote computing system 108 can be used to provide a patient information dashboard (e.g., via a display 266) to a healthcare professional (e.g., a physician), which may enable the healthcare professional to identify and prioritize patients with more critical needs than other patients.

[0049] As shown in Figure 3, the computer system 210 may include a communication mechanism such as a bus 221, or other communication mechanisms for communicating information within the computer system 210. The computer system 210 further includes one or more processors 220 coupled to the bus 221 for processing information. The processors 220 may include one or more CPUs, GPUs, or any other processors known in the art.

[0050] The computer system 210 also includes system memory 230 coupled to bus 221 for storing information and instructions executed by processor 220. System memory 230 may include computer-readable storage media in the form of volatile and / or non-volatile memory, such as read-only memory (ROM) 231 and / or random access memory (RAM) 232. System memory RAM 232 may include other dynamic storage devices (e.g., dynamic RAM, static RAM, and synchronous DRAM). System memory ROM 231 may include other static storage devices (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, system memory 230 may be used to store temporary variables or other intermediate information during instruction execution by processor 220. A basic input / output system (BIOS) 233 may include routines for transferring information that can be stored in system memory ROM 231 between elements within the computer system 210, such as at startup. RAM 232 may include data and / or program modules that are immediately accessible to the processor 220 and / or currently being manipulated by the processor 220. System memory 230 may additionally include, for example, an operating system 234, application programs 235, other program modules 236, and program data 237.

[0051] The illustrated computer system 210 also includes a disk controller 240 coupled to a bus 221 for controlling one or more storage devices for storing information and instructions, such as a magnetic hard disk 241 and a removable media drive 242 (e.g., a floppy disk drive, a compact disk drive, a tape drive, and / or a solid-state drive). Storage devices may be added to the computer system 210 using a suitable device interface (e.g., small computer system interface, SCSI, integrated device electronics, IDE, Universal Serial Bus, USB, or FireWire).

[0052] The computer system 210 may also include a display controller 265 coupled to a bus 221 for controlling a monitor or display 266, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. The illustrated computer system 210 includes a user input interface 260 and one or more input devices, such as a keyboard 262 and a pointing device 261, for interacting with a computer user and providing information to a processor 220. The pointing device 261 may be, for example, a mouse, trackball, or pointing stick for communicating instruction information and command selections to the processor 220 and controlling cursor movement on the display 266. The display 266 may provide a touchscreen interface, which may allow input that complements or replaces the communication of instruction information and command selections by the pointing device 261 and / or the keyboard 262.

[0053] The computer system 210 may perform some or each of the functions and methods described herein in response to a processor 220 that executes one or more sequences of one or more instructions contained in memory, such as system memory 230. Such instructions can be read into system memory 230 from another computer-readable medium, such as a hard disk 241 or a removable media drive 242. The hard disk 241 may include one or more data stores and data files used by the embodiments described herein. The data store contents and data files may be encrypted to improve security. The processor 220 may also be employed in a multi-processing configuration to execute one or more sequences of instructions contained in system memory 230. In alternative embodiments, hardwired circuitry may be used instead of or in combination with software instructions. Thus, embodiments are not limited to any particular combination of hardware circuitry and software.

[0054] As described above, the computer system 210 may include at least one computer-readable medium or memory for holding instructions programmed according to the embodiments described herein and for containing data structures, tables, records, or other data described herein. As used herein, the term computer-readable medium refers to any non-temporary tangible medium involved in providing instructions to the processor 220 for execution. Computer-readable mediums can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-limited examples of non-volatile media include optical disks, solid-state drives, magnetic disks, and magneto-optical disks, such as the hard disk 241 or removable media drive 242. Non-limited examples of volatile media include dynamic memory, such as the system memory 230. Non-limited examples of transmission media include coaxial cables, copper wires, and optical fibers, such as the wires that make up the bus 221. Transmission media can also take the form of acoustic waves or light waves, such as those generated during radio and infrared data communications.

[0055] The computing environment 200 may further include a computer system 210 operating in a networked environment using logical connections to a local computing device 106 and to one or more other devices such as a personal computer (laptop or desktop), a mobile device (e.g., a patient mobile device), a server, a router, a network PC, a peer device, or other common network node, and typically includes many or all of the elements described above with respect to the computer system 210. When used in a network environment, the computer system 210 may include a modem 272 for establishing communication over a network 120 such as the Internet. The modem 272 may be connected to the system bus 221 via a network interface 270 or via another suitable mechanism.

[0056] Network 120, as shown in Figures 2 and 3, may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or a series of connections, a cellular telephone network, or any other network or medium that can facilitate communication between computer system 610 and other computers (e.g., local computing device 106).

[0057] Figure 4A is a block diagram of an exemplary device 300 that can implement one or more features of the present disclosure. Device 300 may be, for example, a local computing device 106. Device 300 may include, for example, a computer, a game device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. Device 300 includes a processor 302, memory 304, a storage device 306, one or more input devices 308, and one or more output devices 310. Device 300 may also optionally include an input driver 312 and an output driver 314. It is understood that device 300 may include additional components not shown in Figure 4A, including an artificial intelligence accelerator.

[0058] In various alternatives, the processor 302 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, each of which may be a CPU or a GPU. In various alternatives, the memory 304 is located on the same die as the processor 302 or is located separately from the processor 302. The memory 304 includes volatile memory or non-volatile memory, such as random access memory (RAM), dynamic RAM, or cache.

[0059] The storage device 306 includes fixed or removable storage means, such as a hard disk drive, solid-state drive, optical disc, or flash drive. The input device 308 includes, but is not limited to, a keyboard, keypad, touchscreen, touchpad, detector, microphone, accelerometer, gyroscope, biometric scanner, or network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals). The output device 310 includes, but is not limited to, a display, speaker, printer, haptic feedback device, one or more lights, antenna, or network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals).

[0060] The input driver 312 communicates with the processor 302 and the input device 308, enabling the processor 302 to receive input from the input device 308. The output driver 314 communicates with the processor 302 and the output device 310, enabling the processor 302 to send output to the output device 310. Note that the input driver 312 and the output driver 314 are optional components, and if the input driver 312 and the output driver 314 are not present, device 300 will operate in the same manner. The output driver 316 includes an accelerated processing device ("APD") 316 coupled to the display device 318. The APD receives computation and graphic rendering commands from the processor 302, processes those computation and graphic rendering commands, and provides pixel outputs to the display device 318 for display. As will be described in more detail below, the APD 316 includes one or more parallel processing units that perform computations according to the single-instruction-multiple-data ("SIMD") paradigm. Therefore, although various functions are described herein as being performed by or in conjunction with the APD316, in various alternative examples, functions described as being performed by the APD316 are additionally or alternatively performed by other computing devices that are not driven by a host processor (e.g., processor 302) but have similar capabilities to provide graphical output to the display device 318. For example, any processing system that performs processing tasks according to the SIMD paradigm is intended to be able to perform the functions described herein. Alternatively, a computing system that does not perform processing tasks according to the SIMD paradigm is intended to perform the functions described herein.

[0061] Figure 4B illustrates a graphic depiction of an artificial intelligence system 200 incorporating the exemplary device shown in Figure 4A. System 400 includes data 410, a machine 420, a model 430, a set of outcomes 440, and underlying hardware 450. System 400 operates by training machine 420 with data 410 while building model 430 capable of predicting the set of outcomes 440. System 400 may operate relative to hardware 450. In such a configuration, data 410 may be related to hardware 450, for example, originating from patient biometric monitoring and processing device 102. For example, data 410 may be ongoing data or output data related to hardware 450. Machine 420 may operate as a controller or data acquisition associated with hardware 450, or may be associated with it. Model 430 may be configured to model the operation of hardware 450 and the data 410 collected from hardware 450 in order to predict the outcomes achieved by hardware 450. The hardware 450 may be configured to provide a predetermined desired outcome 440 from the hardware 450 using the predicted outcome 440.

[0062] Figure 5 illustrates a method 500 performed in the artificial intelligence system of Figure 4B. Method 500 includes, in step 510, collecting data from hardware. This data may include currently collected data, historical data, or other data from the hardware. For example, this data may include measurements taken during a surgical procedure and can be correlated with the outcome of the procedure. For example, the temperature of the heart (e.g., heart 12) may be collected and correlated with the outcome of the cardiac procedure.

[0063] Method 500 includes training a machine on hardware in step 520. The training may include analyzing and correlating the data collected in step 510. For example, in the case of the heart, temperature and outcome data may be trained to determine whether there is a correlation or association between the temperature of the heart during treatment and the outcome.

[0064] Method 500 includes, in step 530, building a model based on hardware-related data. Model building may include physical hardware or software modeling, algorithmic modeling, etc., as described below. This modeling may aim to represent the collected and trained data.

[0065] Method 500 includes predicting the outcome of a hardware-related model in step 540. This outcome prediction can be based on a trained model. For example, in the case of a heart, if a positive outcome is obtained from the procedure when the temperature during the procedure is between 97.7 and 100.2°C, then the outcome of a given procedure can be predicted based on the temperature of the heart during the procedure. This model is rudimentary but is provided for illustrative purposes to enhance the understanding of the present invention.

[0066] The mapping engine 101 described herein provides a solution for reconstructing the shape of the left atrium from partial data derived from catheterization procedures by including a dense encoder-decoder network with a regularization term. The mapping engine 101 can operate to train, construct a model, and predict outcomes using specific algorithms. Using the algorithms of the mapping engine 101, the trained model can be solved and hardware-related outcomes can be predicted. For example, the algorithms of the mapping engine 101 can generally be divided into classification, regression, and clustering algorithms.

[0067] For example, a classification algorithm is used to identify the class of a dependent variable. In other words, given a specific value of a dependent variable that can exist in one of a set of classes, a classification algorithm determines which class the dependent variable belongs to. Thus, classification algorithms are used to predict outcomes from a given number of fixed, predefined outcomes. Classification algorithms may include Naive Bayes algorithms, decision trees, random forest classifiers, logistic regression, support vector machines, and k-nearest neighbors.

[0068] Generally, the simple Bayes algorithm follows Bayes' theorem and employs a probabilistic approach. Other probability-based algorithms are also available and will generally be understood to operate using similar probabilistic principles as the exemplary simple Bayes algorithm described below.

[0069] Generally, a decision tree is a tree structure similar to a flowchart, where each outer node represents an attribute test, and each branch represents the result of that test. Leaf nodes contain the actual predicted labels ("classes"). In a decision tree, attribute values ​​are compared starting from the root of the tree and continuing until a leaf node is reached. Decision trees can be used as classifiers when dealing with high-dimensional data and when little time has been spent preparing the data. Decision trees can take the form of simple decision trees, linear decision trees, algebraic decision trees, deterministic decision trees, randomized decision trees, non-deterministic decision trees, and quantum decision trees.

[0070] A random forest classifier is a committee of decision trees, each given a subset of data attributes and making predictions based on that subset. The mode of the actual predictions of the decision trees is taken into account to provide the final random forest answer. Random forest classifiers are generally more robust and accurate by mitigating the overfitting that exists in standalone decision trees.

[0071] Logistic regression is another algorithm for binary classification tasks. Logistic regression is based on the logistic function, also known as the sigmoid function. This S-shaped curve can take any real value and be mapped between 0 and 1, asymptotically approaching these limits. The logistic model can be used to model the probability of a particular class or event existing, such as pass / fail, win / lose, survive / die, or healthy / sick. This can be extended to model several classes of events, such as determining whether an image contains cats, dogs, lions, etc. Each object detected in the image is assigned a probability between 0 and 1, and the sum of these probabilities is 1.

[0072] In a logistic model, the log odds (logarithm of odds) of a value labeled "1" is a linear combination of one or more independent variables ("predictors"), each of which can be a binary variable (two classes coded by an indicator variable) or a continuous variable (any real number). The value labeled "1" is so named because its corresponding probability can range between 0 (definitely the value "0") and 1 (definitely the value "1"), and the logistic function is so named because it is a function that transforms log odds into probabilities. The unit of measurement for the log odds scale is called a logit, another name for the logistic unit. Similar models can also be used, such as the probit model, where the sigmoid function is used instead of the logistic function. A key feature of the logistic model is that increasing one of the independent variables multiplicatively scales the odds of a given outcome at a constant rate, and each independent variable has its own parameters. In the case of a binary dependent variable, this generalizes the odds ratio.

[0073] In a binary logistic regression model, the dependent variable has two levels (categories). Outputs with three or more values ​​are modeled by multinomial logistic regression, and if the multiple categories are ordered, they are modeled by ordered logistic regression (e.g., proportional odds ordered logistic model). The logistic regression model itself models the probability of an output with respect to the input and does not perform statistical classification (it is not a classifier), but it can be used to create a classifier by, for example, choosing a cutoff value and classifying inputs with probabilities greater than the cutoff into one class and inputs with probabilities less than the cutoff into the other class. This is a common way to create a binary classifier.

[0074] A support vector machine (SVM) can be used to sort data by maximizing the margin between two classes. This is called margin-maximizing separation. Unlike linear regression, which uses the entire dataset for this purpose, SVMs can consider support vectors while plotting a hyperplane.

[0075] In regression algorithms, the output is a continuous quantity, so regression algorithms can be used when the target variable is a continuous variable. Linear regression is a common example of a regression algorithm. Linear regression can be used to measure true quality (such as housing cost, number of calls, or total transactions) by considering a consistent variable. The connection between the variable and the outcome is found by fitting an optimal line (hence the name linear regression). This optimal line is known as the regression line and is directly expressed as the condition Y = a × X + b. Linear regression is most often used in approaches with a small number of dimensions.

[0076] Clustering algorithms can be used to model and train datasets. In clustering, inputs are assigned to two or more clusters based on feature similarity. Clustering algorithms typically learn patterns and useful insights from data without guidance. For example, unsupervised learning algorithms such as K-means clustering can be used to cluster audiences into similar groups based on interests, age, geography, etc.

[0077] K-means clustering is generally considered a simple unsupervised learning approach. In K-means clustering, similar data points can be clustered together and bound in the form of clusters. One way to bind data points together is by calculating the centroid of the group of data points. In determining effective clusters, K-means clustering evaluates the distance between each point and the cluster centroid. Depending on the distance between the data point and the centroid, the data is assigned to the nearest cluster. The goal of clustering is to determine the intrinsic grouping of a set of unlabeled data. The "K" in K-means represents the number of clusters formed. The number of clusters (essentially, the number of classes to which new instances of data can be classified) can be determined by the user. This determination can be made, for example, by using feedback and observing the cluster size during training.

[0078] K-means are primarily used when the dataset has distinct and well-separated points; otherwise, if the clusters are not separated, the model may render the clusters inaccurately. Also, K-means can be avoided if the dataset contains many outliers or if the dataset is non-linear.

[0079] According to one or more embodiments, the mapping engine 101 can utilize an ensemble learning algorithm. The ensemble learning algorithm of the mapping engine 101 uses multiple learning algorithms to achieve better predictive performance than would be obtained from any one of the constituent learning algorithms alone. The ensemble learning algorithm performs the task of searching a hypothesis space to find suitable hypotheses that make good predictions about a particular problem. Even if the hypothesis space contains hypotheses that are very suitable for a particular problem, finding a suitable hypothesis can be very difficult. The ensemble algorithm combines multiple hypotheses to form a better hypothesis. The term ensemble is usually used to describe a method of generating multiple hypotheses using the same basic learner. A broader concept of multiple classifier systems also encompasses the hybridization of hypotheses that are not induced by the same basic learner.

[0080] Evaluating ensemble predictions typically requires more computation than evaluating single-model predictions; therefore, ensembles can be seen as a way to compensate for poor learning algorithms by performing a lot of extra computation. Fast algorithms such as decision trees are commonly used with ensemble methods such as random forests, but slower algorithms can also benefit from ensemble methods.

[0081] An ensemble is a supervised learning algorithm in itself, as it can be used to make predictions after training. Therefore, a trained ensemble represents a single hypothesis. However, this hypothesis is not necessarily contained within the hypothesis space of the model from which it was built. Thus, ensembles can be shown to have more flexibility in the functions they can represent. This flexibility theoretically allows for overfitting of the training data than a single model, although in practice, some ensemble methods (particularly bagging) tend to mitigate the problems associated with overfitting the training data.

[0082] Empirically, ensemble algorithms tend to yield better results when there is considerable diversity among the models. Therefore, many ensemble methods aim to promote diversity among the models they combine. Though counterintuitive, more random algorithms (such as random decision trees) can be used to produce a stronger ensemble than very careful algorithms (such as entropy-reducing decision trees). However, the use of various powerful learning algorithms has been shown to be more effective than using techniques that attempt to simplify models to promote diversity.

[0083] The number of component classifiers in an ensemble significantly impacts prediction accuracy. Pre-determining the ensemble size and the volume and speed of the big data stream makes this even more crucial for online ensemble classifiers. Theoretical frameworks suggest that an ideal number of component classifiers exists in the ensemble, and accuracy decreases if there are more or fewer classifiers than this number. Theoretical frameworks indicate that the highest accuracy is achieved using the same number of independent component classifiers as there are class labels.

[0084] Some types of ensembles include Bayesian optimal classifiers, bootstrap aggregation (bagging), boosting, Bayesian model averaging, Bayesian model combinations, and model bucketing and stacking.

[0085] According to one or more embodiments, the mapping engine 101 may include one or more neural networks. A neural network is a network or circuit of neurons, or, in the modern sense, an artificial neural network composed of artificial neurons or nodes. The connections of biological neurons are modeled as weights. Positive weights reflect excitatory connections, and negative values ​​signify inhibitory connections. The inputs are modified by the weights and summed using linear combinations. An activation function can control the amplitude of the output. For example, the acceptable range of the output is usually 0 to 1, but it may also be -1 to 1.

[0086] These artificial networks can be used for predictive modeling, adaptive control, and applications, and can be trained via datasets. Self-learning arising from experience can occur within the network, allowing it to derive conclusions from complex and seemingly unrelated sets of information.

[0087] For completeness, a biological neural network consists of groups of chemically connected or functionally related neurons. One neuron can be connected to many other neurons, and the total number of neurons and connections in the network can be wide-ranging. Connections called synapses are usually formed from axons to dendrites, but interdendritic synapses and other connections are also possible. Apart from electrical signaling, there are other forms of signaling resulting from the diffusion of neurotransmitters.

[0088] Artificial intelligence, cognitive modeling, and neural networks are information processing paradigms inspired by how biological nervous systems process data. Artificial intelligence and cognitive modeling attempt to simulate some of the characteristics of biological neural networks. In the field of artificial intelligence, artificial neural networks have been successfully applied to speech recognition, image analysis, and adaptive control, and are used to build software agents or autonomous robots (in computer and video games).

[0089] Generally, a neural network is a network or circuit of neurons, or, in a modern sense, an artificial neural network (ANN) composed of artificial neurons, nodes, or cells.

[0090] For example, an ANN (Analog Neural Network) contains a network of processing elements (artificial neurons) that can exhibit complex global behavior determined by the connections between processing elements and element parameters. One classic type of artificial neural network is the recurrent Hopfield network. These connections in the network or circuit of neurons are modeled as weights. Positive weights reflect excitatory connections, and negative values ​​signify inhibitory connections. Inputs are modified by the weights and summed using linear combinations. An activation function can control the amplitude of the output. For example, the acceptable range of the output is usually 0 to 1, but it can also be -1 to 1. Often, an ANN is an adaptive system that changes its structure based on external or internal information flowing through the network.

[0091] In more practical terms, neural networks are nonlinear statistical data modeling or decision-making tools that can be used to model complex relationships between inputs and outputs, or to find patterns in data. Therefore, ANNs can be used in predictive modeling and adaptive control applications while being trained through datasets. It should be noted that self-learning arising from experience can occur within ANNs, allowing conclusions to be drawn from a complex and seemingly unrelated set of information. The usefulness of artificial neural network models lies in the fact that they can be used to estimate functions from observations and then utilize them. Unsupervised neural networks can be used to learn representations of inputs that capture prominent features of the input distribution, and more recently, in deep learning algorithms that can implicitly learn distribution functions of observed data. Learning in neural networks is particularly useful in applications where the complexity of the data (e.g., biometric data) or task (e.g., monitoring, diagnosing, and treating any number of different diseases) makes it impossible to design such functions manually.

[0092] Neural networks can be used in a variety of fields. Therefore, in artificial intelligence system 400, machine learning and / or artificial intelligence algorithms within it may include neural networks broadly divided according to the tasks to which they are applied. These divisions tend to fall into categories such as regression analysis (e.g., function approximation) including time series prediction and modeling, classification including pattern and sequence recognition, novelty detection and sequential decision-making, data processing including filtering, clustering, blind signal separation, and compression. For example, application areas of ANNs include identification and control of nonlinear systems (vehicle control, processing control), game play and decision-making (backgammon, chess, racing), pattern recognition (radar systems, face recognition, object recognition), sequence recognition (gestures, speech, handwritten text recognition), medical diagnosis and treatment, financial applications, data mining (or knowledge discovery in databases, i.e., "knowledge discovery in database, KDD"), visualization, and email spam filtering. For example, it is possible to create semantic profiles of patient biometric data obtained from medical procedures.

[0093] According to one or more embodiments, the neural network may implement a long-short-term memory neural network architecture, a convolutional neural network (CNN) architecture, or other similar architectures. The neural network may be configurable with respect to a large number of layers, a large number of connections (e.g., encoder / decoder connections), a rule-making technique (e.g., dropout), and optimization features.

[0094] A long-short-term memory neural network architecture, including feedback connections, can process a single data point (e.g., an image) along with an entire sequence of data (e.g., speech or video). The units of a long-short-term memory neural network architecture can consist of cells, input gates, output gates, and forget gates, where cells store values ​​over arbitrary time intervals, and gates regulate the flow of information to and from cells.

[0095] A CNN architecture is a shared-weight architecture with translational invariance, where each neuron in one layer is connected to all neurons in the next layer. Regularization techniques in CNN architectures can leverage hierarchical patterns in the data, using smaller and simpler patterns to organize more complex ones. Other configurable aspects of the architecture when a neural network implements a CNN architecture may include a large number of filters in each stage, kernel sizes, and a large number of kernels per layer.

[0096] Referring here to Figure 6, an example of a neural network 600 and a block diagram of a method 601 performed within the neural network 600 are shown according to one or more embodiments. The neural network 600 of Figure 6 may be implemented in hardware. The neural network 600 operates to assist in the implementation of machine learning and / or artificial intelligence algorithms described herein (for example, implemented by the mapping engine 101). The neural network 600 may be implemented in hardware such as the machine 420 and / or hardware 450 of Figure 4B. As shown herein, the description of Figure 6 will be made by reference to other figures where appropriate for ease of understanding.

[0097] In an exemplary operation, the mapping engine 101 collects data 410 from the hardware 450. In the neural network 600, the input layer 610 is represented by multiple inputs (e.g., inputs 612 and 614 in Figure 6). With respect to block 620 of method 601, the input layer 610 receives inputs 612 and 614. Inputs 612 and 614 may include biometric data. For example, collecting data 410 may involve accumulating biometric data (e.g., BS ECG data, IC ECG data, and ablation data, along with catheter electrode position data) from one or more procedure records of the hardware 450 into a dataset (represented by data 410).

[0098] In block 625 of Method 601, the neural network 600 encodes inputs 612 and 614 using arbitrary portions of data 410 (e.g., datasets and predictions generated by the artificial intelligence system 400) to generate latent representations or data codes. The latent representations include one or more intermediate data representations derived from multiple inputs. According to one or more embodiments, the latent representations are generated by an element-wise activation function (e.g., a sigmoid function or a normalized linear function) of the mapping engine 101 in Figure 1. As shown in Figure 6, inputs 612 and 614 are given to a hidden layer 630, shown to include nodes 632, 634, 636, and 638. The neural network 600 performs processing through the hidden layer 630 of nodes 632, 634, 636, and 638, exhibiting complex global behavior determined by the connections between processing elements and element parameters. Therefore, the transition between layer 610 and layer 630 can be considered an encoder stage that receives inputs 612 and 614 and forwards them to the deep neural network (within layer 630) to learn a smaller representation of a portion of the input (e.g., the resulting latent representation).

[0099] Deep neural networks can be CNNs, long- and short-term memory neural networks, fully connected neural networks, or a combination thereof. Inputs 612 and 614 can be intracardiac ECGs, body surface ECGs, or intracardiac and body surface ECGs. This encoding results in dimensionality reduction of inputs 612 and 614. Dimensionality reduction is the process of reducing the number of random variables (of inputs 612 and 614) being considered by obtaining a set of major variables. For example, dimensionality reduction can be feature extraction that transforms data (e.g., inputs 612 and 614) from a high-dimensional space (e.g., more than 10 dimensions) to a low-dimensional space (e.g., two to three dimensions). Technical effects and benefits of dimensionality reduction include reducing the time and memory space requirements of the data, improving the visibility of the data, and improving parameter interpretability for machine learning. This data transformation can be linear or nonlinear. The reception (block 620) and encoding (block 625) operations can be considered as the data preparation portion of the multi-stage data manipulation performed by the mapping engine 101.

[0100] In block 645 of Method 610, the neural network 600 decodes the latent representation. The decoding stage takes the encoder output (e.g., the resulting latent representation) and attempts to reconstruct specific forms of inputs 612 and 614 using another deep neural network. In this regard, nodes 632, 634, 636, and 638 are combined to produce output 652 in output layer 650, as shown in block 660 of Method 601. That is, output layer 690 reconstructs inputs 612 and 614 in reduced dimensions, but without signal interference, signal artifacts, and signal noise. An example of output 652 is cleaned biometric data (e.g., a clean / denoised version of IC ECG data). The technical effects and benefits of cleaned biometric data include enabling more accurate monitoring, diagnosis, and treatment of any number of different diseases.

[0101] Referring here to Figure 7, one or more exemplary embodiments of Method 700 (implemented, for example, by the mapping engine 101) are shown. Generally, Method 700 extracts corresponding catheter trajectories for each atrium case to approximate a realistic reconstruction of the anatomical structure. As an example, Method 700 provides a super-secondary neural network reconstruction of an anatomical structure (e.g., LA) using a sparse catheter trajectory. That is, Method 700 addresses the need for early visualization to guide the catheter by using one or more portions of the transverse catheter trajectory (e.g., the initial directional trajectory) to generate a map that includes a super-secondary neural network reconstruction of the endocardial surface of the LA.

[0102] Method 700 begins in block 710, in which the mapping engine 101 performs a pre-training phase. The pre-training phase includes training one or more neural networks to provide one or more pre-trained neural networks. To train one or more neural networks, the mapping engine 101 takes shapes, trajectories, or both, which are defined by output primitives. For example, some of the inputs for training may include one or more geometric families, as well as any identified corresponding shapes and specific primitives within those geometric families. That is, to model anatomical shape configurations, the mapping engine 101 may use primitives such as translation, rotation, size, shape, probability of existence, tapering, and bending as parameters for the neural network. In addition, the input may include trajectories (e.g., of a catheter). In summary, the inputs to the pre-training phase include catheter trajectories and anatomical shapes representing parts of the anatomical structure of the left atrium observed while generating the catheter trajectories. The pre-training phase generates a pre-trained neural network that can generate early left atrial visualization based on the catheterization route.

[0103] In block 720, the mapping engine 101 acquires a dataset. The dataset may include one or more geometric families, such as 3D atrial shapes composed of hyperquadratic functions.

[0104] In block 730, the mapping engine 101 acquires the catheter trajectory. The trajectory can be received in real time during the ablation procedure. According to one or more embodiments, the trajectory may be an input path representing the volume occupied by a portion of the catheter traversing the vicinity of the left atrium. Upon receiving the input path, the mapping engine 101 encodes the input path, as further described herein, to generate feature vectors used to identify the attributes, transformations, and probabilities of existence of primitives of the left atrial shape. The mapping engine 101 maps the catheter trajectory to a dataset, such as a 3D atrial shape, and selects a corresponding pre-trained network.

[0105] Physicians should note that they should follow an initial directional pathway that traverses known anatomical landmarks, such as the four pulmonary vein orifices, within three minutes. The technical effects and benefits of the mapping engine 101 include enabling guidance for catheters, such as single-shot catheters, where sampling the surface of anatomical structures is difficult.

[0106] In block 740, the mapping engine 101 trains a selected pre-trained network. That is, the pre-trained network is further trained based on the acquired dataset and catheter trajectory. The further trained pre-trained network can be considered a trained neural network. According to one or more embodiments, the pre-trained network is trained on the input path using a 3D atrial shape composed of a hyperquadratic function. In other words, the shape that constitutes the entire 3D model generated by the pre-trained network is a "hyperquadratic shape". A hyperquadratic shape is defined by Equation 1, which is described below. In one example, for training, the mapping engine 101 utilizes a set of acquired left atrial CT shapes to train variations in LA anatomical structures, where four pulmonary veins represent the majority of the population dataset. According to one or more embodiments, the mapping engine 101 utilizes a software simulator to create and expand a database that stores the dataset and catheter trajectory, and thus creates a "configured" set of catheter trajectories and left atrial shapes to complement the actual acquired data.

[0107] In block 750, which is part of the reconstruction operation, the mapping engine 101 approximates the atrial shape. According to one or more embodiments, once a pre-trained network is trained, the mapping engine 101 uses the proposed network to accurately approximate the atrial shape based on a given input trajectory (e.g., a measured catheter trajectory).

[0108] In block 760, the mapping engine 101 generates an output. For example, the output could be the output from a trained neural network generated in block 750. The output could be a 3D model including an early visualization of the anatomical structure being scanned.

[0109] In block 770, the mapping engine 101 displays an output (e.g., early visualization). According to one or more embodiments, by displaying an output, the mapping engine 101 can provide a 3D model as a representation of an anatomical structure in a manner meaningful to a physician and / or technician. For example, the meaningful representation of an anatomical structure by the mapping engine 101 may include enabling a physician and / or technician to perform a procedure or manipulate an anatomical structure based on inferences (e.g., via other conventional techniques) that can trigger or enable further mapping through the 3D model.

[0110] In some implementations, in block 780, the mapping engine 101 compares 3D models. The mapping engine 101 can compare 3D models across several network solutions for 3D atrial reconstruction. In this regard, the mapping engine 101 demonstrates that the generated 3D models reduce procedure time and do so with real-time performance. The reconstructed openings and orientations of the pulmonary veins in the generated 3D models have minimal error, and anatomical parts are easily identifiable. The technical effects and advantages of the mapping engine 101 include reducing mapping time while maintaining anatomical accuracy by mapping the endocardial surface of the left atrium (LA) using a portion of the transverse catheter route (initial directional route) to provide early visualization of the pulmonary veins and important anatomical parts.

[0111] According to one or more embodiments, the mapping engine 101 generates additional training data to overcome data deficiencies, and for each generated LA shape, the mapping engine 101 uses an LA instance generator to create the anatomical shape of the left atrium and uses an algorithm to create a clinically feasible simulated sparse catheter pathway.

[0112] In summary, the technique in Figure 7 involves training one or more neural networks to output LA shapes reconstructed based on an input catheter path. The output LA shapes include or are derived from a hyperquadratic shape (defined by Equation 1). In the examples of this disclosure, the input representation is a catheter path describing a measured path of a catheter through an anatomical structure. The primitive representation is a parameterized description of the anatomical structure of the left atrium. The parameters represent geometric aspects of a set of primitives (e.g., shapes) of the anatomical structure of the left atrium. The training regime of the mapping engine 101 allows the mapping engine 101 to incorporate knowledge or “prior information” of the LA anatomical structure. Additional details on how the training incorporates such “prior information” or knowledge about the LA anatomical structure are provided below. The resulting representations generated by the mapping engine 101 provide a flexible framework that allows for semantic segmentation into different parts of the LA, enables post-inference editing, and ensures the anatomical correctness of the reconstruction. Therefore, the mapping engine 101 allows physicians to incorporate clinically relevant information, such as PVLS and LAA separation, into the reconstruction process. In other words, the semantic representation of the anatomical structure of the left atrium, i.e., the representation as a parameterized three-dimensional model, allows for editing of this generated representation by enabling editing of the parameterized three-dimensional model. Data from clinical trials, including human subjects, show that the reconstruction operation by the mapping engine 101 produces more accurate results than those obtained using previous DED and V-Net solutions. In some examples, techniques utilizing steps 730, 740, and 750 (without utilizing other described steps) are possible. In various examples, techniques involving any subset of the steps of method 700 are contemplated. In particular, techniques involving training a neural network based on input data (e.g., acquired catheter trajectories and actual LA shapes) and then performing reconstruction are contemplated.In addition, in some examples, techniques are intended to include only the training step (e.g., 710-740 or 720-740) or only the reconstruction step (e.g., 750 or 750-760).

[0113] Figure 8 depicts System 800 according to one or more embodiments. In some examples, the operations described as being performed by System 800 in Figure 8 represent at least Operation 750 in Figure 7. System 800 is an exemplary schematic diagram of at least a part of Mapping Engine 101. The operation of Mapping Engine 101 provides a reconstruction of the left atrial shape 901 using a superquadratic network (SQNet) that leverages medical knowledge about the general shape of the left atrium (LA). That is, Mapping Engine 101 learns to represent the shape of the LA by transforming a set of geometric primitives that are generalized ellipsoids (superquadratic functions). The combination of these primitive shapes defines the interior of the LA, while the boundary is the surface.

[0114] An input path 810 is provided to the system 800. Herein, the term “input path” is used synonymously with the term “catheter transverse path.” An input transverse path is a recorded path of a catheter or other command. In various embodiments, this path is obtained by any technically feasible method, such as via electroanatomical mapping as described elsewhere in this specification. The input path 810 is encoded using a convolutional neural network ("CNN") 820, which generates feature vectors 830 used by a geometry regression network 840 to independently regress the attributes, transformations, and probabilities of existence of primitives characterizing the left atrial shape. The mapping engine 101 may include and be available a modified version of SQNet called Left Atrial Shape Reconstruction (LASQNet), which uses a super-second-order network specifically designed for left atrial shape reconstruction. Thus, the overall structure of LASQNet may be similar to that of SQNet. More specifically, the geometry regression network 840 is a network that generates output 3D models containing multiple hyperquadratic shapes, and the network is trained to incorporate and therefore reproduce LA anatomical structures.

[0115] Regarding how the primitives characterizing the left atrial shape are defined (e.g., the format of the shape output by the geometry regression network 840), the shape of the primitives is fully defined using analytical formulas that enable the mapping engine 101 to determine whether any given 3D point in space is inside, on the surface of, or outside the shape. These formulas also allow for approximation of the distance of a point from the surface of the shape.

[0116]

number

[0117]

number

[0118] The distance from a 3D point to the surface of a shape can be approximated using Equation 2, which is:

[0119]

number

[0120]

number

[0121]

number

[0122] Equation 2 provides an upper bound on the Euclidean distance, which is equal to zero when a point lies on a surface. Using this Equation 2, the mapping engine 101 restricts the signed distance transformation (in absolute value) for all points by a function of each shape.

[0123]

number

[0124]

number

[0125] The function (see Equation 4) indicates whether a point is inside, on, or outside any of the shape primitives, and provides an approximate distance to the surface of the shape.

[0126]

number

[0127] According to one or more embodiments, the training operation involves training a neural network of the mapping engine 101 to generate a 3D model of the left atrium using several techniques. These techniques include incorporating prior knowledge of specific shapes appropriate to the anatomical structure of the left atrium. The techniques also include training the neural network to apply a set of transformations to make the resulting shape more realistic (e.g., a loss function is used to control different scenarios in the optimization process). The neural network is also trained to apply overlapping constraints to keep the resulting shape connected. The neural network is trained using a specific loss function combined from different loss parts, and the training of the neural network is performed using specific stages of the optimization and post-inference optimization processes (e.g., constraints can be fixed).

[0128] Some of the actions described above define what happens during reconstruction. For example, during reconstruction, the mapping engine 101 accepts an input catheter pathway and obtains a set of output parameters that define the geometry of the reconstructed left atrium. These output parameters include translational parameters, rotational parameters, size parameters, shape parameters, probability of existence parameters, tapering parameters, and bending parameters. For each of the set of shapes that are thought to be included in the reconstruction of the atrium based on prior knowledge, the probability of existence defines whether such a shape is included in the reconstruction. These “prior knowledge shapes” are described below as “selected shapes.” The translational, rotational, size, and shape parameters define the shape and size of the selected shapes, as well as the position and rotation of such shapes. The tapering and bending parameters define the tapering and bending applied to each of the shapes. As can be seen from the figure, reconstruction generates parameters that define the 3D geometry of the reconstructed left atrium based on the input catheter pathway.

[0129] Regarding the control of the selected shape, the mapping engine 101 incorporates anatomical knowledge into the optimization (part of training) process by using SQNet to decompose the atrial shape into primitives. Knowledge of the anatomical parts that make up the atrium guides the optimization process of the mapping engine 101 so that it can learn shapes that are similar to the human understanding of anatomical structures. The LA includes an ellipsoidal body with four pulmonary veins attached to the apex, a bulge (appendage) below the left pulmonary vein, and a stenosis (valve) at the base. For training (e.g., step 740) to guide the optimizer to use these building blocks, the mapping engine 101 manipulates the inferred primitive sizes of the neural network. More specifically, SQNet predicts the size of each axis for each primitive. To control the range of primitive sizes, the mapping engine 101 applies a linear transformation aσ(x)+b (where σ is a nonlinear function) with different parameters for each primitive. Scaling directly affects the gradient, and therefore directly affects how training is performed, and therefore controls the optimization. In other words, the parameters set for this linear transformation control the optimization of the neural network for training, causing the neural network to learn appropriate parameters for the shapes that make up the left atrium. For the LA body, the mapping engine 101 sets the values ​​a of the first two shapes of the generated 3D model of the left atrium to large values ​​and the values ​​of the other two shapes of the generated 3D model to medium values ​​(i.e., smaller than large values). For the remaining shapes of the left atrium intended to describe the pulmonary veins, appendages, and other smaller structures, the mapping engine 101 sets the range of a to values ​​smaller than medium values ​​(e.g., 1 to 3 voxels, where a voxel is the smallest unit of volume that can be described in the 3D model). To encourage the solution to include the LA body and to prevent convergence to trivial shapeless solutions, the mapping engine 101 sets the probability of existence of the first shape to 1.It should be understood that applying these linear transformations and setting these parameters is done during training to train the neural network of the mapping engine 101 to generate the desired shape.

[0130] Regarding transformations, each primitive is defined in a local coordinate system whose center is at the origin. Points in the primitive coordinate system undergo a series of reversible nonlinear transformations, followed by rigid body rotations Ri and translations Ti to a point in "world" coordinates. The rigid body transformations are rotations Ri and translations Ti, expressed using quaternions and translation vectors.

[0131]

number

[0132]

number

[0133] Figure 9 illustrates transformations 910, 920, and 930 according to one or more embodiments. In some cases, the mapping engine 101 uses tapering transformations and bending transformations (for example, as described above). The tapering transformation transforms the boundary of each coordinate axis z of a particular shape as a polynomial function of the reference axis S. z For each coordinate axis a∈x,y having the shape and size of , the tapering of the second angle is defined by equation 5, where k,w are the identity functions of the z coordinate. are the learned tapering parameters (i.e., the parameters output by the neural network of the mapping engine 101).

[0134]

number

[0135] The bending transformation translates each of the two endpoints of the shape by a 2D translation vector. The rest of the space is then linearly bent according to these points by z values. As shown in Equation 6, β and γ are the learned bending parameters (i.e., the parameters output by the neural network of the mapping engine 101). Note that the field function filters out z values ​​outside the shape.

[0136]

number

[0137] Loss represents the manner of training. During training, the mapping engine 101 attempts to correct its own weights by comparing its actual output with a desired output. The comparison is performed using one or more loss functions. The loss function defines how and to what extent the neural network's weights deviate from the “desired” output. In various examples, the neural network's output is a 3D model generated in response to an input catheter path, and the “desired output” is the recorded actual geometry recorded for the input catheter path. With respect to loss, the mapping engine 101 uses one or more loss parts described herein. Parts of the overall loss function are referred to herein as “loss parts”. A combination of loss parts (e.g., sum) defines the overall loss function. Several loss parts are described below.

[0138] parameter p i This indicates the probability that primitive i is part of the output shape. Average shape loss L mse L is the sum of these probabilities (with some weights) to keep the solution sparse. Given a list of surface points, the primitive loss L is calculated from the point cloud. pcl→primThe cost is defined as the expected distance (exceeding the probability of the primitive's existence) from each point to the nearest primitive. This cost guarantees that the primitive's surface is close to the target surface. A complementary form of this cost is primitive versus surface point cloud L, where the distance from each point on the primitive's surface to the nearest point in the surface point cloud is measured and this is averaged over all points in the primitive. prim→pcl This is obtained by the expected value of the primitive's probability of existence. These last two costs are calculated by sampling points from the primitive's surface. To improve the accuracy of the optimization process in fine details of the output shape, the mapping engine 101 uses a set of active shapes (p i Three additional losses are used, which apply only to those with a value > 0.5. The three additional losses include the inside-outside loss, the surface boundary loss, and the overlapping constraint loss.

[0139] According to one or more embodiments, the inside-outside loss aims to minimize the error by sampling points in space and defining in-loss and out-loss for each point that is misclassified as either inside or outside, as shown in Equations 7 and 8, where CE is the cross-entropy function.

[0140]

number

[0141] The in-loss is calculated for points where the label is inside, and the out-loss is calculated for points where the label is outside. The mapping engine 101 uses the same number of points for each group.

[0142] According to one or more embodiments, with respect to surface boundary loss, the function Q(X) is an upper bound of the signed distance inside the shape, since the distance from the surface of the shape union is greater than the distance of each primitive. This means that a point at the intersection of two primitives can be assigned a distance much shorter than the actual distance. The mapping engine 101 uses the inside indicator function

[0143]

number

[0144]

number

[0145] The mean excess field (MEF) is defined by equation 10, where ε is a small constant. The MEF provides a gradient that pushes each point outward from the interior of the shape. In practice, the mapping engine 101 minimizes both the SEC and the MEF.

[0146]

number

[0147] According to one or more embodiments, with respect to overlap constraint loss, the use of shape primitives with implicit field representations does not guarantee that the generated shapes are connected. To check connectivity, a connectivity graph must be constructed by comparing each primitive with all other primitives. The constraint is satisfied if the graph has only one connected component. A simple implementation of this process is:

[0148]

Number

[0149]

Number

[0150] The field is calculated for a set of surface points sampled from the remaining primitives (transformed to world coordinates). The function

[0151]

Number

[0152]

Number

[0153]

Number

[0154] The overlap loss is OL = Σ i max(ρ - RO iDefined as ,0)·MDI, where ρ is the required overlap percentile. This cost requires that each external primitive has an overlap rate of at least ρ, while MD i The term provides the gradient direction when the constraint is active.

[0155] According to one or more embodiments, with respect to the total cost function, the target cost function of the network is a weighted linear combination of the aforementioned losses, where the weights are hyperparameters of the optimization stage. See, for example, Equation 13.

[0156]

number

[0157] The cost is evaluated by sampling. Each primitive is sampled by N points (for example, by using a high-speed sampler). In addition, the mapping engine 101 evaluates the K values ​​inside, on the surface, and outside the target to give the target point different losses. in K surface ,K out Sample a point.

[0158] According to one or more embodiments, with respect to stepwise learning, during the optimization process, network parameters are updated in a somewhat greedy manner based on gradients. This process typically converges to a valid solution within a few epochs. As the solution approaches a valid one, some of the inferred parameters tend to remain stable. In the case of LA reconstruction, the selection of active shapes and the sizes of these shapes tend to remain constant after several epochs. These properties result in the ability to modify the optimization process by changing the weights of the loss terms, adding additional cost terms, and incorporating prior knowledge about the shapes. In the first stage, the mapping engine 101 runs the optimization process to convergence without using the bending transformation because the bending transformation introduces instability to the optimizer. As a final step, the mapping engine 101 introduces the bending transformation and overlapping constraint losses into the optimization process for fine-tuning.

[0159] According to one or more embodiments, with respect to post-inference optimization, PVLS and appendages can be fused together in FAM ("Fast Anatomical Mapping" - a technique for mapping anatomical structures) and other reconstructions. The ridges between these two parts are important anatomical features, and it is important to ensure that they are separated in reconstruction. To address this problem, the mapping engine 101 provides an additional cost term that can be used to separate these two anatomical parts within the framework. The translation vectors of the primitives describing the PVLS and appendages are respectively → μls and → Let μapp be the function.

[0160]

number

[0161]

number

[0162]

number

[0163] This cost is assigned a positive value only when the PVLS intersects with an appendage, and this value is inversely proportional to the distance vector to directly relate the gradient of this cost to the center position of the primitive. This cost function can be called the “appendage separation cost”. Overlap constraints are introduced in a soft way during the training process, but the constraints may be violated in the inferred results. To address this problem, the mapping engine 101 implements a method for performing local optimization fixes on the inferred LA samples. After inferring the LA samples, the mapping engine 101 manipulates the resulting shape parameter vector using a local optimization procedure, with the cost function being a linear combination of the appendage separation cost and the overlap constraint loss. In some examples, the mapping engine 101 uses the LBFGS optimizer to modify the shape parameter vector (i.e., the set of parameters described above that define the shape of the reconstruction) according to the cost.

[0164] According to one or more embodiments, examples of the operation of the mapping engine are provided herein. For example, results of experiments reconstructing the shape of the left atrium using LASQNet are provided. As described herein, the mapping engine 101 demonstrates the effectiveness of LASQNet shape induction in obtaining valid solutions. Furthermore, the mapping engine 101 conducts experiments on synthetic pathway reconstruction using human clinical cases and provides exemplary use cases for anatomical representation of LASQNet (including semantic segmentation of the left atrium, acquisition of a single connected component solution, and separation of PVLI from appendages).

[0165] According to one or more embodiments, with respect to the shape configuration induction and initial testing phases, the mapping engine 101 seeks solutions that include primitive shapes that closely resemble various anatomical parts of the subject. To do this, the mapping engine 101 uses two parameters, namely, shape size transformation (controlled using a variable for scale and a variable for bias) and w mse The weights applied to the average shape loss manipulate the weights to encourage the network to find solutions with fewer shapes. The shape-size transformation values ​​can include a=0.8, b=0.03 for primitives 1-2, a=0.5, b=0.02 for primitives 3-4, and a=0.02, b=0.01 for the remaining primitives. In this regard, the mapping engine 101 performs a process that induces optimization to a solution consisting of anatomical parts, as shown in Figure 10.

[0166] Figure 10 shows models 1010, 1020, 1030, and 1040 for shape configuration induction by one or more embodiments, with the original SQNet in the top row (1010, 1020) and the size being induced in the bottom row (1030, 1040). Anatomical parts are colored pink in the upper right (RS), blue in the lower right (RI), green in the upper left (LS), cyan in the lower left (LI), and yellow for appendages. Note that a solution with anatomically correct parts is found on model 1040. Model 1010 is the original SQNet w mse Provides = 0. Model 1010 shows that without average shape loss, the network generated a mixture of shapes that do not clearly correspond to specific anatomical parts. Model 1020 shows the original SQNet w mse This provides a value of =0.05. Model 1020 shows that the average shape loss alone was not sufficient to find the anatomical solution.

[0167] Model 1030 is w mse Provides a size induced to =0. As shown in Model 1030, the mapping engine 101 was able to identify anatomically corresponding parts along with many redundant primitives. Model 1040 shows wmse This provides a size induced to =0.5. Model 1040 shows that when the mapping engine 101 applies shape-size induction (by combining both methods), the mapping engine 101 was able to find an anatomical solution.

[0168] According to one or more embodiments, with respect to LA reconstruction across synthetic pathways, the mapping engine 101 used datasets and evaluation metrics. Regarding datasets, experiments were conducted on a dataset of 1800 pairs of synthetic catheter pathways and corresponding left atrial shapes. Left atrial shapes were generated, and synthetic catheter pathways were generated. These experiments allowed the mapping engine 101 to evaluate the performance of the proposed method on a wide variety of datasets. Regarding evaluation metrics, the mapping engine 101 evaluated the network's performance. For example, the mapping engine 101 used two metrics: DICE and the average distance between boundary contours (AVDist = 5 mm). DICE measures the similarity between the resulting volume and the ground truth shape, while the average distance between boundary contours compares the resulting boundary to the true boundary. The boundary is defined as a set of voxels separating the inside of the chamber from the outside. By using these two metrics, the mapping engine 101 can evaluate the overall accuracy of the network's reconstruction with respect to both the overall volume and specific boundary contours. AVDist is described using Equation 15, where d(u,v) is the Euclidean distance between voxels u and v.

[0169]

number

[0170] Both SQNet and LASQNet demonstrated fast inference times, taking less than 5ms to process 20 samples on a mid-range consumer Nvidia GTX2080 graphics card.

[0171] Figure 11 depicts Table 1110 by one or more embodiments. Table 1110 reports the results of ablation studies of SQNet and various deformations of LASQNet using different losses and transformations in sequential training runs A and B for LASQNet. Models 1210, 1220, and 1230 (Figure 12) demonstrate that in all deformations, SQNet tended to produce inflated solutions due to the low cost of the SQNet loss formulation for slightly extracardiac regions. Models 1210, 1220, and 1230 provide reconstruction examples for the test set. Blue is ground truth, and reconstructions are green. The reconstructions of SQNet are "inflated" compared to ground truth, which did not occur in LASQNet reconstructions. Model 1210 provides an derived SQNet, wmse=0.05. Model 1220 provides the original SQNet, wmse=0.0. Model 1230 provides LASQNet B2. SQNet tended to produce inflated solutions due to the low cost of the SQNet loss formulation for slightly covering the extracardiac region, but the use of additional loss, as seen in model 1230, helped address this issue. Mapping engine 101 also found that the additional transformation had minimal impact on quantitative measurements, as it primarily affected the local structure.

[0172] While LASQNet showed only slightly better quantitative results compared to the mean shape, a visual examination of spatial distance errors, as shown in the model in Figure 13, revealed that the errors were larger for the mean shape solution in some of the pulmonary veins and their hilum, while LASQNet had more evenly distributed errors over smaller areas, resulting from having fewer degrees of freedom to describe the left atrial body. Figure 13 shows models 1310, 1320, 1330, and 1340 in one or more embodiments. Models 1310, 1320, 1330, and 1340 provide reconstructions colored by distance to ground truth compared to the mean shape solution across two views. The mean shape solution has larger errors for the PV and adnexa. Model 1310 provides the upper LASQNET B. Model 1320 provides the upper mean LA. Model 1330 provides the lateral LASQNET B. Model 1340 provides the lateral mean LA.

[0173] According to one or more embodiments, for LA reconstruction across human clinical cases, the mapping engine 101 performed data acquisition, contact point evaluation, and ground truth CT evaluation. For data acquisition, the mapping engine 101 used a dataset consisting of 80 clinical cases, 26 of which had appropriately aligned meshes from CT segmentation. Initial directional pathways and tagged points belonging to each pulmonary vein (PV) orifice were acquired in less than 3 minutes at the start of the procedure. The input point cloud was converted to occupied volume, which is the expected input to the network. The network output was converted to a triangular mesh using the marching cube algorithm. Figure 14 shows graphs 1410, 1420, and 1430 from clinical cases. Acquired pathways are red, synthesized template pathways are blue, and CT is gray. Tagged PV points are color-coded. Graph 1410 (e.g., focal catheter point cloud pathway) shows the input point cloud (pathway) with tagged PV centroids. Graphs 1420 (e.g., Focal Catheter Voxelization Pathway) and 1430 (e.g., Lasso Catheter Voxelization Pathway) show the input volume in red, along with color-tagged points for each PV (PVRS yellow, PVRI green, PVLI blue, PVLS light blue) and the aligned CT. Reconstruction performance was evaluated using two methods: one using contact point IV-C.2 and the other comparing it to ground truth CTs IV-C.3. Input point sets and paths may differ depending on the type of catheter used, and discontinuities may exist in the acquired paths due to interruptions in position acquisition when the catheter exceeds a predetermined speed. The protocol may deviate from the specified protocol, and multiple arm catheters or different catheter operations may be used. As described below, the performance of LASQNet was compared to the performance of the conventional LA reconstruction method, DED and V-Net for LA reconstruction.

[0174] According to one or more embodiments, the mapping engine 101 provided contact point evaluation. In this evaluation, the mapping engine 101 used a force-sensing catheter (THERMOCOOL SMARTTOUCH® SF Catheter) to acquire points on the surface during respiratory gating (end-expiration) with contact forces of 5g to 15g. These points were accurate (less than 1mm error) and often located near important anatomical landmarks and ablation areas. The study focused on cases using a focal catheter, which had a point cloud covering most of the synthetic pathway portion and yielded a total of nine suitable cases. Reconstruction accuracy was measured as the mean distance between the ground truth point and the nearest vertex on the reconstructed mesh. Figure 15 shows Table 1500, which provides clinical case results. The left column shows inter-surface distances (mean and standard deviation) comparing LA reconstructions with ground truth CT across 26 clinical cases. Results are shown for LASQNets, DED, V-Net[], and mean atrial distances and four different radii. The p-value tests for significant differences between the network and the average shape. P-values ​​below a significance level of 0.05 are italicized. The right column shows that the distance to the contact point across nine cases is essentially constrained to be a connected component. This effect was observed for the LA reconstruction task using all networks, including the DED network and the V-Net network. The reconstruction accuracy results for contact point evaluation are compared in Table 1500 (far right column). Thus, Table 1500 shows advantages for LASQNets, comparable results for DED, while V-Net lagged behind. All results were significantly better than the average shape.

[0175] Figures 16 and 17 show the reconstruction results for two clinical cases using LASQNet A+ and LASQNet B+ networks. In each figure, the top row is a top view, the middle row is a side view of the PV, and the bottom row is a transparent reconstruction showing all ground truth points colored according to their distance to the reconstruction. In the figures, the ground truth points are superimposed and colored based on their distance to the reconstruction. The coloring of the ground truth points indicates the distance between them and the provided reconstruction. The figures show that clinically important locations, such as points around the PV orifice and ridges, have low error values ​​(typically less than 10 mm). Defined transformations allow LASQNets to better adapt to the PV shape. The bending transformation effect can be seen in the LASQNet B+ results.

[0176] According to one or more embodiments, the mapping engine 101 provides evaluation using ground truth CT. For example, the dataset includes 26 cases with CTs aligned to acquired point clouds, which were used to reconstruct and align the CTs to a network. Focusing on clinically relevant regions, the mapping engine 101 compares the CT surface to a reconstructed mesh using inter-surface distances. Inter-surface distances were measured by taking the nearest vertex on a second mesh and averaging the distances for all examined vertices. To focus on important regions, the mapping engine 101 considered only vertices within a defined radius of tagged PV points. A radius of 15 mm was found to capture the periphery of PV openings well. In Table 1500, the mapping engine 101 presents quantitative results for all 26 CT cases. The reported p-values ​​indicate the significance of improvement in LASQNet B versus mean shape (paired t-test, one-sided), as well as improvement compared to V-Net and DED. For unbounded distances, accuracy improved by 0.4 mm, while for clinically relevant regions, the improvement ranged from 0.6 to 0.8 mm. LASQNet-A lagged behind the average shape in unbounded metrics, but its accuracy was better than the average shape in clinically relevant regions and comparable to V-Net.

[0177] Figures 18A and 18B depict clinical cases according to one or more embodiments. For example, two clinical cases with CT reconstruction yield LASQNet A+ and LASQNet B+ networks, compared to the average shape color-coded by distance to reconstruction. Furthermore, the figure shows an example of LASQNet reconstruction. In the figure, additional visualizations, including a side view of the reconstruction, are provided in the third column. The error map in the magnified area (PV) is much lower for LASQNets compared to the average shape, as visualized by a shift toward darker colors, particularly near the periphysis and ridges between PVs.

[0178] According to one or more embodiments, the mapping engine 101 provides the use of a primitive representation framework. For example, the mapping engine 101 demonstrates the use of primitive representations to add anatomical information and enforce anatomical constraints during the training process and after the results are obtained. A notable task is to segment the reconstruction into semantically meaningful anatomical parts. The LASQNet solution performs anatomical sub-segmentation as shown in Figure 19. That is, Figure 19 provides a LASQNET A+ clinical case reconstruction with anatomical segmentation. RS is pink, RI is blue, LS is green, LI is cyan, and appendages are yellow. The remainder constitutes the LA body and values. Each PV and appendage is represented using a single primitive, while the body and valves include the remainder of the primitives.

[0179] It should be noted that proper segmentation requires the results to be linked components. Experiments demonstrating procedures for handling unlinked results are described in the next section.

[0180] Figure 20 shows 3D reconstructions 2001, 2002, 2003, 2004, 2005, and 2006 according to one or more embodiments. The mapping engine 101 can generate each of the 3D reconstructions 2001, 2002, 2003, 2004, 2005, and 2006. Each initial LASQNet training does not use loss because the primitive set has not yet converged. 3D reconstruction 2003 shows an example of LASQNet A reconstruction at this stage, where fragile connections between parts of the PV and the LA body can be observed. 3D reconstruction 2004, after the network has been trained, shows properly connected LAs.

[0181] Even after this training with overlapping losses, disconnected components may still occur due to the flexibility of the constraints. In these cases, the mapping engine 101 improved post-inference reconstruction using the procedure described herein. The mapping engine 101 used three iterations of LBFGS (which converge best after a single iteration) for the results of LASQNet A. The results of disconnected LASQNet A are shown in 3D reconstruction 2005, while the corrected results (LASQNet A+) are shown in 3D reconstruction 2006.

[0182] As described herein, removing the ridge between the PVLS and its appendages is common in various reconstructions. By using the post-separation cost inference optimization described herein, the system can reopen this ridge for a given LASQNETB reconstruction result.

[0183] Figure 21 shows reconstructions 2101 and 2102 before and after applying optimization according to one or more embodiments. As shown in reconstruction 2101 (e.g., the original LASQNETB solution), the PV LS (green) and appendages (yellow) overlap. As shown in reconstruction 2102, the mapping engine 101 using shape separation optimization separates the PV LS (green) and appendages (yellow).

[0184] According to one or more embodiments, the mapping engine 101 can demonstrate how to optimize primitive-based neural networks in a controlled manner to recover the shape and anatomical parts of the LA from the catheter pathway. Furthermore, the mapping engine 101 (due to its robust configuration) provides appropriate reconstruction and segmentation of the anatomical parts required in human clinical cases.

[0185] According to one or more embodiments, the mapping engine 101 can derive the size and number of shapes in a semi-supervised manner or the like to arrive at an anatomically "correct" solution (e.g., reconstruction). According to one or more embodiments, deriving the size and number of shapes can be fully supervised if the target shape has been previously fully modeled by the mapping engine 101 or the like.

[0186] Using anatomically "correct" solutions, the mapping engine 101 defines a set of anatomical constraints and transformations. The mapping engine 101 demonstrates this capability by using an example that employs overlapping constraints. As shown in Figure 20, 3D reconstructions 2001 and 2002 each include, for example, DED and V-Net, which represent disconnected components and use LASQNet overlapping constraints during the training phase of the mapping engine 101. Using the same input, the mapping engine 101 can generate an unconstrained 3D reconstruction 2003 and a constrained 3D reconstruction 2004. The mapping engine further generates disconnected reconstructions, such as 3D reconstruction 2005, and post-inference corrected reconstructions, such as 3D reconstruction 2006.

[0187] These test cases use shape induction and additional loss functions to achieve a suitable solution using SQNets. These test cases show that LASQNet is slightly more favorable to the average shape solution in global metrics such as AVdist and DICE, but less favorable to the DED solution TODO REF (i.e., the LASQNet model has far fewer degrees of freedom compared to DED and is less likely to achieve a global fit to shape). The degrees of freedom of the mapping engine 101 are used to fit clinically interesting locations (e.g., around anatomical features and their properties) and also exhibit a better spatial error distribution compared to the average shape solution.

[0188] According to one or more embodiments, the mapping engine 101 can utilize additional feature costs and constraints, such as the size and center of the ridge and constraints on the PV direction. These feature costs and constraints can be integrated by the mapping engine 101 as both pre-cost optimization and post-inference optimization. The mapping engine 101 can then modify the model based on physician input controlling specific features. For example, a physician might estimate the location of a specific point that is the center of a ridge. If the mapping engine 101 defines a feature to find the ridge center based on anatomical parts, the mapping engine 101 can optimize the results to fit the point specified by the physician by using post-inference optimization.

[0189] According to an exemplary embodiment, the mapping engine 101 can construct one or more 3D models of the atrium using a trained neural network based on the recorded trajectory of a catheter within the atrium. Furthermore, the trained neural network can receive one or more inputs, which may include a dataset of at least three-dimensional (3D) atrial shapes and corresponding catheter trajectories.

[0190] While features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used individually or in combination with other features and elements. In addition, the methods described herein may be implemented in computer programs, software, or firmware embedded in a computer-readable medium for execution on a computer or processor. Examples of computer-readable mediums include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital versatile disks (DVDs).

[0191] The flowcharts and block diagrams in the figures illustrate the structure, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing the indicated logical function. In some alternative implementations, the functions shown in the blocks may be performed in an order other than that shown in the figures. For example, two consecutively shown blocks may actually be executed substantially simultaneously, or they may, depending on the relevant functionality, be executed in reverse order. It should also be noted that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified function or operation, or by operating or executing a combination of dedicated hardware and computer instructions.

[0192] Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element may be used individually or in combination with other features and elements. In addition, the methods described herein may be implemented in computer programs, software, or firmware incorporated into a computer-readable medium for execution on a computer or processor. When used herein, the computer-readable medium should not be interpreted as being a transient signal in itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses passing through optical fiber cables), or electrical signals transmitted through conductors.

[0193] Examples of computer-readable media include electrical signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media, but not limited to, include registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, optical media such as compact disks (CDs) and digital versatile disks (DVDs), random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random-access memory (SRAM), and memory sticks. A processor can be used with software to implement a radio frequency transceiver for use in terminals, base stations, or any host computer.

[0194] The terms used herein are intended solely to describe specific embodiments and are not intended to be limiting. Where used herein, unless otherwise specified in the context, the singular forms "a," "an," and "the" also include the plural forms. It should be further understood that the terms "comprise" and / or "comprising," as used herein, indicate the presence of a described feature, integer, step, operation, element, and / or component, but do not exclude the presence or addition of another feature, integer, step, operation, element, component, and / or group thereof.

[0195] The descriptions of the various embodiments in this specification are illustrative and not intended to be exhaustive or limitful to the embodiments disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the embodiments described. The terms used herein have been selected to best describe the principles, practical applications, or technical improvements of the embodiments compared to the art available on the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0196] [Implementation Method] (1) A method, Acquiring one or more catheter trajectories in real time during the ablation procedure, To provide a trained neural network, the pre-trained neural network is trained based on the dataset and one or more catheter trajectories. The atrial shape is approximated using the aforementioned trained neural network and one or more portions of the transverse catheter pathway. To generate a three-dimensional model output from the trained neural network and the atrial shape, The three-dimensional model output as early visualization in the ablation procedure is displayed, Methods that include... (2) The method according to Embodiment 1, further comprising performing pre-training on one or more neural networks to provide the pre-trained neural networks. (3) The method according to Embodiment 2, wherein the input for pre-training includes training one or more geometric primitives and one or more trajectories. (4) The method according to Embodiment 2, wherein the output primitive is modified by translation, rotation, size, shape, probability of existence, tapering, and bending parameters. (5) The method according to Embodiment 1, wherein one or more catheter trajectories include an input path representing the occupied volume.

[0197] (6) The method according to Embodiment 1, wherein the pre-trained neural network includes a superquadratics neural network. (7) The method according to Embodiment 1, wherein the training is performed using one or more loss portions. (8) The method according to Embodiment 1, wherein the training is performed using a linear transformation. (9) The method according to embodiment 8, wherein the linear transformation is configured to bias the output of the neural network to generate geometric features of the left atrium. (10) A non-temporary computer-readable medium for storing instructions, wherein, when an instruction is executed by a processor, the processor receives Acquiring one or more catheter trajectories in real time during the ablation procedure, To provide a trained neural network, the pre-trained neural network is trained based on the dataset and one or more catheter trajectories. The atrial shape is approximated using the aforementioned trained neural network and one or more portions of the transverse catheter pathway. To generate a three-dimensional model output from the trained neural network and the atrial shape, The three-dimensional model output as early visualization in the ablation procedure is displayed, A non-temporary computer-readable medium that enables the execution of actions including [specific actions].

[0198] (11) The non-temporal computer-readable document according to Embodiment 10, further comprising performing pre-training of one or more neural networks to provide the pre-trained neural networks. (12) The non-temporary computer-readable medium according to Embodiment 11, wherein the input for pre-training includes training one or more geometric primitives and one or more trajectories. (13) A non-transient computer-readable medium according to Embodiment 11, wherein the output primitive is modified by translation, rotation, size, shape, probability of existence, tapering, and bending parameters. (14) A non-temporary computer-readable medium according to Embodiment 10, wherein one or more catheter trajectories include an input path representing the occupied volume. (15) The pre-trained neural network includes a super-secondary neural network, as described in Embodiment 10, for the non-temporal computer-readable medium.

[0199] (16) The training is performed using one or more loss portions in the non-temporal computer-readable medium according to Embodiment 10. (17) The training is performed using a linear transformation in the non-temporary computer-readable medium according to Embodiment 10. (18) The non-temporal computer-readable medium according to Embodiment 17, wherein the linear transformation is configured to bias the output of the neural network to generate geometric features of the left atrium. (19) A system, A catheter configured to acquire one or more catheter trajectories in real time during an ablation procedure, A processing system is provided, and the processing system is To provide a trained neural network, a pre-trained neural network is trained based on the dataset and one or more catheter trajectories. Using the aforementioned trained neural network and one or more portions of the transverse catheter pathway, the atrial shape is approximated. A three-dimensional model is generated from the trained neural network and the atrial shape. A system configured to display the three-dimensional model output as early visualization in the ablation procedure. (20) The system according to embodiment 19, wherein the processing system further comprises a user interface for the user to edit a model based on input from a physician.

Claims

1. A non-temporary computer-readable medium for storing instructions, wherein, when an instruction is executed by a processor, the processor... To acquire one or more catheter trajectories in real time during the ablation procedure, To provide a trained neural network, the pre-trained neural network is trained based on the dataset and one or more catheter trajectories. The atrial shape is approximated using the aforementioned trained neural network and one or more portions of the transverse catheter pathway. To generate a three-dimensional model output from the trained neural network and the atrial shape, The three-dimensional model output as early visualization in the ablation procedure is displayed, A non-temporary computer-readable medium that enables the execution of actions including [specific actions].

2. The non-temporal computer-readable document according to claim 1, further comprising performing pre-training of one or more neural networks to provide the pre-trained neural networks.

3. The non-temporary computer-readable medium according to claim 2, wherein the input for pre-training includes training one or more geometric primitives and one or more trajectories.

4. The non-temporary computer-readable medium according to claim 2, wherein the output primitives are modified by translation, rotation, size, shape, probability of existence, tapering, and bending parameters.

5. The non-temporary computer-readable medium according to claim 1, wherein the one or more catheter trajectories include an input path representing the occupied volume.

6. The non-temporal computer-readable medium according to claim 1, wherein the pre-trained neural network includes a super-secondary neural network.

7. The training is performed using one or more loss portions in the non-temporary computer-readable medium according to claim 1.

8. The training is performed using a linear transformation in the non-temporary computer-readable medium according to claim 1.

9. The non-temporal computer-readable medium according to claim 8, wherein the linear transformation is configured to bias the output of the neural network to generate geometric features of the left atrium.

10. It is a system, A catheter configured to acquire one or more catheter trajectories in real time during an ablation procedure, A processing system is provided, and the processing system is To provide a trained neural network, a pre-trained neural network is trained based on the dataset and one or more catheter trajectories. Using the aforementioned trained neural network and one or more portions of the transverse catheter pathway, the atrial shape is approximated. A three-dimensional model is generated from the trained neural network and the atrial shape. A system configured to display the three-dimensional model output as early visualization in the ablation procedure.

11. The system according to claim 10, further comprising a user interface for a user to edit a model based on input from a physician.

12. It is a method, To acquire one or more catheter trajectories in real time during the ablation procedure, To provide a trained neural network, the pre-trained neural network is trained based on the dataset and one or more catheter trajectories. The atrial shape is approximated using the aforementioned trained neural network and one or more portions of the transverse catheter pathway. To generate a three-dimensional model output from the trained neural network and the atrial shape, The three-dimensional model output as early visualization in the ablation procedure is displayed, Methods that include...

13. The method according to claim 12, further comprising performing pre-training on one or more neural networks to provide the pre-trained neural networks.

14. The method according to claim 13, wherein the input for pre-training includes training one or more geometric primitives and one or more trajectories.

15. The method according to claim 13, wherein the output primitive is modified by translation, rotation, size, shape, probability of existence, tapering, and bending parameters.

16. The method according to claim 12, wherein one or more catheter trajectories include an input path representing the occupied volume.

17. The method according to claim 12, wherein the pre-trained neural network includes a super-second-order neural network.

18. The method according to claim 12, wherein the training is performed using one or more loss portions.

19. The method according to claim 12, wherein the training is performed using a linear transformation.

20. The method according to claim 19, wherein the linear transformation is configured to bias the output of the neural network to generate geometric features of the left atrium.

Images