Imaging to determine electrode geometry

Abstract

The present disclosure relates to imaging to determine electrode geometry. A method includes placing a set of electrodes on a body surface of a patient's body. The method also includes digitizing locations of the electrodes that span the body surface based on one or more image frames using range imaging and / or monoscopic imaging. The method also includes estimating locations of hidden electrodes of the electrodes on the body surface that are not visible during the range imaging and / or monoscopic imaging. The method also includes registering the locations of the electrodes on the body surface with pre-determined geometry information that includes the body surface and an anatomical envelope inside the patient's body. The method also includes storing geometry data based on the registration that defines spatial relationships between the electrodes and the anatomical envelope in a non-transitory memory.

Description

[0001] This application is a divisional application of the invention patent application entitled "Imaging to determine electrode geometry", with an international filing date of July 12, 2018, international application number PCT / US2018 / 041834, and Chinese national phase application number 201880046691.5.

[0002] Cross-references to related applications

[0003] This application claims priority to U.S. Provisional Application No. 62 / 531643, filed July 12, 2017, entitled “IMAGING TO DETERMINEELECTRODE GEOMETRY”, the contents of which are incorporated herein by reference. Technical Field

[0004] This disclosure relates to using imaging to determine the geometry of an electrode array. Background Technology

[0005] Electrocardiographic imaging (ECGI) is a non-invasive imaging modality for cardiac electrophysiology (EP) and arrhythmias. It can be used to reconstruct epicardial potentials and to provide electrographs and isochrones from body surface potentials such as BSPM and / or other electrocardiographic models. To perform non-invasive ECGI, the location and geometry of body surface electrodes are determined by placing electrodes on the patient during a computed tomography (CT) scan or another medical imaging modality. In many healthcare facilities, this can be difficult to schedule, time-consuming, and expensive. For example, during a CT scan, electrodes need to be placed on the patient, and the same vest needs to be worn until the patient's EP procedure time. In some cases, the wait time between a CT scan and the actual EP procedure can exceed 6 hours due to hospital scheduling. Summary of the Invention

[0006] This disclosure relates to using imaging to determine the geometry of an electrode array.

[0007] As an example, one method includes placing an electrode array on a body surface of a patient's body. The method also includes digitizing the positions of the electrodes across the body surface based on one or more image frames using range imaging and / or monoscopic imaging. The method further includes estimating the positions of hidden electrodes on the body surface that are not visible during range imaging and / or monoscopic imaging. The method also includes registering the positions of the electrodes on the body surface with pre-determined geometric information, including the body surface and an anatomical envelope within the patient's body. The method further includes storing the geometric data based on this registration, which defines the spatial relationship between the electrodes and the anatomical envelope, in a non-transient memory.

[0008] As another example, a system includes an imaging acquisition system comprising a portable ranging imaging device and / or a single-view imaging device configured to generate image data comprising one or more image frames of a body surface including multiple electrodes positioned on a body surface and within the direct line of sight during image capture by the ranging imaging device and / or the single-view imaging device. A non-transient memory stores machine-readable instructions and data, including predetermined geometric information about the internal cardiac envelope and the body surface. At least one processor has access to the memory and executes the instructions to perform a method. The method executed by the processor includes determining three-dimensional electrode locations and surface geometry from the image data. The determined three-dimensional electrode locations and surface geometry include estimated locations of hidden electrodes that are outside the direct line of sight during image capture by the portable ranging imaging device and / or the single-view imaging device. The determined three-dimensional electrode locations and surface geometries are registered with pre-defined geometric information of the cardiac envelope and body surface to provide aggregated geometric data that describes the three-dimensional spatial relationship between each of the multiple electrodes and the cardiac envelope. This aggregated geometric data is stored in memory. Attached Figure Description

[0009] Figure 1 Examples of methods for digitizing electrodes and anatomical structures to generate geometric data for use in electrophysiological surgery are depicted.

[0010] Figure 2 An example method for image processing to stitch and register multiple acquired image frames is described.

[0011] Figure 3 An example of an electrode containing electrode identification text is depicted.

[0012] Figure 4 An example of a tracking tag is depicted.

[0013] Figure 5 An example method for estimating the location of a hidden electrode using electromagnetic tracing is shown.

[0014] Figure 6 An example of a system for digitizing electrodes and anatomical structures to generate geometric data for use in electrophysiological surgery is depicted.

[0015] Figure 7 An example of another system is depicted for digitizing electrodes and anatomical structures to generate geometric data for use in electrophysiological surgery.

[0016] Figure 8 An example of an electrode array containing tracking markers attached to the front of the torso is depicted.

[0017] Figure 9 An example of another electrode array attached to the rear of the torso is depicted.

[0018] Figure 10 An example of a portion of an electrode array is depicted, which includes tracking markers used to identify the positions of the electrodes.

[0019] Figure 11 It is a side view depicting an example of an electrode panel attached to the front and rear of the torso while lying on a table.

[0020] Figure 12 An example of a reconstructed three-dimensional surface model of the torso showing the locations of the electrode markers is depicted.

[0021] Figure 13 Examples of mapping and treatment systems are depicted. Detailed Implementation

[0022] This disclosure relates to using imaging (via range imaging devices and / or monoscopic imaging devices) to acquire electrode positions on a patient’s body surface in a manner decoupled from methods used to obtain anatomical geometry information describing the spatial relationship between the internal anatomical envelope (cardiac envelope, such as the surface of the heart) and the body surface.

[0023] Instead of holding electrodes on the patient during medical imaging (e.g., via computed tomography or other imaging modalities), the method described herein enables medical imaging to be performed without electrodes on the patient. Therefore, electrodes can be placed on the patient's skin temporally adjacent to the time of the procedure (e.g., electrophysiological (EP) surgery). After the electrodes are positioned on the patient's body, they are digitized using a ranging imaging device (e.g., a ranging imaging camera or a 3D scanning device). In another example, markers can be placed at predefined locations, either at or near the corresponding electrode locations. For example, the markers can be integrated into patches, vests, or other forms of clothing containing the electrode array, such that the relative spatial position of the markers to the electrodes is known.

[0024] After the electrodes and markers are positioned on the patient's body, optical imaging (e.g., using a portable rangefinder imaging camera or a single-view imaging device) is performed to record a series of image frames of the marked electrodes. Image processing can be performed to stitch the image frames together so that these frames and the corresponding electrode positions within these frames are aligned. Image processing further includes estimating the position of a hidden electrode, which may, for example, reside on a backplate when the patient is supine on a table. Regardless of the image acquisition method (or all image acquisition methods) used, registration is then performed on the patient's skin surface based on medical imaging, and the electrodes and the body surface are digitized using optical imaging (e.g., using rangefinder imaging or a single-view imaging device) to provide corresponding geometric data. In some examples, the patient has a pre-existing medical image (CT or MRI volume of the patient's torso) of a portion of the body at the electrode placement site, and the systems and methods disclosed herein can directly use these previously acquired images to work alongside images acquired using a rangefinder imaging device. The process is facilitated because optical imaging devices can be portable (e.g., telephones, tablets, handheld digital cameras, or other portable camera devices) and can be easily used without special training.

[0025] The method disclosed herein enables medical imaging scans to be acquired independently of electrode placement on the patient. This provides flexibility in allocating medical resources and significantly reduces the support burden on suppliers and healthcare providers, while improving diagnostic quality. For example, medical imaging scans can be acquired independently of electrode placement on the patient. As a result of the method disclosed herein, patient wait times are reduced, which similarly increases patient comfort. This method further leads to improved signal quality by avoiding potential contact loosening and patient movement that could occur during such long wait times. Finally, the method disclosed herein helps improve the quality of ECGI outcome mapping and may also increase patient success rates. Thus, the digitization of electrodes on the patient's body and the acquisition of electrical information over time intervals (e.g., for screening purposes) can be completed, whether the cardiac geometry from other medical imaging modalities are known or unknown.

[0026] As used herein, the term ranging imaging device (e.g., camera) can refer to a variety of different types of imaging devices. For example, potential sensing mechanisms utilized by the device may include distance-gated time-of-flight (ToF), radio-frequency modulated ToF, pulsed light ToF, and projected light stereo. Regardless of the sensing mechanism implemented, a camera provides image frames, which include an image (sometimes a color image) and depth information for each pixel (depth image). Ranging imaging devices are also referred to by other names, including flash lidar, time-of-flight (ToF) cameras, and RGB-D cameras, to name just a few. Single-view imaging may include digital optical cameras that are not depth-based and are capable of acquiring and reconstructing images of a three-dimensional object (patient) from multiple different camera perspectives (e.g., a mobile phone or other portable handheld devices that include single-view cameras).

[0027] As used herein, medical imaging refers to any technique and / or process that creates visual representations of the internal anatomical structures of the body, as well as visual representations of the (physiological) functions of organs or tissues, for clinical analysis and medical intervention. A medical imaging device thus refers to one or more devices configured to perform medical imaging. Therefore, medical imaging can reveal internal structures hidden by skin and bone. Some examples of medical imaging include devices configured to perform one or more imaging techniques, including: X-ray radiography, fluorescein examination, medical ultrasound imaging or ultrasound, computed tomography (CT), positron emission tomography (PET), single-photon emission computed tomography (SPECT), magnetic resonance imaging, etc.), and other imaging techniques. In other examples, other imaging modalities may be utilized, individually or in combination, to provide medical imaging.

[0028] Additionally, while many examples in this paper are described in the context of the detection and analysis of cardiac electrical signals, it will be understood that the methods disclosed herein are equally applicable to determining the geometry used in solving the inverse problem of noninvasively estimating other electrophysiological signals, such as those acquired as part of electroencephalography, electromyography, electrooculography, etc.

[0029] Figure 1 An example of a workflow method 10 for digitizing sensing electrodes and anatomical structures used in EP surgery is depicted. Workflow 10 allows the digitization of the electrode geometry to be decoupled from the geometry of the anatomical structures used as part of the EP study, as mentioned above. Figure 1 In the example, workflow 10 includes medical imaging 12. Medical imaging 12 can employ any medical imaging modality to internally image the patient's anatomical structures, such as those disclosed herein. The workflow also involves image segmentation at 14. The segmentation at 14 can thus determine the three-dimensional geometry of the body surfaces and (e.g., the heart's) anatomical envelope based on the performed medical imaging using image processing. The anatomical geometry can be generated, for example, from the medical imaging performed at 12, to identify the relationship between the external body surfaces (e.g., skin) and the anatomical envelope (e.g., the heart surface or another surface between the heart and the patient's body's external surface, e.g., a spherical model at or near the heart surface). For example, segmentation can provide a three-dimensional model or mesh of the body surfaces and the anatomical envelope.

[0030] Prior to the EP procedure, workflow 10 includes the placement of electrodes on a body surface, as shown at point 16. As mentioned, the electrode placement process involves placing electrodes on the body at point 16, which can be decoupled from the generation of the geometry of medical imaging 12 and the segmentation 14. This decoupling is shown by a dashed box surrounding medical imaging 12 and segmentation 14. Electrodes on the body surface can be implemented in various forms. As an example, the electrodes may correspond to a high-density arrangement of body surface sensors (e.g., more than about 200 electrodes or more) distributed on and around (e.g., completely encircling) a portion of the patient's torso for measuring electrical activity associated with the patient's heart. As another example, the array may be a reduced array of electrodes that does not completely cover the entire torso of the patient and is designed for measuring electrical activity for a specific purpose (e.g., an array of electrodes specifically designed for analyzing atrial and / or ventricular fibrillation) and / or for monitoring electrical signals in predetermined spatial regions of the heart (e.g., (multiple) atrial regions or (multiple) ventricular regions).

[0031] After the electrodes have been positioned on the body surface, optical imaging is performed to digitize the electrodes on the body surface, as shown at point 18. Imaging at point 18 can utilize any form of ranging imaging device, such as the portable optical imaging device disclosed herein, to acquire a series of image frames corresponding to images of the body surface taken from multiple different viewpoints relative to the body surface. For example, the ranging imaging device has a field of view that can image only a portion of the electrode array in a given image frame. Thus, the imaging device is moved around the patient to image different portions within its field of view until each of the visible portions of the torso surface and the associated electrodes has been captured in one or more image frames. In some examples, the entire torso containing the electrodes can be digitized via the ranging imaging device to produce multiple image frames that collectively include all electrodes. In other examples, such as when the patient is lying on an EP couch, bed, or other surface, during such imaging, multiple portions of the torso and electrodes within the imaging device's line of sight are directly imaged in multiple image frames, while multiple portions of the torso and electrodes that are hidden are not included in these image frames.

[0032] At 20 locations, the three-dimensional surface geometry of the torso is acquired based on imaging at 18 locations. For example, image frames are stitched together and used to form a 3D image of the patient's torso, which includes the arrangement of electrodes distributed across the torso. Regarding Figure 2 An example of a stitching process for registering and combining image frames is shown. This allows for the construction of a complete 3D surface from the image frames, where each image frame is associated with a previous frame to provide accumulated surface information. The surface geometry provided at point 20 can be a single-volume mesh structure, such as corresponding to a set of point clouds of that surface.

[0033] Electrodes are identified at location 22. In some examples, electrodes are identified within a 3D surface geometry provided at location 20. In other examples, electrodes may be identified directly based on image processing of each acquired image frame—either when each frame is acquired or after multiple frames have been acquired. For example, circles or other predetermined shapes may be segmented from the image to identify electrode locations, and the centroid may be used to represent each such electrode location. Additionally or alternatively, camera-visible text or other markings may be printed on the electrode array and identified via image processing (e.g., using optical character recognition). The identified electrode group obtained at location 22 may include a complete or partial electrode group, depending on whether any electrodes are hidden—i.e., not visible from the torso and the complete digitized surface of the electrode or partial electrode group—and whether all electrodes have been identified. Within the stitched image, the positions of electrodes on the torso surface imaged at location 18 and included in the stitched torso image and the corresponding surface geometry at location 20 can be determined using known relationships between other electrodes (using known electrode distributions).

[0034] In some examples, the identification of the electrode at 22 can be used to guide the user. For example, in response to determining that one or more expected electrodes (at 22) have not yet been identified in the geometry data, method 10 can provide the user with an indication (e.g., a graphical indication on a user interface, or an audible or textual indication) that an electrode is missing in the image data and additional imaging is required, as shown by the dashed line 23 returning from 22 to 18. This indication can specify the missing one or more electrodes based on the placement at 16 by numbering or the location where such missing electrodes are expected to reside in the anatomical region of the patient's torso.

[0035] At point 24, the location of any hidden surface geometry and electrodes (e.g., not in the line of sight of the imaging device during image capture at point 18) is estimated. For example, imaging at point 18 may be performed when the patient is in the same position as during EP surgery, such as lying on a table or bed. Therefore, during imaging, a portion of the electrodes (e.g., electrodes on the patient's back) is engaged with the bed or other surface, or otherwise hidden from the camera's line of sight during image capture. Therefore, when the patient is lying down (i.e., supine), electrode location information of the hidden electrodes cannot be directly derived from one or more images generated by the optical scanning mechanism. Thus, at point 24, method 10 is implemented to determine the location of the hidden electrodes on the patient's dorsal side, such as by utilizing information from the optical scan combined with the known structure and arrangement of the electrodes on the posterior side.

[0036] By way of example, an optical scan performed at 18 locations can visualize the electrodes on that side of the patient (see example). Figure 11 A portion of the electrodes visible on the patient's side is connected to (i.e., is part of) the back panel electrode array. The back panel electrode array may be configured as a monolithic mesh of a flexible material (e.g., a fabric panel), to which the electrodes are attached. For example, the panel may be configured to be compliant in the lateral direction to facilitate attachment to the patient's back, but not to provide stretch in the planar or lateral direction. Therefore, the panel is not stretched, and the electrodes maintain a predetermined relative position on such a panel (see, for example...). Figure 9 The predetermined locations can be stored in memory. Therefore, given that some electrodes and / or other markings from the back panel are visible on one or both sides of the patient, and at the top of the shoulder region, the 3D locations of hidden electrodes across the visible portions of the electrode array can be calculated based on the positions of the back panel electrodes identified at point 22 on that side and predetermined electrode location data. A similar method can be used to calculate the electrode locations of other electrodes that may be hidden on other parts of the patient's torso.

[0037] For example, on a flat EP bed without a pad, the back shape will be a continuation of the optical scanning surface until it comes to rest on a flat plane corresponding to the surface of the EP bed. The 3D position of the platform plane is derived from the optical scans (18 locations). That is, the optical scans, including those individually positioned on the supine side, will not only record the patient or electrode array but also capture the surface beneath and supporting the patient (e.g., the bed). Thus, the surface geometry of the hidden flat surface can be estimated as a plane in 3D space that coincides with the surface supporting the bed or other structure.

[0038] In an example where the pad is placed under the patient (between the bed and the patient, or as part of the bed), the physical compression of the pad can be calculated to account for deflection due to the patient's weight. For example, the patient's weight, the size of the patient's cross-section, and the center of the patient's cross-section can be input parameters used by estimates at 24 (e.g., as a model) to represent the resulting pad compression profile, which can be adjusted by pad material stiffness settings (e.g., the indentation load deflection level of the foam). The hidden back panel shape can then be calculated to follow the resulting compressed pad profile. The calculated back panel surface geometry estimated at 24 is linked to the surface geometry of the optically visible portion of the surface obtained at 20 (e.g., via splicing) to provide the complete torso surface geometry of the patient and electrode locations. The surface geometry and the electrode locations distributed across the surface geometry can be stored in memory.

[0039] At position 26, the segmented body surface geometry (from position 14) is registered in a 3D coordinate system with the surface geometry derived at positions 18-24 and the electrode positions. As an example, images(s) of the digitized electrodes generated at position 20 can be registered to the coordinate system of the segmented body surface geometry generated at position 14. As another example, the segmented body surface geometry generated at position 14 can be registered to the coordinate system of the surface generated from the digitized electrodes at position 24. In yet another example, the generated geometry at position 14 and the geometry of the digitized electrodes from position 20 can be registered to different but universal 3D coordinate systems.

[0040] Once the torso surface has been reconstructed from the optical scan, it needs to be associated with the segmented body surface from the imaging modality (e.g., CT or MRI). Due to the differences between the surfaces the patient lies on during 12- and 18-point imaging, the gross structures of the two skin surfaces may differ. For example, the CT system table profile is curved, while the EP lab table is flat. For accurate registration, and therefore the accurate positioning of the heart relative to the skin, the registration module must take this into account. Alternatively, one table surface can be modified to match the profile of the other.

[0041] As an example, the 26-point registration includes rigid registration to provide a coarse alignment between two skin surfaces, such as using features like the surface centroid and visible body features that do not substantially deform (e.g., the shoulder and nearby bones). The 26-point registration can then be followed by deformable registration to deform one or both of the geometry for further alignment. The registration is applied to a CT / MRI volume to produce a segmented heart or cardiac envelope within a reference frame that is identical to the acquired skin surface and includes electrode locations distributed across that surface.

[0042] In some examples, additional geometric information can be utilized to enhance the digitization of the electrodes at 18 locations, such as electromagnetic sensing systems (e.g., one from the Aurora tracking system from Northern Digital). In additional or alternative examples, another imaging modality (e.g., fluorescence imaging) can be used to identify markers to facilitate registration at 26 locations. For example, radiographic markers can be placed on or in known locations relative to a predetermined set of electrodes, such as outside the line of sight from a ranging imaging device (e.g., electrodes on the patient's back when the patient is lying face down). Fluorescence imaging or another portable radiographic imaging modality can be used to identify sets of one or more markers on the back panel or in other locations hidden and not in the direct line of sight during image acquisition. One or more markers may also be placed on the front (or other visible locations). Fluorescence imaging systems are typically available in EP laboratories, while other medical imaging modalities can also be used to detect markers. For example, fluorescence imaging can be performed at different angles to determine the 3D location of markers via back projection. In some examples, the markers can be of different shapes, making them easily distinguishable from each other and from other hardware such as catheter electrodes. The markers can be segmented and registered to the image, and because their positions relative to the sensor electrodes are known, the registered marker positions can be used to estimate the sensor positions around the torso.

[0043] Once the surfaces from points 14 and 24 have been registered together at point 26, ensuring that the electrode positions on the body surface and anatomical envelope are registered in a shared coordinate system, the corresponding geometric data can be generated and stored in memory at point 28 for use in EP surgery. This article is relative to... Figure 13 An example of a system utilizing geometric data generated using method 10 is disclosed.

[0044] Figure 2 An example method 50 for image processing to stitch and register multiple acquired image frames is described (e.g., typically with...). Figure 1The method corresponds to imaging 18, geometry acquisition 20, and electrode identification 22. That is, in order to establish a complete 3D surface for the complete electrode assembly, each image frame needs to be associated with other frames. The process of registering and combining images in method 50 is referred to herein as "stitching". At 52, image frames are acquired from a camera. For example, the camera is a ranging imaging device (e.g., in an RGB+D camera), such that the image frames include RGB and depth information of visible objects within the camera's field of view. Each of these multiple image frames acquired by the imaging device can be stored as image data. The subsequent parts of the stitching can be implemented as each subsequent frame is acquired. Alternatively, the method can be performed after multiple image frames have been acquired.

[0045] At point 54, the electrodes in a given acquired image frame are located and identified. The location and identification of each image frame can be based on a visual representation associated with the electrode. The electrodes have known visual features on the electrode array, which are identifiable via automated methods to enable their identification. For example, the representation may contain text, such as one or more numbers and / or letters printed on the outer surface of each electrode. Figure 3 An electrode with the number "32" printed on a circular outer layer is shown. The circular outer layer is surrounded by a black substrate layer, which facilitates the detection of the electrode boundaries in optical image data. Other numbers or reference characters, such as..., may be used. Figures 8-10 As shown in the diagram. Additionally, or alternatively, the electrode array can be enhanced using a unique tracking marker corresponding to each respective electrode, such as... Figure 4 The block pattern shown in the diagram can be printed on the outer layer of a selected electrode group. Alternatively or additionally, tracking marks can be positioned at predetermined locations across the outer surface of the electrode array, which has known spatial positions of the electrode groups on the array. Figure 4 The example in the example is a block code, but in other examples, other codes (e.g., barcodes and QRS codes) can be used to identify the electrode locations on the array.

[0046] By way of example, to locate the electrodes, at point 54, image processing uses known visual features to search for suitable shapes (e.g., circles) or block patterns (e.g., tracking markers) in the image frame. For example, the location of each electrode or tracking marker can be defined as a pixel location and a pixel depth associated with the center (centroid) of the electrode or tracking marker, respectively. To identify each located electrode, text within the located electrode can be identified (e.g., using OCR methods) to determine its identity. If the text is illegible for a given electrode, identifying the vicinity of the electrode provides sufficient information to determine the identity of the given electrode. When markers are used, the block pattern of each marker is encoded with a unique identifier that can be associated with the electrode(s) it marks. The location and identity of the electrodes in each acquired image frame can be determined, and the electrode locations and identities are stored in the memory of a computing device.

[0047] After the positions and identities of the electrodes have been determined for at least two image frames, at 56, the method uses the positions of the identified electrodes to perform an initial coarse registration between the pairs of image frames. For example, the positions of identified electrodes in different frames can be aligned to spatially overlap each other. Errors caused by large camera movements can be mitigated by using such identified electrodes in newly acquired frames to perform alignment with previously discovered electrode positions in one or more previous frames. Once all common electrodes in each of the frames have been aligned to provide such a coarse initial alignment, the method proceeds to 58.

[0048] At 58, final registration refinement is performed using the complete surface information of the captured corresponding image frame. Subsequently, at 60, the registered surface data from the image frame is accumulated and stored in memory. For example, surface accumulation is performed using a truncated signed distance volume. The surface data of the coarsely registered view is projected out of the distance volume to feed to the final registration stage at 58. Thus, the method returns from 60 to 58 to iteratively refine the registration for the next image frame, which has been coarsely aligned at 56. Method 50 may, for example, employ iterative nearest point alignment at 58 after a coarse initial alignment of the two frames at 56. By using the alignment of the previous frame as the coarse alignment, followed by iterative nearest point alignment on the complete acquired 3D surface information, it helps ensure that the coarse alignment is correctly calculated for all frames, while utilizing the full 3D surface alignment to promote good accuracy. This coarse alignment at 56 also allows for easy restarting of scanning for missing regions if necessary, without needing to maintain continuity with previous frames. As mentioned, for example, missing electrodes can be identified in response to determining that one or more electrodes are missing from the complete 3D surface based on the electrode group identified at 54.

[0049] All visible surfaces on which electrodes are located should be scanned by the user via an optical camera to ensure the system has torso electrode locations for generating geometric data used in solving inverse problems. Additionally, by identifying electrodes for each acquired image frame during surface scanning, the system is able to guide the user in response to the detection that one or more parts of the patient's body have not yet been scanned. For example, if one or more missing electrodes are identified, the method can generate guidance (e.g., by providing visual and / or audible feedback) to indicate which(s) electrode(s) is missing. If one or more electrodes are hidden or otherwise cannot be imaged, the user can input user input to ignore that(e.g.) for guidance purposes. Guidance can also include indications of areas of the body surface requiring further image scanning based on the known relative arrangement of electrodes on the array. In response to the acquisition of one or more such new images, each newly acquired image (acquired at 52) ​​can be processed according to method 50 and stitched into a complete 3D surface image, as described herein. For example, in Figure 8 If electrodes 25, 28, and 27 are missing from the acquired image frame data, the computing device performing method 50 can provide visual (and / or audible) feedback to instruct the user to further scan the right chest region using a portable optical imaging device (e.g., a camera).

[0050] As mentioned, in many instances, during image scanning using optical imaging equipment, the patient lies supine on an EP table, which may conceal numerous electrodes distributed across the patient's back. Accordingly, the positions of these concealed electrodes in 3D space need to be estimated as described herein for use in constructing the complete 3D surface geometry for electrodes on the patient's body surface.

[0051] As an example, Figure 5 A method 100 is described that can be implemented to estimate the location of hidden electrodes using electromagnetic tracking (e.g., corresponding to an estimate at 24). It is understood that such EM tracking can be used in place of backplane computation and / or 3D camera surface acquisition, or in addition to backplane computation and / or 3D camera surface acquisition. For example, a six-degree-of-freedom (6DOF) sensor can be attached to an electrode array (e.g., an electrode array integrated on a vest, or other form of electrode array) at a known location relative to the electrode location group. In cases where these sensors lack visibility, the location and orientation of each of the six DOF sensors can be tracked in 3D space by a ranging imaging camera. An example of a usable 6DOF sensor is one available from Northern Digital to minimize the number of sensors, as the orientation information provided by the 6DOF sensors eliminates the need to place a sensor at each electrode. In one example, the sensors are positioned at a portion of the electrodes, such as being evenly distributed across the backplane electrode array. In another example, the sensors do not need to be co-located with the electrodes. By placing some sensors along with some electrodes in a co-location manner, the positions of these electrodes in 3D space can be easily determined to be the same as those of the sensors.

[0052] At position 102, the 3D position and orientation of the 6DOF sensor are determined based on EM tracking data. In an example where the sensor is co-located with corresponding electrodes (e.g., some or all), the positions and orientations of these identified electrodes in 3D space can be specified as the same as the positions and orientations of the corresponding sensors. As an example, the tracking system includes a field generator that provides an EM field to induce current in the sensors on the electrode array. The induced current is provided as a tracking signal to the tracking system, which is configured to calculate the 3D position and orientation of each of the sensors in the tracking system's coordinate system.

[0053] At point 104, the surface traversing the EM sensor location (and any electrodes co-located with the sensor) is reconstructed. For example, given the known spatial arrangement of electrodes on a given electrode array (e.g., the back panel from a vest design) and the surface locations with orientation information at a subset of those electrodes co-located with the EM sensor, the remainder of the surface can be reconstructed to match the tangential information at known locations corresponding to the orientation and tangential information of the tracking data. The surface can be reconstructed as a mesh structure or other 3D surface construction having nodes at each defined sensor / electrode location specified by the tracking data.

[0054] At position 106, the remaining electrode positions are calculated on the reconstructed surface. The remaining unknown electrode positions can be calculated using the known relative distances between the electrodes in the electrode array and the distances across the reconstructed surface relative to the known electrode positions (e.g., obtained at position 102). By way of example, the unknown electrode positions are known to reside on the reconstructed surface (from position 104). Since the relative positions of all electrodes are known from the electrode array geometry, the relative distances from the known electrodes to the unknown electrodes can be obtained from the constraints of the electrode array geometry. Therefore, the position of each unknown electrode can be determined as a point where the distance across the surface to the nearest known electrode matches the known distance between each corresponding electrode pair (within a specified distance tolerance).

[0055] Figure 6 An example of a system 150 is depicted that can be used to generate geometric data 152 (such as geometric data that can be used in EP studies as disclosed herein). System 150 includes a medical imaging device 154 used to generate body imaging data 156. The medical imaging device can implement any one or more imaging modalities, such as those disclosed herein, that generate body imaging data, including internal structures within a patient and body imaging data of body surfaces. Body imaging data 156 can be stored in a memory accessible by one or more processors (not shown). The processor can execute instructions corresponding to methods for performing the functions disclosed herein.

[0056] For example, a processor may execute segmentation method 158 to segment structures from body imaging data 156. Segmentation may include determining the location of one or more structures of interest (such as the patient's heart) on the body surface and within the patient's body. Segmentation 158 may be used to generate torso and cardiac envelope geometry 160 that can be stored in memory. The torso and cardiac envelope geometry may correspond to a 2D or 3D mesh structure, or to other structures that define the spatial relationship between the body surface of the torso and the cardiac envelope (epicardial surface).

[0057] System 150 also includes a ranging imaging device 162, such as a portable optical ranging imaging device. The ranging imaging device 162 is configured to image the outer surface of a patient's body, on which an arrangement of electrodes is placed. The ranging imaging device 162 thereby provides ranging imaging data 164 corresponding to a set of image frames acquired by the device, which can be stored in a memory. In some examples, the ranging imaging data may also include or be combined with other positional information associated with the body surface. For example, such as relative to... Figure 5 The description suggests that additional position information can be generated from the electromagnetic tracking system based on the position of a tracking sensor attached to an electrode or substrate (to which the electrode is connected).

[0058] The ranging imaging data 164 includes a set of points in three-dimensional space sufficient to determine the location of each of a plurality of electrodes that have been positioned on the patient's torso. The ranging imaging data 164 may include multiple image frames that collectively encompass a sufficient portion of the patient's body for the entire torso to be determined. For example, each pixel may include image values ​​(e.g., color values ​​and depth values). Because there may be overlap between the images from the ranging imaging data 164, the processor executes instructions at 166 to perform image stitching. Therefore, image stitching can generate a three-dimensional image corresponding to a surface in a single volumetric mesh structure. For example, the stitched image generated via image stitching 166 can thus define a set of point clouds for the patient's body surface on which the electrodes have been placed. For example, image stitching 166 can perform image processing such as visual SLAM (simultaneous localization and mapping), image registration, and 3D reconstruction (e.g., with...). Figure 2 Method 50 corresponds to providing electrodes and 3D surface geometry 168. For example, geometry 168 may correspond to a dataset describing a mesh structure, or to a dataset describing other spatial structures (such as points represented on a body surface) describing the spatial relationships between electrodes. The set of point clouds generated for the surface constructed by image stitching 166 can thus generate the electrode and surface geometry 168. As disclosed herein, the electrode and surface geometry 168 may be stored in memory as corresponding data. In system 150, the torso and cardiac envelope geometry 166 and the electrode and surface geometry 168 may be decoupled from each other.

[0059] System 150 also includes registration method 170 (e.g., with...) Figure 1(Corresponding to 26), registration method 170 registers the trunk and cardiac envelope geometry 160 with the electrode and surface geometry 168 and provides corresponding geometry data 152. Registration method 170 can thus generate geometry data 152 including the 3-D spatial geometry location (e.g., corresponding to a point on a mesh structure) of each of a plurality of electrodes positioned on the patient's body surface and the 3-D spatial geometry location (e.g., corresponding to a point on a mesh structure) of the cardiac envelope (cardiac surface). Geometry data 152 can thus be stored in memory and used in conjunction with EP studies, as disclosed herein.

[0060] Figure 7 An example of another system 200 is depicted that can be used to generate geometric data 202 (such as geometric data used in EP studies performed on patients). System 200 and Figure 2 The example demonstrates a similar approach, except that medical imaging is not required to generate the patient's torso and cardiac envelope geometry. In system 200, template data is stored in memory. The template data may correspond to multiple templates in a database, such as atlases of multiple different body shapes. Template selection can occur based on patient criteria and in response to user input. This can be in response to selecting a specific template for a given patient or in response to measurements and other patient demographic information to adjust (deform) the body atlas. It can also be in response to previous medical imaging enhancement template selection for a given patient. In response to template selection and user input, the corresponding torso and cardiac envelope geometry can be generated at 208 and stored in memory. Thus, in this example, the torso and cardiac envelope geometry can represent a general or customized anatomical model defining the body surface and the cardiac envelope (heart surface).

[0061] System 200 also includes a ranging imaging device 210, such as a portable device. The ranging imaging device 210 is configured to image the outer surface of a patient's body, on which an arrangement of electrodes is placed. The ranging imaging device 210 thereby provides ranging imaging data 212 corresponding to a set of image frames acquired by the device, which can be stored in memory. In some examples, the ranging imaging data 212 may also include other location information associated with or in combination with the body surface, such as that disclosed herein (e.g., EM tracking data). The ranging imaging data 212 includes a set of points in three-dimensional space sufficient to determine the location of each of a plurality of electrodes that have been positioned on the patient's torso.

[0062] like Figure 6In one example, system 200 includes an image stitching method 214 that generates a set of point clouds of a three-dimensional image, such as a point cloud of a patient's body surface, corresponding to the electrodes and surface geometry 216, where the electrodes have been placed on the patient's body surface. As disclosed herein, the 3D electrodes and surface geometry 216 may be stored in memory.

[0063] System 200 also includes a registration method 218, which is executed to combine the trunk and cardiac envelope geometry 208 with electrode and surface geometry data 116 to generate geometry data. Thus, in some examples, the position of the heart in the general geometry data 208 can be roughly estimated for a given patient, and the registration method 218 can be fine-tuned in response to local electrical information acquired from the patient during the registration process. For example, the registration method 218 can be implemented and adjusted during EP studies, such as by identifying signal features, such as QRS complexes along a given portion of the patient's heart. In one example, a QRS complex can be detected or known to pass through a given path corresponding to the patient's diaphragm. Since the diaphragm can be spatially identified from the cardiac envelope geometry 208, the position of the patient's heart can be adjusted accordingly through the registration process to more accurately represent the estimate of the heart's surface position and orientation in the generated 3D geometry data 202.

[0064] For example, QRS complexes can be monitored at multiple points over a time period (e.g., 15 milliseconds) to determine the location of the diaphragm in a patient's heart. Once the location of the diaphragm has been identified within the patient, the cardiac envelope geometry represented in geometry data 202 and the location of the patient's heart can be adjusted to match the coordinates of the diaphragm, along which the QRS signal travels. As a further example, the QRS signal can be localized as a dipole within the patient to arrange and reposition the heart location represented in geometry data 202. While the QRS complex has been described above as an example of localized electrical information, it should be understood that any organized pattern can be utilized and localized within the patient's heart to adjust the location of the heart surface represented in the geometry data. Thus, locally organized patterns can be used to shift or adapt the cardiac model based on the location determined for the organized pattern.

[0065] exist Figure 6 as well as Figure 7 In each of the example systems, the ranging imaging device and the data it generates can be replaced by a single-view imaging device and the corresponding imaging data it generates. For example, markers can be applied at locations known relative to the electrodes, such as... Figure 8 , Figure 10 as well as Figure 11As illustrated in the example, a single-view imaging device can be used to generate image data comprising a series of images of electrodes and markers at different angles relative to the line of sight of the single-view imaging device. In this example imaging method (using markers in conjunction with single-view imaging), depth information of pixels in the image data can be derived at the center of the markers. The depth information (at the center or other marker locations) combined with pixels distributed throughout the image data can be used to construct (by image stitching) a three-dimensional image corresponding to the electrodes and surface geometries 168, 216 (such as a set of point clouds of a patient's body surface), wherein the electrodes have been placed on the patient's body surface. The electrodes and surface geometries can be stored in memory and registered with trunk and cardiac envelope geometries, as disclosed herein.

[0066] Figure 8 An example of an electrode array 300 attached to the front of a patient's torso model is depicted. For example, array 300 is configured to be attached to the patient's right torso. Another torso (not shown) will be configured to be attached to the left torso. Figure 8 The array shown includes a collection of multiple electrodes 302 and tracking markers 304 distributed at known locations on a flexible substrate material mesh. In this example, at least some of the tracking markers 304 are co-located, covering a subset of the electrodes 302. Thus, the 3D positions of the tracking markers 304 are determined from the acquired image frames, as disclosed herein. The tracking markers 304 are encoded to locate and identify the corresponding electrodes based on the determined position of each tracking marker. The positions of the remaining electrodes (not co-located with the tracking markers) can be determined based on the acquired positions of the tracking markers and the known relative positions of the electrodes across the array. Other tracking markers 306 can be located together with other portions of the array 300, such as for identifying the edges of the array or other predetermined locations.

[0067] Figure 9 An example of another electrode array (e.g., a back panel electrode) 400 is depicted, which is attached to the rear of the torso 402 (e.g., to cover the patient's back and perhaps some of the patient's sides). Each of the electrodes (e.g., numbered 127-252) has a known spatial position relative to the array 400. Thus, the 3D spatial positions of the remaining electrodes can be estimated as disclosed herein by determining the position of a portion of the electrodes and / or the substrate carrying the electrodes via ranging and / or single-view imaging.

[0068] Figure 10An example depicting a portion of another electrode array 500 (e.g., an electrode array corresponding to the electrodes on the right front panel) is shown. In this example, multiple tracking marks 502 are co-located with correspondingly identified electrodes and are annotated to assign unique indicators to the respective electrodes and provide orientation information (e.g., a 3D Cartesian coordinate system) determined for each such tracking mark. Tracking marks 504 may also be placed along one or more side edges of the array to facilitate array location identification when a portable imaging device is positioned to capture images from other viewpoints. By determining the location of a portion of the electrodes via ranging and / or single-view imaging, the 3D spatial location of the remaining electrodes can be estimated as disclosed herein.

[0069] As a further example, Figure 11 This is a side view depicting an example arrangement of the electrode (array) 600, showing the front electrode panel 610 and the rear electrode panel 620 attached to the torso when the patient is lying on the table 630. From this view, the imaging device can be positioned on tracking marks on both panels 610 and 620. From this view, one or more electrodes on each panel can also be visiblely identified, thereby enabling the reconstruction of the complete 3D surface geometry of the electrodes on the body surface.

[0070] Figure 12 An example of a 3D surface model 700 depicting a partial reconstruction of the torso is provided, showing the electrode marking locations of an array (e.g., a patient's right anterior torso electrode panel). In this example, the electrode locations are annotated as nodes at corresponding 3D locations as determined according to the system and method described herein. The electrode geometry spanning the body surface can be registered with body surface and cardiac envelope geometry calculated based on image data acquired via another imaging modality (by a processing device) to generate geometry data for use in performing an EP procedure that employs non-invasive electrical measurements from the electrodes to reconstruct cardiac electrical activity across the cardiac envelope.

[0071] Figure 13 Examples of system 750 are depicted, which can be used to perform medical tests (diagnosis, screening, and / or monitoring) and / or patient treatment. In some examples, system 750 may be implemented to generate in real time a corresponding electrocardiogram of the patient's heart 752 as part of a diagnostic procedure (e.g., an electrophysiological study) to help assess electrical activity in the patient's heart and identify drivers of arrhythmias. Additionally, or alternatively, system 750 may be used as part of a therapeutic procedure, such as to help a physician determine parameters for delivering treatment to a patient (e.g., delivery location, dosage, and type of treatment) based on one or more identified connection trajectories.

[0072] In examples of treatments where the treatment will be delivered to the patient's heart during such procedures, a catheter having one or more treatment delivery devices 756 attached thereto may be inserted into the patient's body 754 to contact the patient's heart 752 on the endocardium or epicardium. The placement of the treatment delivery device 756 may be guided by various positioning techniques and electrical information associated with the patient's heart. Those skilled in the art will understand and recognize that various types and configurations of treatment delivery devices 756 may be utilized, which may vary depending on the type of treatment and procedure. For example, the treatment device 756 may be configured for delivering electrotherapy, chemotherapy, sound therapy, thermotherapy, or any combination thereof.

[0073] As an example, the treatment delivery device 756 may include one or more electrodes positioned at the tip of an ablation catheter configured to generate heat for ablation of tissue in response to an electrical signal (e.g., radiofrequency energy) supplied by the treatment system 758. In other examples, the treatment delivery device 756 may be configured to deliver cooling to perform ablation (e.g., cryoablation), deliver chemicals (e.g., drugs), perform ultrasound ablation, high-frequency ablation, or a combination of these or other treatment mechanisms. In still other examples, the treatment delivery device 756 may include one or more electrodes positioned at the tip of a pacing catheter for delivering electrical stimulation, such as for pacing the heart, in response to an electrical signal (e.g., a pacing pulse) supplied by the treatment system 758. Other types of treatment may also be delivered via the treatment system 758 and the invasive treatment delivery device 756 positioned within the body.

[0074] As a further example, the treatment system 758 may be positioned outside the patient's body 754 and configured to control treatment being delivered by the device 756. For example, the treatment system 758 includes control (e.g., hardware and / or software) 760 that can transmit (e.g., provide) electrical signals via a conductive link electrically connected between the delivery device (e.g., one or more electrodes) 756 and the treatment system 758. The control system 760 can control parameters (e.g., current, voltage, repetition rate, trigger delay, sensed trigger amplitude) of signals provided to the device 756 for delivering treatment (e.g., ablation or stimulation) to one or more locations on the heart 752 via the electrodes(multiple) 754. The control circuitry 760 can set treatment parameters and apply stimulation based on automatic, manual (e.g., user input), or a combination of automatic and manual control (e.g., semi-automatic), which may be based on detection and associated characteristics of connection trajectories on the cardiac envelope. One or more sensors (not shown) may also transmit sensor information from the treatment device 756 back to the treatment system 758. The position of the device 756 relative to the heart 752 can be determined and tracked intraoperatively via imaging modalities (e.g., fluorescence examination, X-ray), mapping system 762, direct visualization, or other positioning systems. The position of the device 756 and treatment parameters can then be combined to determine and control the corresponding treatment parameter data.

[0075] exist Figure 13 In examples, such as the sensor array 764 disclosed herein, one or more electrodes may be used to record a patient's electrical activity. The sensing electrodes forming the array 764 may be mounted to a substrate (e.g., wearable clothing) applied to sensing electrode strips or individually mounted electrodes. As an example, the sensor array 764 may correspond to a high-density arrangement of body surface sensors (e.g., more than about 200 electrodes) distributed on a portion of a patient's torso for measuring electrical activity associated with the patient's heart (e.g., as part of an electrocardiogram (ECG) procedure). Examples of usable non-invasive sensor arrays are shown and described in International Application No. PCT / US2009 / 063803, filed November 10, 2009, which is incorporated herein by reference. Other arrangements and additional numbers of sensing electrodes may be used as sensor array 764. As an example, the array could be a reduced set of electrodes that does not cover the entire torso of the patient and is designed to measure electrical activity for a specific purpose (e.g., an electrode array specifically designed for analyzing atrial fibrillation and / or ventricular fibrillation) and / or to monitor electrical signals for a predetermined spatial region of the heart (e.g., (multiple) atrial regions or (multiple) ventricular regions).

[0076] One or more sensors may also be positioned on the device 756 inserted into the patient's body. Such sensors may be used separately or in combination with non-invasive sensors 764 for mapping electrical activity on endocardial surfaces (such as the walls of heart chambers) or epicardial surfaces. Additionally, such electrodes may be used to aid in positioning the device 756 within the heart 752, the device 756 being registered to an image or graph generated by the system 750. Alternatively, such positioning may be achieved without emitting signals from electrodes within or on the heart 752.

[0077] In each of these example methods for acquiring patient electrical information (including non-invasive, or a combination of invasive and non-invasive sensing), a sensor array 764 provides the sensed electrical information to a corresponding measurement system 766. The measurement system 766 may include appropriate controls and associated circuitry 768 for providing corresponding electrical measurement data 770, which describes the electrophysiological signals detected by sensors in the sensor array 764 on the body surface. The measurement data 770 may include analog and / or digital information.

[0078] Measurement control 768 can also be configured to control data acquisition processes (e.g., sampling rate, line filtering) for measuring electrical activity and providing measurement data 770. In some examples, control 768 may operate separately from the treatment system, such as controlling the acquisition of measurement data 770 in response to user input. In other examples, measurement data 770 may be acquired in real time, simultaneously and synchronously with the delivery of treatment by treatment system 758, such as for detecting electrical activity of the heart 752 in response to the application of a given treatment (e.g., according to treatment parameters). For example, appropriate timestamps may be used to index the temporal relationship between the corresponding measurement data 770 and the treatment parameters used to deliver treatment to facilitate its evaluation and analysis.

[0079] The mapping system 762 is programmed (e.g., instructions stored in non-transient memory and executable by one or more processing devices) to combine measurement data 770 corresponding to the electrical activity of the heart 752 with geometric data 772 (e.g., corresponding to the geometric data determined at 26 in method 10 and geometric data 152, 202) by applying appropriate processing and calculations to provide corresponding output data 774. As an example, the output data 774 may include one or more graphical diagrams illustrating the determined electrical activity reconstructed relative to the geometric surface (cardiac envelope) of the patient's heart 752 (e.g., information derived from electrical measurements superimposed on the surface of the heart 752).

[0080] For example, the electrogram reconstruction 780 can be programmed to compute an inverse solution and provide a corresponding reconstructed electrogram based on non-invasive electrical measurement data 770 and geometric data 772. Thus, the reconstructed electrogram can correspond to electrocardiographic activity across the cardiac envelope and can include static (three-dimensional at a given instant) and / or dynamic (e.g., a four-dimensional graph changing over time) graphs in one or more time intervals. Examples of inverse algorithms that can be implemented by the electrogram reconstruction 780 include those disclosed in U.S. Patent Nos. 7,983,743 and 6,772,004. The EGM reconstruction 780 can thus reconstruct body surface electrical activity measured via sensor array 764 at multiple locations (e.g., more than 1,000 locations, such as 2,000 or more locations) within the cardiac envelope.

[0081] As disclosed herein, the cardiac envelope may correspond to a three-dimensional surface geometry corresponding to the patient's heart (whose surface may be epicardium or endocardium). Alternatively or additionally, the cardiac envelope may correspond to a geometric surface residing between the epicardial surface of the patient's heart and the outer surface of the patient's body, wherein the electrodes forming the sensor array 764 are positioned in the outer surface of the patient's body. Additionally, the geometric data 772 utilized by the electrogram reconstruction 780 may correspond to the electrode locations determined according to the systems and methods disclosed herein. The cardiac envelope defined in the geometric data may be an actual patient anatomical geometry, a pre-programmed generic model, or a combination thereof (e.g., a model / template modified based on patient anatomy), such as those disclosed herein.

[0082] As an example, geometry data 772 represents the geometrical relationship in a three-dimensional coordinate system between the cardiac envelope (e.g., the cardiac surface) and electrodes positioned on the torso surface. As described herein, the geometrical relationship between the cardiac envelope and the torso surface can be obtained via a 3D medical imaging modality (such as CT or MRI) performed without the sensor array 764 being placed on a patient. A ranging imaging camera (e.g., a portable handheld RGB-D camera) can be used to capture multiple image frames, which are stitched together to digitize the electrodes positioned on the patient's torso and provide a point cloud set of the body surface including the electrode locations. The electrode and surface geometry data, along with the anatomical data of the cardiac envelope and the torso surface, are registered together to provide geometry data 772, as disclosed herein.

[0083] As mentioned above, the geometric data 772 may correspond to a mathematical model, such as a general model or a model constructed based on the patient's image data. Appropriate anatomical or other landmarks (including the positions of electrodes in the sensor array 764) may be identified in the geometric data 772 to facilitate the registration of the electrical measurement data 770, and the inverse method may be performed thereon. The identification of these landmarks may be done manually (e.g., by a human via image editing software) or automatically (e.g., via image processing techniques). By further example, ranging imaging and the generation of geometric data 772 may be performed before or concurrently with the recording of electrical activity, where the electrical activity is utilized to generate the electrical measurement data 770, or they may be performed separately (e.g., before or after the acquisition of measurement data). In some examples, the electrical measurement data 770 may be processed by a mapping system to extract locally organized patterns of activity to refine (refine) the geometric data 772 over one or more time intervals (e.g., during which a model or template is used to provide initial geometric data).

[0084] After (or simultaneously with) determining the potential data across the geometrical surface of the heart 752 (e.g., electrogram data calculated from non-invasively acquired measurements or from both non-invasive and invasive measurements), the electrogram data may further undergo signal processing by mapping system 762 to generate output data 774, which may include one or more graphical representations. Mapping system 762 may include one or more methods programmed to characterize electrical information across the cardiac envelope. For example, output generator 784 may be programmed to generate one or more graphical outputs (e.g., waveforms, electroanatomical diagrams, etc.) based on output data 774 for visualization on display device 794. Visualization engine 788 may control the characteristics of the displayed output. For example, parameters associated with the displayed graphical output can be selected in response to user input via a graphical user interface (GUI) 790. These parameters correspond to the output visualization of the calculated graph or waveform, and may include selecting time intervals, time and spatial thresholds, and the type of information to be presented on the display 794. For example, the user can use the GUI 790 to selectively program one or more parameters (e.g., time and spatial thresholds, filtering parameters, etc.) used by one or more methods to process the electrical measurement data 770. Thus, the mapping system 762 can generate corresponding output data 774, which can then be presented as a corresponding graphical output 792 on the display device 794. For example, the output generator 784 can generate an electrocardiogram and other output visualizations 792 on the display 794.

[0085] Because the measurement system 766 can simultaneously measure the electrical activity of a predetermined region, or simultaneously measure the electrical activity of the entire heart (e.g., where the sensor array 764 covers the entire thoracic cavity of the patient's body 754), the resulting output data (e.g., visually identified stable rotor and / or other electrocardiogram attributes) 774 can thus represent concurrent data of the predetermined region or the entire heart in a temporally or spatially consistent manner. The time interval can be selected based on user input (e.g., selecting a timer interval based on one or more waveforms), and the output data / graph is calculated for that time interval. Additionally or alternatively, the selected interval can be synchronized with the treatment applied by the treatment system 758.

[0086] Additionally, in some examples, output data 774 may be utilized by the treatment system 758. For example, the control system 760 may implement fully automatic control, semi-automatic control (partially automatic and responsive to user input), or manual control based on output data 774. In some examples, the control 760 of the treatment system 758 may utilize output data 774 to control one or more treatment parameters. As an example, the control 760 may control the delivery of ablation therapy to a site of the heart (e.g., the epicardium or endocardial wall) based on one or more arrhythmia drivers identified by one or more methods. In other examples, an individual may manually control the treatment system by viewing a graph generated on a display, such as using identified locations of connecting trajectories on the graph (e.g., areas between connecting trajectories) as treatment sites. Other types of treatments and devices may also be controlled based on output data 774 and the corresponding graph 792.

[0087] In view of the foregoing structural and functional description, those skilled in the art will understand that various parts of the systems and methods disclosed herein can be implemented as methods, data processing systems, or computer program products, such as non-transient computer-readable media. Accordingly, these parts of the methods disclosed herein can take the form of entirely hardware embodiments, entirely software embodiments (e.g., in non-transient machine-readable media), or embodiments combining software and hardware. Furthermore, various parts of the systems and methods disclosed herein can be computer program products on computer-usable storage media having computer-readable program code on that medium. Any suitable computer-readable medium can be utilized, including but not limited to: static and dynamic storage devices, hard disks, optical storage devices, and magnetic storage devices.

[0088] Certain embodiments are also described herein with reference to block diagrams of methods, systems, and computer program products. It will be understood that the illustrated blocks and combinations thereof can be implemented by computer-executable instructions. These computer-executable instructions can be provided to one or more processors of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus (or combination of devices and circuits) to produce a machine, whereby the instructions, executed via the processor, perform the functions specified in one or more blocks.

[0089] These computer-executable instructions can also be stored in a non-transitory computer-readable storage medium. These computer program instructions can direct a computer or other programmable data processing apparatus to operate in a particular manner, causing the instructions stored in the computer-readable storage medium to produce an article of writing including the instructions, which implement the functions specified in one or more flowchart blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing steps for implementing the functions specified in one or more flowchart blocks, whereby the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowchart blocks.

[0090] The above description is an example. It is certainly impossible to describe every conceivable combination of structure, component, or method, but those skilled in the art will recognize that many other combinations and substitutions are possible. Therefore, this invention is intended to cover all such alternatives, modifications, and variations falling within the scope of this application (including the appended claims). When this disclosure or claims refer to the element “a / an,” “a first,” or “another,” or its equivalents, it should be interpreted as including one or more such elements, without requiring or excluding two or more such elements. As used herein, the term “includes / including” means including but not limited to. The term “based on” means at least partially based on.

Claims

1. A method for medical imaging, comprising: Electrode arrays are placed on the surface of the patient's body. The positions of visible electrodes on the body surface are digitized based on one or more image frames acquired using optical imaging; Estimate the location of hidden electrodes on the body surface that are not visible during the optical imaging process; The positions of the visible and hidden electrodes on the body surface are registered with geometric information that spatially describes the anatomical envelope of the body surface and the patient's body. as well as Based on the registration, the geometric data is stored in a non-transient memory, and the geometric data defines the spatial relationship between the electrode and the anatomical envelope.

2. The method according to claim 1, characterized in that, The geometric information further includes a template that defines the geometric relationship between internal and external anatomical features.

3. The method according to claim 2, characterized in that, It further includes selecting the template from multiple templates in response to user input.

4. The method according to claim 2, characterized in that, Further includes: The geometrical relationship between at least some of the internal and external anatomical features is adjusted using local electrical information measured via at least some of the electrodes.

5. The method according to claim 1, characterized in that, Estimating the location of the hidden electrode further includes: The storage includes electrode geometry data describing the relative spatial positions of at least some of the hidden electrodes, with at least one visible electrode or tracking marker included in the electrode geometry data during optical imaging; and Determine the three-dimensional position of the at least one visible electrode or tracking marker. Specifically, the three-dimensional position of the hidden electrode is estimated based on the three-dimensional position determined for the at least one visible electrode or tracking mark and the electrode geometry data.

6. The method according to claim 1, characterized in that, Estimating the location of the hidden electrode further includes: The optical imaging is performed using an optical imaging device to generate imaging data of the electrode assembly within the direct line of sight during the optical imaging; Electromagnetic tracking is performed to generate additional data describing the three-dimensional positions of at least some of the hidden electrodes outside the direct line of sight during optical imaging; and The other data is registered with the imaging data to provide the three-dimensional position of the electrodes on the body surface.

7. The method according to claim 1, characterized in that, Estimating the location of the hidden electrode further includes: Optical imaging is performed using an optical imaging device to generate imaging data of the electrode assembly within the direct line of sight during the optical imaging process; Perform radiographic imaging to generate additional data describing the three-dimensional positions of at least some of the hidden electrodes outside the direct line of sight during the optical imaging; and The other data is registered with the imaging data to provide the three-dimensional position of the electrodes on the body surface.

8. The method according to claim 1, characterized in that, Further includes: The optical imaging is used to acquire multiple image frames from different perspectives; as well as The multiple image frames are stitched together to provide a combined image by associating them based on the position and identity of the electrodes in the multiple image frames.

9. The method according to claim 8, characterized in that, Each image frame includes electrodes and surface information within the line of sight of the optical imaging device performing the optical imaging, and the stitching further includes: The locations of the identified electrodes are used to perform coarse alignment between the plurality of image frames; and Final registration is performed across the multiple image frames based on pixel values ​​and depth information shared across the surface of the corresponding multiple images; and The registered surface data from each of the image frames is accumulated based on the final registration.

10. The method according to claim 8, characterized in that, Further includes: Guidance is generated for the user based on the positions of the electrodes determined for the plurality of image frames and identification information for one or more electrodes and / or positions missing from the plurality of image frames.

11. The method according to claim 1, characterized in that, Further includes: The electrodes are used to measure electrical signals from the body surface; Stores electrical measurement data representing electrical signals measured during one or more time intervals; as well as The electrical signal is reconstructed onto the anatomical envelope based on the electrical measurement data and the geometric data; as well as An output visualization is generated on the display based on the electrical measurement data and the geometric data.

12. A system for medical imaging, comprising: An optical imaging device configured to generate imaging data comprising one or more image frames of a patient’s body surface, the one or more image frames including a plurality of electrodes positioned on the body surface and within direct line of sight during image capture by the optical imaging device; Non-transient memory, the non-transient memory being used to store machine-readable instructions and data, the data including anatomical envelope and geometric information of the body surface; At least one processor, the at least one processor being configured to access the memory and execute the instructions to perform a method, the method comprising: The three-dimensional electrode positions and surface geometry are determined from the imaging data, including the estimated positions of hidden electrodes that are outside the direct line of sight during image capture by the optical imaging device. The determined three-dimensional electrode locations and surface geometries are registered with the geometric information of the anatomical envelope and the body surface to provide aggregated geometric data describing the three-dimensional spatial relationship between the plurality of electrodes and the anatomical envelope; and The summarized geometry data is stored in the memory.

13. The system according to claim 12, characterized in that, The method further includes tracking markers located at known positions relative to a portion of the electrodes, wherein the optical imaging device is configured to generate a series of image frames including the tracking markers, and wherein the processor is configured to access the memory and execute instructions to further determine the three-dimensional electrode position and surface geometry from the series of image frames based on the identification of the tracking markers in the series of image frames.

14. The system according to claim 12, characterized in that, The system further includes a radiographic imaging apparatus that generates radiographic imaging data, the radiographic imaging data including the hidden electrodes located outside the direct line of sight of the optical imaging apparatus. The processor is further configured to access the memory and execute the instructions to register the radiographic imaging data with the three-dimensional electrode positions and surface geometry determined from the imaging data, thereby providing registered imaging data, which is further registered with the anatomical envelope and the geometric information of the body surface to provide the aggregated geometric data.

15. The system according to claim 12, characterized in that, A further electromagnetic tracking system is provided, the electromagnetic tracking system being used to generate tracking data, the tracking data describing the three-dimensional position of an electromagnetic sensor having a known position relative to at least some of the hidden electrodes. The processor is further configured to access the memory and execute the instructions to register the tracking data with the three-dimensional electrode positions and surface geometry determined from the imaging data, thereby providing corresponding registered data, which is further registered with the anatomical envelope and the geometric information of the body surface to provide the aggregated geometric data.

16. The system according to claim 12, characterized in that, The optical imaging device is a portable optical imaging device, and the imaging data includes multiple image frames acquired from different viewpoints. The processor is further configured to access the memory and execute the instructions to stitch together the multiple image frames by associating them based on the position and identity of electrodes in the multiple image frames, in order to provide a combined image.

17. The system according to claim 12, characterized in that, Further includes: The sensor arrangement includes the plurality of electrodes and is configured to be placed on the body surface to non-invasively measure electrophysiological signals from the body surface. as well as A measurement system configured to provide electrical measurement data representing the measured electrophysiological signals. The at least one processor is further programmed to be used for: Electrophysiological signals are reconstructed onto the anatomical envelope based on the electrophysiological data and the summarized geometric data; and Based on the reconstructed electrophysiological signals, the graphical output is visualized.

18. One or more non-transient computer-readable media having instructions programmed to perform a method comprising: The system stores imaging data containing one or more image frames of a patient’s body surface, the image frames including multiple electrodes positioned on the body surface and within the direct line of sight during image capture by an optical imaging device. The three-dimensional electrode position and surface geometry are determined based on the imaging data, wherein the determined three-dimensional electrode position and surface geometry include the estimated position of a hidden electrode that is outside the direct line of sight during image capture by the optical imaging device; The determined three-dimensional electrode positions and surface geometry are registered with pre-determined anatomical geometry information to provide aggregated geometric data. The pre-determined anatomical geometry information describes the spatial relationship between the body surface and the anatomical envelope, while the aggregated geometric data describes the three-dimensional spatial relationship between the plurality of electrodes and the anatomical envelope. The summarized geometry data is stored in memory.

19. The medium according to claim 18, characterized in that, The method further includes: Generate additional spatial data describing the three-dimensional positions of at least some of the hidden electrodes and / or the three-dimensional positions of objects having known positions relative to at least some of the hidden electrodes; and The other spatial data is registered with the three-dimensional electrode locations and surface geometry determined from the imaging data to provide registered imaging data, which is further registered with the anatomical geometry information to provide the aggregated geometry data.

20. The medium according to claim 18, characterized in that, The method further includes: Storing electrical measurement data, said electrical measurement data representing electrophysiological signals measured from the body surface by at least some of the electrodes; and Electrophysiological signals are reconstructed onto the anatomical envelope based on the electrophysiological data and the summarized geometric data; and Based on the reconstructed electrophysiological signals, the graphical output is visualized.

Images