Integrated medical imaging

Abstract

Presented herein are methods and systems for integrating pre-operative and post-operative images of a subject to create an enhanced image. According to one aspect, a method includes obtaining a first image at a first time and obtaining second image at a second time. The first image is a three-dimensional (3D) image and the second image is a two-dimensional (2D) image. The method also includes determining a view angle associated with the first image. The view angle is associated with a 2D projection of the first image and is arranged to match the 2D projection with the second image. Matching the second image with the 2D projection includes extracting a feature from the first image and projecting the feature at the view angle onto a 2D plane to form a projected feature. The projected feature is transferred to the second image to create an enhanced second image.

Description

Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1INTEGRATED MEDICAL IMAGING BACKGROUNDField of the Invention[oooi] The present invention relates generally to techniques for integrating pre-operative and post-operative medical images to, for example, determine the positioning of an implant within a patient.Related Art

[0002] Medical devices are devices that are intended to be used for medical purposes. They can vary in both their intended use and indications for use. Examples range from simple, low-risk medical supplies to complex, potentially high-risk devices that are implanted and / or sustain life, such as deep brain stimulators and brain-computer interfaces. Other categories of medical device include diagnostic equipment.

[0003] Hearing devices act on an actual or potential auditory perception of an individual, including to improve perception of sound signals, to reduce perception of sound signals, etc. In particular, a hearing device can deliver sound signals to a user in any form, including in the form of acoustical stimulation, mechanical stimulation, electrical stimulation, etc., and / or can operate to suppress all or some sound signals. As such, a hearing device can be a device for use by a hearing-impaired person (e.g., hearing aids, middle ear auditory prostheses, bone conduction devices, direct acoustic stimulators, electro-acoustic hearing prostheses, auditory brainstem stimulators, bimodal hearing prostheses, bilateral hearing prostheses, dedicated tinnitus therapy devices, tinnitus therapy devices, etc.) or a device for use by a person with normal hearing (e.g., a consumer device that provides audio streaming, a consumer headphone, an earphone, etc.), a hearing protection device (e.g., a noise cancellation headset, a loudness reduction apparatus, etc.), etc.SUMMARY

[0004] In one aspect, a method is provided. The method comprises: obtaining a first image of a patient at a first time, the first image being a three-dimensional (3D) image; obtaining second image of the patient at a second time, the second image being a two-dimensional (2D) image, the second time being after the first time; determining a view angle associated with the first image, wherein the view angle is associated with a 2D projection of the first image, theAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1view angle being arranged to match the 2D projection of the first image with the second image; matching the second image with the 2D projection of the first image, wherein matching the second image with the 2D projection of the first image includes extracting a first feature from the first image and projecting the first feature at the view angle onto a 2D plane to form a projected first feature; and transferring the projected first feature to the second image to create an enhanced second image.

[0005] In another aspect, a method is provided. The method comprises: obtaining a computed tomography (CT) image, the CT image including a three-dimensional CT representation of a patient; obtaining an x-ray image, the x-ray image including a two-dimensional x-ray representation of the patient; processing the CT image, wherein processing the CT image includes projecting the three-dimensional CT representation to a plurality of two-dimensional CT representations; performing image registration, wherein performing image registration includes identifying a selected two-dimensional CT representation of the plurality of two-dimensional CT representations that aligns with the two-dimensional x-ray representation; and merging the x-ray image and the selected two-dimensional CT representation to create an enhanced image.

[0006] In yet another aspect, a method is provided. The method comprises: obtaining a preoperative image of a patient of an implantable medical device; obtaining a post-operative image of the patient, wherein the post-operative image includes an image of an implantable component of the implantable medical device; determining at least a view angle of the postoperative image; processing the pre-operative image to create a processed image, wherein the processed image is a view angle of the pre-operative image that aligns with the view angle of the post-operative image; and creating an enhanced image, wherein creating the enhanced image includes merging the post-operative image and the processed image, and wherein the enhanced image includes the image of the implantable component of the implantable medical device.

[0007] In still another aspect, a system is provided. The system comprises a memory, wherein instructions are stored in the memory; at least one processor operably coupled to the memory, wherein the at least one processor is configured to execute the instructions to: obtain a first image of a patient at a first time, the first image being a three-dimensional (3D) image; obtain second image of the patient at a second time, the second image being a two-dimensional (2D) image, the second time being after the first time; determine a view angle associated with the first image, wherein the view angle is associated with a 2D projection of the first image,Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1the view angle being arranged to match the 2D projection of the first image with the second image; match the second image with the 2D projection of the first image, wherein the instructions that cause the processor to match the second image with the 2D projection of the first image includes instructions that cause the processor to extract a first feature from the first image and to project the first feature at the view angle onto a 2D plane to form a projected first feature; and transfer the proj ected first feature to the second image to create an enhanced second image.

[0008] In yet another aspect, a method is provided. The method comprises obtaining an original pre-operative image of a subject, the original pre-operative image including at least one marker associated with the subject; obtaining a post-operative image of the subject; creating a projected pre-operative image from the original pre-operative image, wherein creating the projected pre-operative image includes estimating a view angle with respect to a plane, wherein estimating the view angle includes selecting a view angle that provides a match between the projected pre-operative image and the post-operative image, and wherein creating the projected pre-operative image includes extracting the at least one marker and projecting the at least one extracted marker onto the plane at the view angle; and generating an enhanced postoperative image using the projected pre-operative image and the post-operative image, wherein generating the enhanced image includes matching the projected pre-operative image and the post-operative image using an image registration method, and wherein generating the enhanced image further includes emphasizing at least one feature included in the post-operative image and transferring the at least one extracted marker to the enhanced post-operative image.

[0009] In another aspect, there is provided one or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to: obtain a computed tomography (CT) image, the CT image including a three-dimensional CT representation of a patient; obtain an x-ray image, the x-ray image including a two-dimensional x-ray representation of the patient; process the CT image, wherein the instructions operable to process the CT image include instructions operable to project the three-dimensional CT representation to a plurality of two-dimensional CT representations; perform image registration, wherein the instructions operable to perform image registration include instructions operable to identify a selected two-dimensional CT representation of the plurality of two-dimensional CT representations that aligns with the two-dimensional x-ray representation; and merge the x-ray image and the selected two-dimensional CT representation to create an enhanced image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1

[0010] In yet another aspect, a system is provided. The system comprises a memory, wherein instructions are stored in the memory; at least one processor operably coupled to the memory, wherein the at least one processor is configured to execute the instructions to: obtain a pre-operative image of a patient of an implantable medical device; obtain a post-operative image of the patient, wherein the post-operative image includes an image of an implantable component of the implantable medical device; determine at least a view angle of the postoperative image; process the pre-operative image to create a processed image, wherein the processed image is a view angle of the pre-operative image that aligns with the view angle of the post-operative image; and create an enhanced image, wherein the instructions operable to create the enhanced image include instructions operable to merge the post-operative image and the processed image, and wherein the enhanced image includes the image of the implantable component of the implantable medical device.

[0011] In still another aspect, a system is provided. The system comprises a memory, wherein instructions are stored in the memory; at least one processor operably coupled to the memory, wherein the at least one processor is configured to execute the instructions to: obtain an original pre-operative image of a subject, the original pre-operative image including at least one marker associated with the subject; obtain a post-operative image of the subject; create a projected pre-operative image from the original pre-operative image, wherein the instructions operable to create the projected pre-operative image includes estimating a view angle with respect to a plane, wherein estimating the view angle includes selecting a view angle that provides a match between the projected pre-operative image and the post-operative image, and wherein the instructions operable to create the projected pre-operative image include instructions operable to extract the at least one marker and to project the at least one extracted marker onto the plane at the view angle; and generate an enhanced post-operative image using the projected pre-operative image and the post-operative image, wherein the instructions operable to generate the enhanced image include instructions operable to match the projected pre-operative image and the post-operative image using an image registration method, and wherein the instructions operable to generate the enhanced image further include instructions operable to emphasize at least one feature included in the post-operative image and to transfer the at least one extracted marker to the enhanced post-operative image

[0012] In yet another aspect, a method is provided. The method comprises: obtaining a preoperative image of a patient of an implantable medical device; obtaining a post-operative image of the patient, wherein the post-operative image includes an image of an implantableAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1component of the implantable medical device; determining at least a view angle of the postoperative image; processing the pre-operative image to create a processed image, wherein the processed image is a view angle of the pre-operative image that aligns with the view angle of the post-operative image; and creating an enhanced image, wherein creating the enhanced image includes merging the post-operative image and the processed image, and wherein the enhanced image includes the image of the implantable component of the implantable medical device.

[0013] In still another aspect, there is provided one or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to: obtain a pre-operative image of a patient of an implantable medical device; obtain a post-operative image of the patient, wherein the post-operative image includes an image of an implantable component of the implantable medical device; determine at least a view angle of the post-operative image; process the pre-operative image to create a processed image, wherein the processed image is a view angle of the pre-operative image that aligns with the view angle of the post-operative image; and create an enhanced image, wherein the instructions operable to create the enhanced image include instructions operable to merge the post-operative image and the processed image, and wherein the enhanced image includes the image of the implantable component of the implantable medical device.

[0014] In yet another aspect, a system is provided. The system comprises: a memory, wherein instructions are stored in the memory; at least one processor operable coupled to the memory, wherein the at least one processor is configured to execute the instructions to: obtain a computed tomography (CT) image, the CT image including a three-dimensional CT representation of a patient; obtain an x-ray image, the x-ray image including a two-dimensional x-ray representation of the patient; process the CT image, wherein the instructions operable to process the CT image include instructions operable to project the three-dimensional CT representation to a plurality of two-dimensional CT representations; perform image registration, wherein the instructions operable to perform image registration include instructions operable to identify a selected two-dimensional CT representation of the plurality of two-dimensional CT representations that aligns with the two-dimensional x-ray representation; and merge the x-ray image and the selected two-dimensional CT representation to create an enhanced image.

[0015] In still another aspect, there is provided one or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause theAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1processor to: obtain an original pre-operative image of a subject, the original pre-operative image including at least one marker associated with the subject; obtain a post-operative image of the subject; create a projected pre-operative image from the original pre-operative image, wherein the instructions operable to create the projected pre-operative image includes estimating a view angle with respect to a plane, wherein estimating the view angle includes selecting a view angle that provides a match between the projected pre-operative image and the post-operative image, and wherein the instructions operable to create the projected preoperative image include instructions operable to extract the at least one marker and to project the at least one extracted marker onto the plane at the view angle; and generate an enhanced post-operative image using the projected pre-operative image and the post-operative image, wherein the instructions operable to generate the enhanced image include instructions operable to match the projected pre-operative image and the post-operative image using an image registration method, and wherein the instructions operable to generate the enhanced image further include instructions operable to emphasize at least one feature included in the postoperative image and to transfer the at least one extracted marker to the enhanced post-operative image.BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Embodiments of the present invention are described herein in conjunction with the accompanying drawings, in which:

[0017] FIG. 1 is a schematic diagram of a cochlear implant with which aspects of the techniques presented herein can be implemented;

[0018] FIG. 2A is a cross-sectional view of a patient's cochlea that has been partially cut-away to display the canals and to illustrate a position of a stimulating assembly in the cochlea;

[0019] FIG. 2B is a simplified schematic view of the cochlea of FIG. 2A;

[0020] FIG. 3 is a process flow diagram which illustrates a general method of generating and utilizing an enhanced medical image from at least one pre-operative image of a patient and at least one post-operative image of the patient in accordance with an embodiment presented herein;

[0021] FIG. 4 is a diagrammatic representation of a process and timeline associated with creating an enhanced image of a patient, in accordance with an embodiment presented herein;Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1

[0022] FIG. 5 is a process flow diagram which illustrates a method of generating and utilizing an enhanced image from a pre-operative three-dimensional (3D) image of a patient and a postoperative two-dimensional (2D) image of the patient in accordance with an embodiment;

[0023] FIG. 6 is a process flow diagram which illustrates a method of processing a preoperative 3D image to create a plurality of 2D pre-operative representations, in accordance with an embodiment presented herein;

[0024] FIG. 7 is a diagrammatic representation of creating a 2D representation from a 3D image utilizing a view angle in accordance with an embodiment presented herein;

[0025] FIG. 8 is a process flow diagram which illustrates a method of performing image registration to align a 2D pre-operative representation with a post-operative 2D image in accordance with an embodiment presented herein;

[0026] FIG. 9A is a diagrammatic representation of a process of performing image registration in accordance with an embodiment presented herein;

[0027] FIG. 9B is a diagrammatic representation of a process of performing image registration using images of a human head in accordance with an embodiment;

[0028] FIG. 10 is a process flow diagram which illustrates a method of merging a postoperative image with a selected pre-operative representation to create an enhanced image, in accordance with an embodiment presented herein;

[0029] FIG. 11A is a diagrammatic representation of an enhanced image in accordance with an embodiment presented herein;

[0030] FIG. 1 IB is a diagrammatic representation of an enhanced image of a human head in accordance with an embodiment presented herein;

[0031] FIG. 12 is a diagrammatic representation of an overall system which can generate an enhanced image in accordance with an embodiment presented herein;

[0032] FIG. 13 is a schematic diagram illustrating a vestibular stimulator system with which aspects of the techniques presented herein can be implemented; and

[0033] FIG. 14 is a schematic diagram illustrating a retinal prosthesis system with which aspects of the techniques presented herein can be implemented.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1DETAILED DESCRIPTION

[0034] Presented herein are techniques for integrating a plurality of medical images with one another to generate a so-called “enhanced” or “composite” medical image (referred to herein as an “enhanced image”) of a patient. For example, in accordance with certain embodiments presented a medical image obtained prior to a surgical procedure (referred to herein as a “preoperative image”) is integrated with an image obtained after, or during, the surgical procedure (referred to herein as a “post-operative image”). As described further below, the pre-operative image and the post-operative image are effectively integrated or merged to generate an enhanced image which can be used for a variety of purposes. For example, the enhanced image can be assessed to identify potential issues which can have arisen as a result of an operation, e.g., an issue with the positioning of a cochlear implant inserted into a body chamber of the patient during an operation.

[0035] As noted above, the term “post-operative image” is used to refer to an image that is taken during or after a surgical procedure. Therefore, in accordance with certain embodiments presented herein, the post-operative is taken intra-operatively (e.g., before a surgical procedure has been completed).

[0036] In certain embodiments, the techniques presented herein relate to integrated one or more three-dimensional (3D) images with one or more two-dimensional (2D) images. For example, a pre-operative 3D image can be obtained prior to a surgical procedure, while a post-operative 2D image can be obtained after (or during) the surgical procedure. Pre-operative 3D images can be obtained in a variety of manners, such as through computer tomography (CT) imaging or magnetic resonance imaging (MRI), while post-operative 2D images are generally obtained through radiography x-ray imaging. It should be appreciated, however, that the types of images obtained prior to and after a surgical procedure can vary and that the techniques presented herein can be used with a wide variety of imaging modalities. It should be appreciated that pre-operative 3D images may also be positron emission tomography (PET) images and images with other virtual 3D modalities.

[0037] CT images generally provide more information than x-ray images, and therefore, typically allow for a better interpretation of structures than provided by x-ray images. For example, tissues in x-ray images often appear noisier than tissues in CT images such as CBCT, PCCT, and Micro CT images. Further, as will be appreciated by those skilled in the art, tissues in x-ray images are super positioned or overlay ed. As such, detecting structures such as cochleaAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1and surround structures including, but not limited to including, semi-circular canals can be difficult in x-ray images. However, x-ray imaging provides a faster and less expensive solution than CT imaging after / during a surgical procedure or operation, and x-ray imaging is more accessible than CT imaging. Further, x-ray imaging provides lower radiation exposure than CT imaging.

[0038] In certain cases, x-ray images can be obtained to evaluate the positioning of a stimulating assembly (electrode array) of a cochlear implant after insertion. Automated evaluation of electrode insertion using x-ray images can be a relatively complex image processing task. Variability in x-ray images obtained from different devices, and / or from different clinical centers, can have significant variations. In addition, x-ray imaging acquisition protocols vary as there is not generally accepted protocol or standard.

[0039] However, at least some of the above issues can be addressed by the techniques presented herein that integrate / merge at least one pre-operative image with at least one postoperative image. In particular, by effectively merging, for example, a pre-operative CT image with a post-operative x-ray image, the assets of both types of images can be effectively leveraged to generate / form a composite or enhanced image. A composite or enhanced image (enhanced image herein) can include detailed structures shown in a pre-operative image and an implant shown in a post-operative image. As such, the positioning of the implant relative to detailed structures including, but not limited to including, landmarks, segmentations, and curves can be determined using the composite or enhanced image. For example, the positioning of an implant relative to a cochlea and surrounding structures can be determined by processing an enhanced image.

[0040] There are a number of different types of device in / with which the techniques presented herein can be implemented. Merely for ease of description, the techniques presented herein are primarily described with reference to a specific device. However, it is to be appreciated that the techniques presented herein can also be partially or fully implemented by any of a number of different types of devices or systems, including consumer electronic devices (e.g., consumer hearing devices, consumer computing devices such as mobile phones and tablets, audio equipment such as home theatre and car audio systems, etc.), computing systems (e.g., servers in data centers, Intemet-of-Things (loT) devices), various types of software systems, such as databases, machine learning and artificial intelligence systems, other medical devices, such as diagnostic equipment or life sustaining equipment, etc. For example, the techniques presented herein could be used in or with sensory protheses, including hearing aids and cochlear implants,Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1and various medical devices, such as pacemakers, drug delivery systems, implantable defibrillators, functional electrical stimulation devices, sleep disorder devices (e.g., sleep apnea devices), seizure devices (e.g., devices for monitoring and / or treating epileptic events), balance or movement disorder devices (e.g., vestibular stimulation devices), tinnitus management devices, visual implants (e.g., bionic eyes), and other neuromodulation devices (e.g., braincomputer interfaces).

[0041] FIG. 1 is perspective view of an exemplary cochlear implant 100 that can be implanted in a patient using techniques in accordance with embodiments presented herein. The cochlear implant 100 includes an external component 102 and an intemal / implantable component 104. The external component 102 is directly or indirectly attached to the body of the patient and typically comprises an external coil 106 and, generally, a magnet (not shown in FIG. 1) fixed relative to the external coil 106. The external component 102 also comprises one or more sound input elements 108 (e.g., microphones, telecoils, etc.) for detecting sound and a sound processing unit 112. The sound processing unit 112 can include, for example, a power source (not shown in FIG. 1) and a sound processor (also not shown in FIG. 1). The sound processor is configured to process electrical signals generated by a sound input element 108 that is positioned, in the depicted embodiment, by auricle 110 of the patient. The sound processor provides the processed signals to external coil 106 via a cable (not shown in FIG. 1).

[0042] The implantable component 104 comprises an implant body 114, a lead region 116, and an elongate intra-cochlear stimulating assembly 118. The implant body 114 comprises a stimulator unit 120, an internal / implantable coil 122, and an internal receiver / transceiver unit 124, sometimes referred to herein as transceiver unit 124. The transceiver unit 124 is connected to the internal coil 122 and, generally, a magnet (not shown) fixed relative to the internal coil 122.

[0043] The magnets in the external component 102 and implantable component 104 facilitate the operational alignment of the external coil 106 with the internal coil 122. The operational alignment of the coils enables the implantable coil 122 to transmit / receive power and data to / from the external coil 106. More specifically, in certain examples, external coil 106 transmits electrical signals (e.g., power and stimulation data) to implantable coil 122 via a radio frequency (RF) link. Implantable coil 122 is typically a wire antenna coil comprised of multiple turns of electrically insulated single-strand or multi-strand platinum or gold wire. The electrical insulation of implantable coil 122 is provided by a flexible molding (e.g., silicone molding). In use, transceiver unit 124 can be positioned in a recess of the temporal bone of the patient.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1Various other types of energy transfer, such as infrared (IR), electromagnetic, capacitive and inductive transfer, can be used to transfer the power and / or data from an external device to cochlear implant and FIG. 1 illustrates only one example arrangement.

[0044] Elongate stimulating assembly 118 is configured to be at least partially implanted in cochlea 130 and includes a plurality of longitudinally spaced intra-cochlear contacts 128. The contacts 128 collectively form a contact array 126 and can comprise electrical contacts and / or optical contacts.

[0045] Stimulating assembly 118 extends through an opening in the cochlea 130 (e.g., cochleostomy 132, the round window 134, etc.) and has a proximal end connected to stimulator unit 120 via lead region 116 that extends through mastoid bone 119. Lead region 116 couples the stimulating assembly 118 to implant body 114 and, more particularly, stimulator unit 120.

[0046] An intra-cochlear stimulating assembly, such as stimulating assembly 118, can be a perimodiolar stimulating assembly or a non-perimodiolar stimulating assembly. A perimodiolar stimulating assembly is a stimulating assembly that is configured to adopt a curved configuration during and / or after implantation into the patient's cochlea so as to have at least the distal section positioned close to the wall of the patient's modiolus (i.e., close to the modiolar wall). One type of non-perimodiolar stimulating assembly is a lateral stimulating assembly that is configured to be implanted so as to be positioned along the lateral wall of the patient's scala tympani (i.e., the wall that is opposite the modiolar wall). Another type of non-perimodiolar stimulating assembly is a mid-scala stimulating assembly which assumes a mid-scala position during or following implantation (i.e., positioned approximately midway between the modiolar wall and the lateral wall).

[0047] In general, the sound processor in sound processing unit 112 is configured to execute sound processing and coding to convert a detected sound into a coded signal corresponding to electrical signals for delivery to the patient. The coded signal generated by the sound processor is then sent to the stimulator unit 120 via the RF link between the external coil 106 and the internal coil 122. The stimulator unit 120 includes one or more circuits that use the coded signals, received via the transceiver unit 124, so as to output stimulation (stimulation current) via one or more stimulation channels that terminate in the intra-cochlear stimulating contacts 128. As such, the stimulation is delivered to the patient via the intra-cochlear stimulating contacts 128. In this way, cochlear implant 100 stimulates the patient's auditory nerve cells,Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1bypassing absent or defective hair cells that normally transduce acoustic vibrations into neural activity.

[0048] FIG. 2A is cross-sectional view of cochlea 130 illustrating stimulating assembly 130 partially implanted therein. FIG. 2B is a simplified top view of cochlea 130 illustrating stimulating assembly 130 partially implanted therein. Referring first to FIG. 2 A, cochlea 130 is a conical spiral structure that comprises three parallel fluid-filled canals or ducts, collectively and generally referred to herein as canals 236. Canals 236 comprise the tympanic canal 237, also referred to as the scala tympani 237, the vestibular canal 238, also referred to as the scala vestibuli 238, and the median canal 239, also referred to as the scala media 239. Cochlea 130 includes the modiolus 240 which is a conical shaped central region around which the cochlea canals 236 spiral. The modiolus 240 consists of spongy bone in which the cochlea nerve cells, sometimes referred to herein as the spiral ganglion cells, are situated. The cochlea canals 236 generally turn 2.5 times around the modiolus 240.

[0049] To insert intra-cochlear stimulating assembly 118 into cochlea 130, an opening (facial recess) is created through the patient's mastoid bone 119 (FIG. 1) to access the patient's middle ear cavity 141 (FIG. 1). The surgeon then creates an opening from the middle ear into the cochlea 130 through, for example, the round window, oval window, the promontory, etc. of the cochlea 130. The surgeon then gently advances (pushes) the stimulating assembly 118 forward into the cochlea 130 until the stimulating assembly 118 achieves a final implanted position. As shown in FIGS. 2A and 2B, the stimulating assembly 118 follows the helical shape of the cochlea 130. That is, the stimulating assembly 118 spirals around the modiolus 212.

[0050] In normal hearing, sound entering auricle 110 (FIG. 1) causes pressure changes in cochlea 130 that travel through the fluid-filled tympanic and vestibular canals 237, 238. The organ of Corti 210, which is situated on basilar membrane 244 in scala media 239, contains rows of hair cells (not shown) which protrude from its surface. Located above the hair cells is the tectoral membrane 245 which moves in response to pressure variations in the fluid-filled tympanic and vestibular canals 237, 238. Small relative movements of the layers of membrane 245 are sufficient to cause the hair cells to move, thereby causing the creation of a voltage pulse or action potential which travels along the associated nerve fibers that connect the hair cells with the auditory nerve 246. Auditory nerve 246 relays the impulses to the auditory areas of the brain (not shown) for processing.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1

[0051] Typically, in cochlear implant patients some portion of the cochlea 130 (e.g., the hair cells) is damaged such that the cochlea cannot transduce pressure changes into nerve impulses for relay to the brain. As such, the contacts 128 of the stimulating assembly 118 are used to directly stimulate the cells to create nerve impulses resulting in perception of a received sound. In the specific embodiments illustrated herein, stimulating assembly 118 comprises twenty-two (22) intra-cochlear contacts 128(1) through 128(22) that can deliver stimulation to the cochlea 130. Contact 128(1) is the most proximal / basal contact (i.e., the contact configured to be implanted closest to the basal end of the cochlea 130), while intra-cochlear contact 128(22) is the most distal / apical contact (i.e., located closed to the cochlea apex 243). Due to the illustrative view, only a subset of the twenty-two (22) intra-cochlear contacts 128(1) through 128(22) are visible in FIG. 2A.

[0052] A reference contact (not shown in FIGS. 2A and 2B) can also be provided. The reference contact is positioned outside of the patient's cochlea 130 and, as such, is sometimes referred to as an extra-cochlear electrode (ECE).

[0053] As noted above, the contacts 128(1)- 128(22) deliver stimulation to the cochlea 130 to evoke a hearing percept. The effectiveness of the stimulation depends, at least in part, on the place along basilar membrane 244 where the stimulation is delivered. That is, the cochlea 130 has characteristically been referred to as being “tonotopically mapped” in that regions of the cochlea toward the basal end are more responsive to high frequency signals, while regions of cochlea 130 toward the apical end are more responsive to low frequency signals. These tonotopic properties of cochlea 130 are exploited in a cochlear implant by delivering stimulation within a predetermined frequency range to a region of the cochlea that is most sensitive to that particular frequency range. However, this stimulation relies on the particular contacts 128(1)- 128(22) having a final implanted positioned adjacent to a corresponding tonotopic region of the cochlea 130 (i.e., a region of the cochlea that is sensitive to the frequency of sound represented by the contact).

[0054] To achieve a correct final implanted position, the distal end / tip 250 of the stimulating assembly 118 should be placed at a correct angular position, sometimes referred to herein as a correct angular insertion depth. As used herein, the angular position or angular insertion depth of the stimulating assembly 118 refers to the angular rotation of the distal end 250 from the cochlea opening 251 (e.g., round window, cochleostomy, etc.) through which the stimulating assembly enters the cochlea. As such, the angular position / angular insertion depth can be expressed in terms of how many angular degrees (°) the distal end 250 has traveled within theAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1cochlea 130 with respect to the cochlea opening 251. For example, an angular insertion depth of one hundred and eighty (180) degrees indicates that the distal end 250 has traveled around half (’A) of the first turn 276 of cochlea 130. An angular insertion depth of three hundred and sixty (360) degrees indicates that the distal end 250 has traveled completely around the first turn 276. Angular insertion depth, if achieved accurately, is a constant for all patients that enables correct frequency alignment (i.e., positioning of the contacts 128(1)-128(22) adjacent to a corresponding tonotopic region of the cochlea 130.

[0055] The cochlea 130 shown in FIG. 2B is defined so as to include a central axis 252 extending generally through the geometric center of the cochlea (e.g., through modiolus 240). The cochlea 130 is further defined to include a plurality of different angular reference points with respect to the central axis 252. In particular, a zero (0) degree angular reference point (0° point) 254 is a point within the scala tympani 237 that is located at or adjacent to the cochlea opening 251 through which the stimulating assembly 118 is inserted. A one hundred and eighty (180) degree angular reference point (180° point) 256 is a point within the scala tympani 237 that is diametrically opposite from the 0° point 254 (i.e., 180° point 256 is located on the opposite side of the modiolus 240 from 0° point 254). The 0° point 254 and 180° point 256 both lie within a reference plane 257 that passes through the central axis 252. As noted above, the scala tympani 237 spirals around the modiolus 240. As such, the 180° point 256 is further “up” the cochlea spiral (i.e., at a different level within the reference plane 257) than the 0° point 254.FIG. 2B illustrates the distal end 250 of the stimulating assembly 118 positioned within cochlea 130. The depth with which distal end 250 of stimulating assembly 118 can be positioned within cochlea 130 can vary.

[0056] As noted, the ability to determine whether a component of an implantable medical device, such as an electrode array of a cochlear implant, is positioned in a desired location within a patient can be important to determine during a surgical procedure. Such intraoperative operations can be used to ensure that any deviations from the desired location can be addressed before the surgery is completed. However, for reasons explained above, the modalities used to obtain post-operative images (e.g., obtained during or after a surgical procedure to place the implant in a body chamber) result in images that are often less detailed than the pre-operative images obtained prior to the surgical procedure. Using the techniques presented herein, it is possible to effectively merge or otherwise combining images obtained before and after (during) a surgical procedure to remediate the limitations associated with use of conventional post-operative imaging techniques.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1

[0057] More specifically, FIG. 3 is a process flow diagram which illustrates a general method of generating and utilizing an enhanced image from at least one pre-operative image of a patient and at least one post-operative image of the patient in accordance with an embodiment. A method 305 of generating and utilizing an enhanced image begins at a step 309 in which a preoperative image of a patient, or a subject, is obtained at a time T1. In general, the pre-operative image can be an original image obtained in advance of a surgical procedure or operation to place an implantable portion or an implantable medical device in a patient. The pre-operative image can be a 3D image such as a CT image or MRI image, although it should be understood that the pre-operative image is not limited to being a 3D image. By way of example, the preoperative image can be a 2D image such as an x-ray image.

[0058] In a step 313, a post-operative image of the patient is obtained at a time T2. At the time T2, an implantable portion of an implantable medical device has already been implanted in the body chamber of the patient. The post-operative image can be 2D image such as an x-ray image, and can generally have an associated angle. That is, the post-operative image of the body chamber of the patient can effectively be obtained at a particular view angle, or obtained with respect to a particular plane.

[0059] Once the post-operative image is obtained, process flow proceeds to an optional step 315 in which the pre-operative image is processed. The pre-operative image can be processed to effectively create a representation of the pre-operative image that can enable the preoperative image to be aligned with the post-operative image. That is, a representation of the pre-operative image can be generated in optional step 315 when pre-operative image is not in a format that can be substantially compared to the post-operative image. By way of example, when the pre-operative image is a 3D image, processing pre-operative image can create a 2D version or representation of the 3D image.

[0060] Image registration is performed at a step 321 to align the pre-operative image, or the processed pre-operative image, with the post-operative image. After image registration is performed, the pre-operative image, or the processed pre-operative image, is merged with the post-operative image in a step 325 to create an enhanced image. The enhanced image can include features or markers effectively extracted from the pre-operative image, or the processed pre-operative image, and features or markers obtained from the post-operative image. By way of example, the enhanced image can include one or more landmarks, segmentations, and / or curves as represented in the pre-operative image, and the implantable portion of the implantable medical device as represented in the post-operative image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1

[0061] Metrics are obtained, and post-processing is performed, on the enhanced image in a step 329. Obtaining metrics and performing post-processing on the enhanced image can enable any deviation from a desired or expected positioning of the implantable portion of the implantable medical device in terms of displacement, folding, and the like to be identified and, hence, addressed. Upon obtaining metrics and performing post processing, the method of generating and utilizing an enhanced image is completed.

[0062] FIG. 4 is a diagrammatic representation of a process of a timeline associated with creating an enhanced image of a patient, e.g., a body chamber of a patient, in accordance with an embodiment. At a time tl, an operation or surgical procedure is planned to introduce an implant into a body chamber 454 of a patient. Body chamber 454 includes a feature 452. Feature 452 can be, but is not limited to being, a landmark, a segmentation, or a curve that is present within body chamber 454.

[0063] At a time t2, a pre-operative image 410 of body chamber 454 is obtained. Pre-operative image 410 includes, or otherwise depicts, a body chamber representation 454’ of body chamber 454 and a feature representation 452’ of feature 452. In the described embodiment, preoperative image 410 is a 3D image such as a CT image. Body chamber representation 454’ can generally be a first representation of body chamber 454, and feature representation 452’ can generally be a first representation of feature 452.

[0064] At a time t3, an operation is performed to introduce an implant 404 into body chamber 454. The operation can be a procedure to introduce implant 404 into a cochlea (not shown) in body chamber 454.

[0065] At a time t4, after implant 404 is effectively implanted in body chamber 454, a postoperative image 412 of body chamber 454 is obtained. Post-operative image 412 can include a body chamber representation 454” of body chamber 454, a feature representation 452” of feature 452, and an implant representation 404’ of implant 404. In the described embodiment, post-operative image 412 is a 2D image such as an x-ray image that is effectively taken with respect to a particular plane of body chamber 454. Body chamber representation 454” can generally be a second representation of body chamber 454, feature representation 452” can generally be a second representation of feature 452, and implant representation 404’ can generally be a first representation of implant 404.

[0066] At a time t5, an enhanced image 414 of body chamber 454 is created using aspects of post-operative image 412 and aspects of pre-operative image 410. Enhanced image 414 canAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1be a 2D image, and can include body chamber representation 454” of post-operative image 412, implant representation 404’ of post-operative image 412, and feature representation 452’ of pre-operative image 410. As will be discussed in more detail below, feature representation 452’ can be a processed to form a 2D representation that is included in enhanced image 414.

[0067] As previously mentioned, a pre-operative image can be a 3D image, and a postoperative image can be a 2D image. When a pre-operative image is a 3D image and a postoperative image is a 2D image, a 2D representation of the pre-operative image can be created to facilitate the creation of an enhanced image. The 3D image can be projected from three dimensions to two dimensions. In other words, a pre-operative 3D image can be substantially mapped into a 2D representation that can be aligned with a post-operative 2D image to create an enhanced image. The alignment of the 2D representation of the pre-operative image with the post-operative 2D image can include, but is not limited to including, matching a view angle of the 2D representation of the post-operative 2D image with an angle at which the postoperative 2D image is obtained and / or identifying a view angle of the 2D representation of the post-operative 2D image that enables the 2D representation of the post-operative 2D image to align with a plane in which the post-operative 2D image is obtained.

[0068] Referring next to FIG. 5, a method of generating and utilizing an enhanced image from a pre-operative 3D image of a patient and a post-operative 2D image of the patient will be described in accordance with an embodiment. A method 505 of generating and utilizing an enhanced image begins at a step 509 in which a pre-operative 3D image or scan of a patent is obtained at a time T1.

[0069] In a step 513, a post-operative 2D image of the patient is obtained at a time T2, which is after time T1. Between time T1 and time T2, the patient undergoes an operation to place an implant in a body chamber. Thus, the post-operative 2D image includes a representation of the implant while the pre-operative 3D image does not include a representation of the implant.

[0070] The pre-operative 3D image is processed in a step 517 to create a plurality of 2D preoperative representations of the pre-operative 3D image. That is, the pre-operative 3D image can be mapped into a plurality of 2D representations of different views, or slices, of the preoperative 3D image. One method of creating a plurality of 2D pre-operative representations will be discussed below with reference to FIG. 6.

[0071] Once the pre-operative 3D image is processed, image registration is performed in a step 521 to align the 2D pre-operative representation with the post-operative 2D image such that aAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1suitable 2D pre-operative representation can be selected for margining with the post-operative 2D image. In other words, image registration enables one of the plurality of 2D pre-operative representations to be selected for use in creating an enhanced image. The selected 2D preoperative image can be the image which is characterized as substantially matching the postoperative 2D image. One method of performing image registration will be discussed below with respect to FIG. 8.

[0072] From step 521, process flow moves to a step 525 in which the post-operative 2D image is merged with the selected 2D pre-operative representation to create an enhanced image. The creation of the enhanced image generally includes, but is not limited to including, selecting at least one feature depicted in the selected 2D pre-operative representation to include in the enhanced image and selecting the implant depicted in the post-operative 2D image to include in the enhanced image. One method of merging the post-operative 2D image and the selected 2D pre-operative representation will be described below with reference to FIG. 10.

[0073] After the enhanced image is created, metrics are obtained and post-processing is performed on the enhanced image in a step 529. The metrics can include, but are not limited to including, geometrical metrics which indicate a position of an electrode array of an implant relative to features in a body chamber of a patient. The post-processing performed on the enhanced image can detect electrodes of an implant in the enhanced image, and can indicate a deviation of the implant from a desired location or positioning, e.g., tip foldover and / or dislocation. Upon obtaining metrics and performing post-processing, the method of generating and utilizing an enhanced image is completed.

[0074] With reference to FIG. 6, a method of processing a pre-operative 3D image to create a plurality of 2D pre-operative representations, e.g., step 517 of FIG. 5, will be described in accordance with an embodiment. A method or step 517 of processing a pre-operative 3D image begins at a step 517 in which a pre-operative 3D image is projected at a view angle N to create a 2D pre-operative representation N. View angle N can be measured with respect to a plane, e.g., a Cochlear plane. A view angle can generally be expressed as (0,c|)), and view angle N can be expressed as (0N,<|)N). AS shown in FIG. 7, a body 730 which can represent a body chamber can include an xz-plane which has view angle (0,c|)), and a 2D pre-operative representation 732 is generated when body 730 is projected at view angle (0,c|)). View angles can generally represent a view angle of a source of a C-arm x-ray camera to a three-dimensional structure such as a head of a patient. In one embodiment, and 0 and (|) can represent a pitch andAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1a yaw, respectively. In general, view angles can be view poses of an x-ray image with respect to a CT image. View poses can be expressed as, but are not limited to being expressed as, arc orbital rotations, axial or pivotal rotations, axial or pivotal angulations, horizontal movements, vertical movements, and panning movements.

[0075] In one embodiment, there can be a range of view angles (0,(|)) at which pre-operative 3D images can be projected. By way of example, 0 can be in a range between -0 and 0, and (|) can be in a range between -<F> and <t>, where 0 can have a value of approximately 180 degrees and <t> can have a value of approximately 180 degrees. In general, 0 and (|) can each be in a range between approximately zero degrees and approximately 180 degrees.

[0076] After the pre-operative 3D image is projected at view angle N, a determination is made in a step 613 as to whether the pre-operative 3D image is to be projected at additional view angles. If it is determined that the pre-operative 3D image is to be projected at additional view, the process flow proceeds to a step 617 in which N is incremented. Once N is incremented, process flow returns to step 609 in which the pre-operative 3D image is projected at view angle N to create a 2D pre-operative representation N.

[0077] Alternatively, if it is determined in step 613 that the pre-operative 3D image is not to be projected at additional view angles, the indication is that a desired number of 2D preoperative representations have been created. As such, in a step 621, the one ore more 2D preoperative representations are provided for image registration, and the method of processing a pre-operative 3D image is completed.

[0078] FIG. 8 is a process flow diagram which illustrates a method of performing image registration to align 2D pre-operative representations with a post-operative 2D image to select a 2D pre-operative representation to merge with the post-operative 2D image, e.g., step 521 of FIG. 5, in accordance with an embodiment. A method or step 521 of performing image registration begins at a step 809 in which 2D pre-operative representation M, which corresponds to view angle M, is obtained. Once 2D pre-operative representation M is obtained, 2D pre-operative representation M is processed with respect to a post-operative 2D image in a step 813. Processing 2D pre-operative representation M can generally include, but is not limited to including, transforming 2D pre-operative representation M such that 2D preoperative representation M can align with the post-operative 2D image. By way of example, transforming 2D pre-operative representation M can involve scaling or calibrating 2D preoperative representation M such that 2D pre-operative representation M can effectively alignAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1with, or overlay, the post-operative 2D image. It should be appreciated that in addition to, or in lieu of involving scaling and / or calibration, transforming 2D pre-operative representation M can also involve geometric transformations, color transformations, filtering, morphological transformations, and / or Fourier transformations. A geometric transformation involves changing the position, size, and / or orientation of an image can include, but is not limited to including rotation, scaling, translation, and shearing. A color transformation involves adjusting color properties of an image such as brightness, contrast, saturation, and color balance. Filtering involves applying filters to enhance or detect features within an image such as blurring, sharpening, edge detection, and noise reduction. A morphological transformation involves operations such as dilation, erosion, opening, and closing that process images based on shapes. A Fourier transformation involves converting an image from a spatial domain to a frequency domain to analyze frequency components.

[0079] In a step 817, at least one metric that is an indication of a quality of alignment between 2D pre-operative representation M and the post-operative 2D image. The quality of alignment can be determined based on a cost function such as an image similarity cost function, a mutual information cost function, a normalized cross-correlation cost function, a mean squared error cost function, a structural similarity index cost function, and / or a cost function that considers a distance between common landmarks in a plurality of images. A selected cost function can be processed to identify which 2D pre-operative representation M is to be selected.

[0080] Once at least one metric is determined for 2D pre-operative representation M, it is determined in a step 821 whether there is an additional 2D pre-operative representation to substantially process. If it is determined that there is an additional 2D pre-operative representation to process, then M is incremented in a step 823, and process flow returns to step 809 in which 2D pre-operative representation M is obtained.

[0081] Alternatively, if the determination in step 821 is that no additional 2D pre-operative representations effectively remain substantially unprocessed, then in a step 825, a 2D preoperative representation is selected for use to effectively merge with the post-operative 2D image. The selected 2D pre-operative representation can be identified from a set of 2D preoperative representations as effectively being the most suitable, as for example, “best” match to the post-operative2D image based on one ore more selected criteria. By way of example, the selected 2D pre-operative representation can be identified from the set of 2D pre-operative representations as having the most suitable alignment with the post-operative 2D image. UponAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1selecting a 2D pre-operative representation, the method of performing image registration is completed.

[0082] During an image registration process, a set of 2D pre-operative representations is typically analyzed based on one or more criteria, and the 2D pre-operative representation which meets the one or more criteria, or is determined to most closely match the one or more criteria, can be selected for use to effectively merge with a corresponding post-operative 2D image.

[0083] FIG. 9A is a diagrammatic representation of a process of performing image registration in accordance with an embodiment. A set 932 of 2D pre-operative representations 932a-n that are generated from a pre-operative image (not shown), e.g., a pre-operative 3D image, includes a first 2D pre-operative representation 932a, a second 2D pre-operative representation 932b, and an nth 2D pre-operative representation 932n. Each 2D pre-operative representation 932a-n can correspond to a view angle associated with the pre-operative image (not shown). In other words, each 2D pre-operative representation 932a-n has a different view angle and, hence, each 2D pre-operative representation 932a-n is a different projection of an associated pre-operative image (not shown).

[0084] 2D pre-operative representation 932a includes a first representation of a body chamber 954 and a first representation of a feature 952. First representation of body chamber 954 and first representation of feature 952 are depicted with respect to a first view angle. 2D preoperative representation 932b includes a second representation of a body chamber 954’ and a second representation of a feature 952’. Second representation of body chamber 954’ and second representation of feature 952’ are depicted with respect to a second view angle. 2D pre-operative representation 932n includes an nth representation of a body chamber 954” and an nth representation of a feature 952”. Nth representation of body chamber 954” and nth representation of feature 952” are depicted with respect to an nth view angle.

[0085] As shown, a post-operative 2D image 940 includes a post-operative representation of a body chamber 964, a post-operative representation of a feature 962, and a post-operative representation of an implant 904. It should be appreciated that post-operative representation of feature 962, first representation of feature 952, second representation of feature 952’, and nth representation of feature 952” are each representations of the same feature, as for example the same landmark, segmentation, and / or curve.

[0086] During image registration, 2D pre-operative representations 932a-n are analyzed or otherwise processed to effectively select one of 2D pre-operative representations 932a-n asAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1being suitable for merging with post-operative 2D image 940. Image registration can generally involve selecting one of 2D pre-operative representations 932a-n after comparing each 2D preoperative representation 932a-n with post-operative 2D image 940. In one embodiment, transformations can be performed on 2D pre-operative representations 932a-n to facilitate comparisons between 2D pre-operative representations 932a-n and post-operative 2D image 940. The selected one of 2D pre-operative representations 932a-n can have the most accurate alignment with post-operative 2D image 940 when the selected one of 2D pre-operative representations 932a-n is overlayed with post-operative 2D image 940. That is, the view angle associated with the selected one of 2D pre-operative representations 932a-n effectively matches an angle of post-operative 2D image 940.

[0087] In the embodiment as shown, 2D pre-operative representation 932a can be selected during image registration for merging with post-operative 2D image 940. First representation of feature 952 can provide a more accurate alignment with post-operative representation of feature 962 than provided by second representation of feature 952’ and nth representation of feature 952”.FIG. 9B shows a set 982 that includes a 2D pre-operative representation 982a with a representation of feature and a 2D pre-operative representation 982b with a representation of feature 992’. A selected 2D pre-operative representation from set 982 may be substantially matched with a post-operative 2D image 990 a post-operative representation of a feature 996, and a post-operative representation of an implant 904’. In one embodiment, 2D pre-operative representation 982a can be selected. FIG. 10 is a process flow diagram which illustrates a method of merging a post-operative 2D image with a selected 2D pre-operative representation to create an enhanced image, e.g., step 525 of FIG. 5, in accordance with an embodiment. A method or step 525 of merging a post-operative 2D image with a selected 2D pre-operative representation to create an enhanced image begins at a step 1009 in which common features between the selected 2D pre-operative representation and the post-operative 2D image are identified. By way of example, a feature such as a landmark that is represented in both the selected 2D pre-operative representation and the post-operative 2D image can be identified as a common feature or landmark. The feature can be one selected from a group including, but not limited to including, a cochlea, a vestibule, a cochlea round window, an apex, a modiolus, a scala tympani, a facial nerve, a chorda tympani, and a cochlea semi-circular canal.

[0088] Once features that are common between the selected 2D pre-operative representation and the post-operative 2D image are identified, a first feature that is represented in the post-Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1operative 2D image and is not represented in the 2D pre-operative representation is identified in a step 1013. For example, the first feature can be a representation of an implant that was implanted in a body chamber of a patient during an operation. It should be appreciated that such an implant is not represented in the selected 2D pre-operative representation.

[0089] After the first feature is identified, an enhanced image is created in a step 1017 by transferring representations of common features from the 2D pre-operative representation to an enhanced image. In one embodiment, transferring representations of common features from the 2D pre-operative representation can include effectively overwriting or otherwise replacing the common features in the post-operative 2D image to effectively create the enhanced image. In another embodiment, transferring representations of common features from the 2D preoperative representation can include transferring the representations to a new image to create the enhanced image.

[0090] In a step 1021, the representation of the first feature in the enhanced image is emphasized. When the post-operative 2D image is used to effectively create the enhanced image, the representation of the first feature can be highlighted. Highlighting the representation of the first feature can include, but is not limited to including, using a thresholding process followed by a morphological operations or machine learning-based image processing techniques. In one embodiment, highlighting may include adding color to a first feature such that the first feature appears in a particular color in an image. When the enhanced image is essentially a new image, emphasizing the representation of the first feature can include adding the representation of the first feature to the new image. The method of merging a post-operative 2D image with a selected 2D pre-operative representation to create an enhanced image is completed after the representation of the first feature is emphasized in the enhanced image.

[0091] Referring next to FIG. 11 A, an enhanced image created from a 2D pre-operative representation and a post-operative 2D image, e.g., 2D pre-operative representation 932a and post-operative 2D image 940 of FIG. 9A, will be described in accordance with an embodiment. An enhanced image 1142 generally includes features of both a 2D pre-operative representation and a post-operative 2D image. In the described embodiment, enhanced image 1142 is effectively composed of features from 2D pre-operative representation 932a and post-operative 2D image 940 of FIG. 9 A. That is, enhanced image 1142 is formed from merging 2D preoperative representation 932a with post-operative 2D image 940 of FIG. 9A. First representation of body chamber 954 and first representation of feature 952, which are obtained from 2D pre-operative representation 932a of FIG. 9A, are included in enhanced image 1142.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1Post-operative representation of implant 904 as shown in post-operative 2D image 940 of FIG.9A is also included in enhanced image 1142.

[0092] FIG. 11B is a representation of an enhanced image 1142’ that depicts first representation of feature 996 and a representation of an implant 904’, with respect to FIG. 9B. That is, enhanced image 1142’ is created from image 982a and image 990 of FIG. 9B.

[0093] An overall system which enables an enhanced image such as enhanced image 1142 to be generated will be described in accordance with an embodiment with respect to FIG. 12. An overall system or platform includes a computing system 1200 that is in communication with a network 1220. Computing system 1200 can be a single device, or can be a distributed across multiple devices. Typically, computing system 1200 includes a processing unit 1200a, a memory 1200b, a network adapter 1200c, one or more input devices 1200d, and one or more output devices 1200e. Network 1220 includes a CT scanner 1230 and x-ray equipment 1240 that is in communication with computing system 1220. For example, CT scanner 1230 and x-ray equipment 1240 can provide images or, more generally, data that can be processed and / or analyzed by computing system 1200. It should be appreciated that other equipment, as for example an MRI scanner, can be included in network 1220 in addition to, or in lieu of, CT scanner 1230 and / or x-ray equipment 1240.

[0094] Processing unit 1200a includes one or more hardware or software processors, e.g., central processing units (CPUs), that are configured to obtain and to execute software and / or logic embodied as code devices in memory 1200b. Enhanced image creation logic 1210 is embodied in, or otherwise stored in, memory 1200b. Processing unit 1200a can execute enhanced image creation logic 1210.

[0095] Memory 1200b is one or more software or hardware-based computer-readable storage media operable to store information accessible by the processing unit 1200a. Memory 1200b can store, among other things, instructions executable by the processing unit 1200a to implement applications and / or cause performance of operations described herein, as well as other data. Memory 1200b can be volatile memory (e.g., RAM), non-volatile memory (e.g., ROM), or combinations thereof. Memory 1200b can include transitory memory or non-transitory memory. Memory 1200b can also include one or more removable or non-removable storage devices. In examples, memory 1200b can include RAM, ROM, EEPROM (Electronically-Erasable Programmable Read-Only Memory), flash memory, optical disc storage, magnetic storage, solid state storage, or any other memory media usable to storeAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1information for later access. By way of example, and not limitation, memory 1200b can include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, other wireless media, or combinations thereof.

[0096] Enhanced image creation logic 1210 includes image processing logic 1210a, image registration logic 1210b, merge logic 1210c, and post-processing logic 1210d. When executed by processing unit 1200a, enhanced image creation logic 1210 can cause the techniques described above to be performed. Image processing logic 1210a is configured to process images obtained from network 1220, as for example a pre-operative 3D image obtained from CT scanner 1230. Image processing logic 1210a can process a pre-operative 3D image to obtain one or more 2D representations of the pre-operative 3D image at different view angles and / or in different planes. Image registration logic 1210b is generally configured to identify a selected processed image, e.g., a 2D representation of a pre-operative CT image obtained from CT scanner 1230, that can be merged with a corresponding post-operative image e.g., a postoperative 2D image or x-ray image obtained from x-ray equipment 1240. Image registration logic 1210b can include optional transformation logic 1212 that is arranged to apply one or more transformations to a processed image to effectively facilitate aligning a 2D representation of a pre-operative 3D image with a corresponding post-operative image. Merge logic 1210c is configured to identify features of a selected 2D representation of a pre-operative 3D image and features of a post-operative image to utilize in creating an enhanced image. Post-processing logic 1210d is configured to process an enhanced image, e.g., to identify metrics associated with features represented in the enhanced image. In one embodiment, post-processing logic 1210d can perform image processing, machine learning, and / or deep learning to determine metrics.

[0097] Network adapter 1200c is configured to enable computing system 1200 to access or otherwise communicate with endpoints on network 1220, e.g., CT scanner 1230 and x-ray equipment 1240. Network adapter 1200c can provide wired or wireless network access and can support one or more of a variety of communication technologies and protocols including, but not limited to including, Ethernet, cellular, Bluetooth, near-field communication, and RF, among others. It should be appreciated that in lieu of using network adapter 1200c to facilitate obtaining images on network 1220, images can also be provided through computer-readable storage devices including, but not limited to including, compact discs and / or USB drivesAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1

[0098] Input devices 1200d can include devices such as keyboards, touchscreens, keypads, mice, touchpads, and / or any other user interface device which enables commands to be provided to computing system 1200. Output devices 1200e can include devices such as display screens, printers, ports that can be interfaced with a compact disc, ports that can be interfaced with USB drives, and / or the like. For example, output device 1200e can enable an enhanced image and / or results of processing an enhanced image to be provided to a user of computing system 1200.

[0099] As previously described, the technology disclosed herein can be applied in any of a variety of circumstances and with a variety of different devices. Example devices that can benefit from technology disclosed herein are described in more detail in FIGS. 13 and 14. However, again, these examples are illustrative and the techniques of the present disclosure can be applied to facilitate implantation of other devices, such sleep disorder devices (e.g., sleep apnea devices), seizure devices (e.g., devices for monitoring and / or treating epileptic events), balance or movement disorder devices (e.g., vestibular stimulation devices), tinnitus management devices, visual implants (e.g., bionic eyes) and other neuromodulation devices (e.g., brain-computer interfaces), etc.[ooioo] FIG. 13 illustrates an example vestibular stimulator system 1002, with which embodiments presented herein can be implemented. As shown, the vestibular stimulator system 1002 comprises an implantable component (vestibular stimulator) 1012 and an external device / component 1004 (e.g., external processing device, battery charger, remote control, etc.). The external device 1004 comprises a transceiver unit 1060. As such, the external device 1004 is configured to transfer data (and potentially power) to the vestibular stimulator 1012.[ooioi] The vestibular stimulator 1012 comprises an implant body (main module) 1034, a lead region 1036, and a stimulating assembly 1016, all configured to be implanted under the skin / tissue (tissue) 1015 of the patient. The implant body 1034 generally comprises a hermetically-sealed housing 1038 in which RF interface circuitry, one or more rechargeable batteries, one or more processors, and a stimulator unit are disposed. The implant body 1034 also includes an intemal / implantable coil 1014 that is generally external to the housing 1038, but which is connected to the transceiver via a hermetic feedthrough (not shown).

[0102] The stimulating assembly 1016 comprises a plurality of electrodes 1044(l)-(3) disposed in a carrier member (e.g., a flexible silicone body). In this specific example, the stimulating assembly 1016 comprises three (3) stimulation electrodes, referred to as stimulation electrodesAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC11044(1), 1044(2), and 1044(3). The stimulation electrodes 1044(1), 1044(2), and 1044(3) function as an electrical interface for delivery of electrical stimulation signals to the patient’s vestibular system.

[0103] The stimulating assembly 1016 is configured such that a surgeon can implant the stimulating assembly adjacent the patient’s otolith organs via, for example, the patient’s oval window. It is to be appreciated that this specific embodiment with three stimulation electrodes is merely illustrative and that the techniques presented herein can be used with stimulating assemblies having different numbers of stimulation electrodes, stimulating assemblies having different lengths, etc.

[0104] In operation, the stimulating assembly 1016 or another component of the vestibular stimulator 1012 can be implanted using the enhanced imaging techniques presented herein.

[0105] FIG. 14 illustrates a retinal prosthesis system 1101 that comprises an external device 1110 configured to communicate with an implantable retinal prosthesis 1100 via signals 1151. The retinal prosthesis 1100 comprises an implanted processing module 1125, and a retinal prosthesis sensor-stimulator 1190 is positioned proximate the retina of a patient. The external device 1110 and the processing module 1125 can communicate via coils 1108, 1114.

[0106] In an example, sensory inputs (e.g., photons entering the eye) are absorbed by a microelectronic array of the sensor-stimulator 1190 that is hybridized to a glass piece 1192 including, for example, an embedded array of microwires. The glass can have a curved surface that conforms to the inner radius of the retina. The sensor-stimulator 1190 can include a microelectronic imaging device that can be made of thin silicon containing integrated circuitry that convert the incident photons to an electronic charge.

[0107] The processing module 1125 includes an image processor 1123 that is in signal communication with the sensor-stimulator 1190 via, for example, a lead 1188 that extends through surgical incision 1189 formed in the eye wall. In other examples, processing module 1125 is in wireless communication with the sensor-stimulator 1190. The image processor 1123 processes the input into the sensor-stimulator 1190 and provides control signals back to the sensor-stimulator 1190 so the device can provide an output to the optic nerve. That said, in an alternate example, the processing is executed by a component proximate to, or integrated with, the sensor-stimulator 1190. The electric charge resulting from the conversion of the incident photons is converted to a proportional amount of electronic current which is input to a nearbyAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1retinal cell layer. The cells fire and a signal is sent to the optic nerve, thus inducing a sight perception.

[0108] The processing module 1125 can be implanted in the patient and function by communicating with the external device 1110, such as a BTE unit, a pair of eyeglasses, etc. The external device 1110 can include an external light / image capture device (e.g., located in / on a behind-the-ear device or a pair of glasses, etc.), while, as noted above, in some examples, the sensor-stimulator 1190 captures light / images, in which sensor-stimulator 1190 is implanted in the patient.

[0109] In operation, the processing module 1125 or another component of the retinal prosthesis system 1101 can be implanted using the enhanced imaging techniques presented herein.[oono] As should be appreciated, while particular uses of the technology have been illustrated and discussed above, the disclosed technology can be used with a variety of devices in accordance with many examples of the technology. The above discussion is not meant to suggest that the disclosed technology is only suitable for implementation within systems akin to that illustrated in the figures. In general, additional configurations can be used to practice the processes and systems herein and / or some aspects described can be excluded without departing from the processes and systems disclosed herein.[oom] This disclosure described some aspects of the present technology with reference to the accompanying drawings, in which only some of the possible aspects were shown. Other aspects can, however, be embodied in many different forms and should not be construed as limited to the aspects set forth herein. Rather, these aspects were provided so that this disclosure was thorough and complete and fully conveyed the scope of the possible aspects to those skilled in the art.

[0112] As should be appreciated, the various aspects (e.g., portions, components, etc.) described with respect to the figures herein are not intended to limit the systems and processes to the particular aspects described. Accordingly, additional configurations can be used to practice the methods and systems herein and / or some aspects described can be excluded without departing from the methods and systems disclosed herein.

[0113] According to certain aspects, systems and non-transitory computer readable storage media are provided. The systems are configured with hardware configured to execute operations analogous to the methods of the present disclosure. The one or more non-transitory computer readable storage media comprise instructions that, when executed by one or moreAtty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1processors, cause the one or more processors to execute operations analogous to the methods of the present disclosure.

[0114] Similarly, where steps of a process are disclosed, those steps are described for purposes of illustrating the present methods and systems and are not intended to limit the disclosure to a particular sequence of steps. For example, the steps can be performed in differing order, two or more steps can be performed concurrently, additional steps can be performed, and disclosed steps can be excluded without departing from the present disclosure. Further, the disclosed processes can be repeated.

[0115] Although specific aspects were described herein, the scope of the technology is not limited to those specific aspects. One skilled in the art will recognize other aspects or improvements that are within the scope of the present technology. Therefore, the specific structure, acts, or media are disclosed only as illustrative aspects. The scope of the technology is defined by the following claims and any equivalents therein.

[0116] It is also to be appreciated that the embodiments presented herein are not mutually exclusive and that the various embodiments can be combined with another in any of a number of different manners.

Claims

Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC1CLAIMSWhat is claimed is:

1. A method comprising:obtaining a first image of a patient at a first time, the first image being a three-dimensional (3D) image;obtaining second image of the patient at a second time, the second image being a two-dimensional (2D) image, the second time being after the first time;determining a view angle associated with the first image, wherein the view angle is associated with a 2D projection of the first image, the view angle being arranged to match the 2D projection of the first image with the second image;matching the second image with the 2D projection of the first image, wherein matching the second image with the 2D projection of the first image includes extracting a first feature from the first image and projecting the first feature at the view angle onto a 2D plane to form a projected first feature; andtransferring the projected first feature to the second image to create an enhanced second image.

2. The method of claim 1 further including:identifying a second feature from the second image;identifying common structures in the 2D projection of the first image and the second image, wherein the common structures include a representation of the first feature in the second image, and wherein the second feature is not a common structure; andreplacing the common structures in the second image with the common structures in the 2D projection of the first image, wherein transferring the projected first feature to the second image includes replacing the representation of the first feature in the second image with the projected first feature.

3. The method of claim 2 wherein the patient includes an electrode arrangement at the second time, the electrode arrangement being implanted into the patient between the first time and the second time, wherein the second feature is associated with the electrode arrangement.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC14. The method of claim 3 wherein the second feature is associated with at least one selected from a group including a cochlea, a vestibule, a cochlea round window, an apex, a modiolus, and a cochlea semi-circular canal.

5. The method of claim 1, 2, 3, or 4, wherein the first image is a computed tomography (CT) image and the second image is an x-ray image.

6. The method of claim 1, 2, 3, or 4 wherein the first image is a magnetic resonance imaging (MRI) image or a positron emission tomography (PET) image, and the second image is an x-ray image.

7. The method of claim 1, 2, 3, or 4 wherein determining the view angle associated with the first image includes applying an optimization method to estimate the view angle that matches the 2D projection of the first image with the second image.

8. The method of claim 7 wherein matching the second image with the 2D projection of the first image includes applying an image registration method.

9. The method of claim 7 wherein matching the second image with the 2D projection of the first image includes transforming the 2D projection of the first image to align with the first image.

10. The use of the method of according to any one of claims 1-9 to implant a cochlear implant, a sleep disorder device, a seizure device, a balance or movement disorder device, a tinnitus management device, or a visual device.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC111. A method comprising:obtaining a computed tomography (CT) image, the CT image including a three-dimensional CT representation of a patient;obtaining an x-ray image, the x-ray image including a two-dimensional x-ray representation of the patient;processing the CT image, wherein processing the CT image includes projecting the three-dimensional CT representation to a plurality of two-dimensional CT representations; performing image registration, wherein performing image registration includes identifying a selected two-dimensional CT representation of the plurality of two-dimensional CT representations that aligns with the two-dimensional x-ray representation; and merging the x-ray image and the selected two-dimensional CT representation to create an enhanced image.

12. The method of claim 11 wherein the CT image includes at least one landmark and at least one segmentation, and the selected two-dimensional CT representation includes a first representation of the at least one landmark and a first representation of the at least one segmentation, and wherein merging the x-ray image and the selected two-dimensional CT representation to create the enhanced image includes transferring the first representation of the at least one landmark and the first representation of the at least one segmentation to the enhanced image.

13. The method of claim 12 wherein obtaining the CT image includes obtaining a preoperative CT image of the patient and obtaining the x-ray image includes obtaining a postoperative x-ray image after an implantable component of a medical device has been implanted in the patient.

14. The method of claim 13 wherein the x-ray image includes a first representation of the implantable component, and wherein merging the x-ray image and the first two-dimensional CT representation includes emphasizing the first representation of the implantable component in the enhanced image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC115. The method of claim 14 further including:performing image processing on the enhanced image using metrics associated with the first representation of the at least one landmark, the first representation of the at least one segmentation to the enhanced image, and the first representation of the implantable component.

16. The method of claim 14 further including:performing machine learning on the enhanced image using metrics associated with the first representation of the at least one landmark, the first representation of the at least one segmentation to the enhanced image, and the first representation of the implantable component.

17. The method of claim 11, 12, 13, 14, 15, or 16, wherein processing the CT image includes projecting the plurality of two-dimensional CT representations at a plurality of view angles, and wherein identifying the selected two-dimensional CT representation of the plurality of two-the selected two-dimensional CT representations comprises:identifying a two-dimensional CT representation of the plurality of two-dimensional CT representations that is projected at a view angle of the plurality of view angles that most closely aligns with a view angle of the x-ray image.

18. The use of the method of according to any one of claims 11-17 to implant a cochlear implant, a sleep disorder device, a seizure device, a balance or movement disorder device, a tinnitus management device, or a visual device.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC119. A method comprising:obtaining a pre-operative image of a patient of an implantable medical device; obtaining a post-operative image of the patient, wherein the post-operative image includes an image of an implantable component of the implantable medical device;determining at least a view angle of the post-operative image;processing the pre-operative image to create a processed image, wherein the processed image is a view angle of the pre-operative image that aligns with the view angle of the post-operative image; andcreating an enhanced image, wherein creating the enhanced image includes merging the post-operative image and the processed image, and wherein the enhanced image includes the image of the implantable component of the implantable medical device.

20. The method of claim 19 wherein the pre-operative image is a pre-operative computed tomography (CT) image and the post-operative image is a post-operative x-ray image, and wherein the processed image is a processed CT image and the enhanced image is an enhanced x-ray image.

21. The method of claim 20 wherein processing the pre-operative CT image to create the processed CT image includes:projecting the CT image from three dimensions to two dimensions at a plurality of view angles with respect to a plane to create a plurality of projected CT images; and identifying the processed CT image as a first view angle of the plurality of view angles that corresponds to the view angle of the post-operative x-ray image.

22. The method of claim 21 wherein the CT image of the patient includes a first image of a first feature of the patient, the processed CT image includes a processed first image of the first feature, and wherein the x-ray image of the patient includes a second image of the first feature.

23. The method of claim 22 wherein creating the enhanced x-ray image includes the first processed image of the first feature and not the second image of the first feature.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC124. The method of claim 23 wherein the first feature is one selected from a group including a cochlea, a vestibule, a cochlea round window, an apex, and wherein the implantable component of the implantable medical device is one selected from a group including an electrode, an electrode array, and an electrode contact.

25. The method of claim 24 further including:processing the enhanced x-ray image using image processing to assess positioning of the implantable component of the implantable medical device.

26. The method of claim 24 further including:processing the enhanced x-ray image using machine learning to assess positioning of the implantable component of the implantable medical device.

27. The method of claim 19, 20, 21, 22, 23, 24, 25, or 26, wherein the pre-operative image is a pre-operative magnetic resonance imaging (MRI) image and the post-operative image is a post-operative x-ray image, and wherein the processed image is a processed MRI image and the enhanced image is an enhanced x-ray image.

28. The method of claim 19 wherein the pre-operative image is a pre-operative computed x-ray image and the post-operative image is a post-operative x-ray image, and wherein the processed image is a processed x-ray image and the enhanced image is an enhanced x-ray image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC129. One or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to:obtain a first image of a patient at a first time, the first image being a three-dimensional (3D) image;obtain second image of the patient at a second time, the second image being a two-dimensional (2D) image, the second time being after the first time;determine a view angle associated with the first image, wherein the view angle is associated with a 2D projection of the first image, the view angle being arranged to match the 2D projection of the first image with the second image;match the second image with the 2D projection of the first image, wherein the instructions that cause the processor to match the second image with the 2D projection of the first image includes instructions that cause the processor to extract a first feature from the first image and to project the first feature at the view angle onto a 2D plane to form a projected first feature; andtransfer the projected first feature to the second image to create an enhanced second image.

30. The one or more non-transitory computer readable storage media of claim 29 further operable to:identify a second feature from the second image;identify common structures in the 2D projection of the first image and the second image, wherein the common structures include a representation of the first feature in the second image, and wherein the second feature is not a common structure; andreplace the common structures in the second image with the common structures in the 2D projection of the first image, wherein the instructions that cause the processor to transfer the projected first feature to the second image include instructions that cause the processor to replace the representation of the first feature in the second image with the projected first feature.

31. The one or more non-transitory computer readable storage media of claim 29 or 30 wherein the patient includes an electrode arrangement at the second time, the electrode arrangement being implanted into the patient between the first time and the second time, wherein the second feature is associated with the electrode arrangement.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC132. The one or more non-transitory computer readable storage media of claim 31 wherein the second feature is associated with at least one selected from a group including a cochlea, a vestibule, a cochlea round window, an apex, a modiolus, and a cochlea semi-circular canal.

33. The one or more non-transitory computer readable storage media of claim 29 or 30 wherein the first image is a computed tomography (CT) image and the second image is an x-ray image.

34. The one or more non-transitory computer readable storage media of claim 29 or 30 wherein the first image is a magnetic resonance imaging (MRI) image or a positron emission tomography (PET) image, and the second image is an x-ray image.

35. The one or more non-transitory computer readable storage media of claim 29 or 30 wherein the instructions operable to determine the view angle associated with the first image include instructions operable to apply an optimization method to estimate the view angle that matches the 2D projection of the first image with the second image.

36. The one or more non-transitory computer readable storage media of claim 35 wherein the instructions operable to match the second image with the 2D projection of the first image includes instructions operable to apply an image registration method.

37. The one or more non-transitory computer readable storage media of claim 35 wherein the instructions operable to match the second image with the 2D projection of the first image include instructions operable to transform the 2D projection of the first image to align with the first image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC138. One or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to:obtain a computed tomography (CT) image, the CT image including a three-dimensional CT representation of a patient;obtain an x-ray image, the x-ray image including a two-dimensional x-ray representation of the patient;process the CT image, wherein the instructions operable to process the CT image include instructions operable to project the three-dimensional CT representation to a plurality of two-dimensional CT representations;perform image registration, wherein the instructions operable to perform image registration include instructions operable to identify a selected two-dimensional CT representation of the plurality of two-dimensional CT representations that aligns with the two-dimensional x-ray representation; andmerge the x-ray image and the selected two-dimensional CT representation to create an enhanced image.

39. The one or more non-transitory computer readable storage media of claim 38 wherein the CT image includes at least one landmark and at least one segmentation, and the selected two-dimensional CT representation includes a first representation of the at least one landmark and a first representation of the at least one segmentation, and wherein the instructions operable to merge the x-ray image and the selected two-dimensional CT representation to create the enhanced image include instructions operable to transferring the first representation of the at least one landmark and the first representation of the at least one segmentation to the enhanced image.

40. The one or more non-transitory computer readable storage media of claim 39 wherein the instructions operable to obtain the CT image include instructions operable to obtain a preoperative CT image of the patient and the instructions operable to obtain the x-ray image include instructions operable to obtain a post-operative x-ray image after an implantable component of a medical device has been implanted in the patient.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC141. The one or more non-transitory computer readable storage media of claim 40 wherein the x-ray image includes a first representation of the implantable component, and wherein the instructions operable to merge the x-ray image and the first two-dimensional CT representation include instructions operable to emphasize the first representation of the implantable component in the enhanced image.

42. The one or more non-transitory computer readable storage media of claim 41 further including instructions operable to:perform image processing on the enhanced image using metrics associated with the first representation of the at least one landmark, the first representation of the at least one segmentation to the enhanced image, and the first representation of the implantable component.

43. The one or more non-transitory computer readable storage media of claim 41 further including instructions operable to:perform machine learning on the enhanced image using metrics associated with the first representation of the at least one landmark, the first representation of the at least one segmentation to the enhanced image, and the first representation of the implantable component.

44. The one or more non-transitory computer readable storage media of claim 38, 39, 40, 41, 42, or 43, wherein the instructions operable to process the CT image includes instructions operable to project the plurality of two-dimensional CT representations at a plurality of view angles, and wherein the instructions operable to identify the selected two-dimensional CT representation of the plurality of two-the selected two-dimensional CT representations includes instructions operable to:identify a two-dimensional CT representation of the plurality of two-dimensional CT representations that is projected at a view angle of the plurality of view angles that most closely aligns with a view angle of the x-ray image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC145. One or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to:obtain a pre-operative image of a patient of an implantable medical device; obtain a post-operative image of the patient, wherein the post-operative image includes an image of an implantable component of the implantable medical device;determine at least a view angle of the post-operative image;process the pre-operative image to create a processed image, wherein the processed image is a view angle of the pre-operative image that aligns with the view angle of the postoperative image; andcreate an enhanced image, wherein the instructions operable to create the enhanced image include instructions operable to merge the post-operative image and the processed image, and wherein the enhanced image includes the image of the implantable component of the implantable medical device.

46. The one or more non-transitory computer readable storage media of claim 45 wherein the pre-operative image is a pre-operative computed tomography (CT) image and the postoperative image is a post-operative x-ray image, and wherein the processed image is a processed CT image and the enhanced image is an enhanced x-ray image.

47. The one or more non-transitory computer readable storage media of claim 46 wherein the instructions operable to process the pre-operative CT image to create the processed CT image include instructions operable to:project the CT image from three dimensions to two dimensions at a plurality of view angles with respect to a plane to create a plurality of projected CT images; andidentify the processed CT image as a first view angle of the plurality of view angles that corresponds to the view angle of the post-operative x-ray image.

48. The one or more non-transitory computer readable storage media of claim 47 wherein the CT image of the patient includes a first image of a first feature of the patient, the processed CT image includes a processed first image of the first feature, and wherein the x-ray image of the patient includes a second image of the first feature.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC149. The one or more non-transitory computer readable storage media of claim 48 wherein the instructions operable to create the enhanced x-ray image include instructions operable to include the first processed image of the first feature and not the second image of the first feature.

50. The one or more non-transitory computer readable storage media of claim 49 wherein the first feature is one selected from a group including a cochlea, a vestibule, a cochlea round window, an apex, and wherein the implantable component of the implantable medical device is one selected from a group including an electrode, an electrode array, and an electrode contact.

51. The one or more non-transitory computer readable storage media of claim 50 further including instructions operable to:process the enhanced x-ray image using image processing to assess positioning of the implantable component of the implantable medical device.

52. The one or more non-transitory computer readable storage media of claim 50 further including instructions operable to:process the enhanced x-ray image using machine learning to assess positioning of the implantable component of the implantable medical device.

53. The one or more non-transitory computer readable storage media of claim 45, 46, 47, 48, 49, 50, 51, or 52, wherein the pre-operative image is a pre-operative magnetic resonance imaging (MRI) image and the post-operative image is a post-operative x-ray image, and wherein the processed image is a processed MRI image and the enhanced image is an enhanced x-ray image.

54. The one or more non-transitory computer readable storage media of claim 45, 46, 47, 48, 49, 50, 51, or 52 wherein the pre-operative image is a pre-operative computed x-ray image and the post-operative image is a post-operative x-ray image, and wherein the processed image is a processed x-ray image and the enhanced image is an enhanced x-ray image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC155. A system, comprising:a memory, wherein instructions are stored in the memory; andat least one processor operably coupled to the memory, wherein the at least one processor is configured to execute the instructions to:obtain a first image of a patient at a first time, the first image being a three- dimensional (3D) image;obtain second image of the patient at a second time, the second image being a two-dimensional (2D) image, the second time being after the first time;determine a view angle associated with the first image, wherein the view angle is associated with a 2D projection of the first image, the view angle being arranged to match the 2D projection of the first image with the second image;match the second image with the 2D projection of the first image, wherein the instructions that cause the processor to match the second image with the 2D projection of the first image includes instructions that cause the processor to extract a first feature from the first image and to project the first feature at the view angle onto a 2D plane to form a projected first feature; andtransfer the projected first feature to the second image to create an enhanced second image.

56. The system of claim 55 wherein the at least one processor is further configured to execute the instructions to:identify a second feature from the second image;identify common structures in the 2D projection of the first image and the second image, wherein the common structures include a representation of the first feature in the second image, and wherein the second feature is not a common structure; andreplace the common structures in the second image with the common structures in the 2D projection of the first image, wherein the instructions that cause the processor to transfer the projected first feature to the second image include instructions that cause the processor to replace the representation of the first feature in the second image with the projected first feature.

57. The system of claim 56 wherein the patient includes an electrode arrangement at the second time, the electrode arrangement being implanted into the patient between the first time and the second time, wherein the second feature is associated with the electrode arrangement.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC158. The system of claim 57 wherein the second feature is associated with at least one selected from a group including a cochlea, a vestibule, a cochlea round window, an apex, a modiolus, and a cochlea semi-circular canal.

59. The system of claim 55, 56, 57, or 58, wherein the first image is a computed tomography (CT) image and the second image is an x-ray image.

60. The system of claim 55, 56, 57, or 58, wherein the first image is a magnetic resonance imaging (MRI) image or a positron emission tomography (PET) image, and the second image is an x-ray image.

61. The system of claim 55, 56, 57, or 58, wherein the instructions to determine the view angle associated with the first image include instructions operable to apply an optimization method to estimate the view angle that matches the 2D projection of the first image with the second image.

62. The system of claim 61 wherein the instructions operable to match the second image with the 2D projection of the first image includes instructions operable to apply an image registration method.

63. The system of claim 61 wherein the instructions operable to match the second image with the 2D projection of the first image include instructions operable to transform the 2D projection of the first image to align with the first image.

64. The use of a system according to anyone of claims 55-63 to implant a cochlear implant, a sleep disorder device, a seizure device, a balance or movement disorder device, a tinnitus management device, or a visual device.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC165. A system, comprising:a memory, wherein instructions are stored in the memory; andat least one processor operably coupled to the memory, wherein the at least one processor is configured to execute the instructions to:obtain a computed tomography (CT) image, the CT image including a three- dimensional CT representation of a patient;obtain an x-ray image, the x-ray image including a two-dimensional x-ray representation of the patient;process the CT image, wherein the instructions operable to process the CT image include instructions operable to project the three-dimensional CT representation to a plurality of two-dimensional CT representations;perform image registration, wherein the instructions operable to perform image registration include instructions operable to identify a selected two-dimensional CT representation of the plurality of two-dimensional CT representations that aligns with the two-dimensional x-ray representation; andmerge the x-ray image and the selected two-dimensional CT representation to create an enhanced image.

66. The system of claim 65 wherein the CT image includes at least one landmark and at least one segmentation, and the selected two-dimensional CT representation includes a first representation of the at least one landmark and a first representation of the at least one segmentation, and wherein the instructions to merge the x-ray image and the selected two-dimensional CT representation to create the enhanced image include instructions to transfer the first representation of the at least one landmark and the first representation of the at least one segmentation to the enhanced image.

67. The system of claim 66 wherein the instructions to obtain the CT image include instructions to obtain a pre-operative CT image of the patient and the instructions operable to obtain the x-ray image include instructions to obtain a post-operative x-ray image after an implantable component of a medical device has been implanted in the patient.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC168. The system of claim 67 wherein the x-ray image includes a first representation of the implantable component, and wherein the instructions to merge the x-ray image and the first two-dimensional CT representation include instructions to emphasize the first representation of the implantable component in the enhanced image.

69. The system of claim 68 further including instructions to:perform image processing on the enhanced image using metrics associated with the first representation of the at least one landmark, the first representation of the at least one segmentation to the enhanced image, and the first representation of the implantable component.

70. The system of claim 68 further including instructions to:perform machine learning on the enhanced image using metrics associated with the first representation of the at least one landmark, the first representation of the at least one segmentation to the enhanced image, and the first representation of the implantable component.

71. The system of claim 65, 66, 67, 68, 69, or 70, wherein the instructions to process the CT image includes instructions to project the plurality of two-dimensional CT representations at a plurality of view angles, and wherein the instructions to identify the selected two-dimensional CT representation of the plurality of two-the selected two-dimensional CT representations includes instructions to:identify a two-dimensional CT representation of the plurality of two-dimensional CT representations that is projected at a view angle of the plurality of view angles that most closely aligns with a view angle of the x-ray image.

72. The use of a system according to anyone of claims 65-71 to implant a cochlear implant, a sleep disorder device, a seizure device, a balance or movement disorder device, a tinnitus management device, or a visual device.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC173. A system, comprising:a memory, wherein instructions are stored in the memory; andat least one processor operably coupled to the memory, wherein the at least one processor is configured to execute the instructions to:obtain a pre-operative image of a patient of an implantable medical device; obtain a post-operative image of the patient, wherein the post-operative image includes an image of an implantable component of the implantable medical device;determine at least a view angle of the post-operative image;process the pre-operative image to create a processed image, wherein the processed image is a view angle of the pre-operative image that aligns with the view angle of the post-operative image; andcreate an enhanced image, wherein the instructions operable to create the enhanced image include instructions operable to merge the post-operative image and the processed image, and wherein the enhanced image includes the image of the implantable component of the implantable medical device.

74. The system of claim 73 wherein the pre-operative image is a pre-operative computed tomography (CT) image and the post-operative image is a post-operative x-ray image, and wherein the processed image is a processed CT image and the enhanced image is an enhanced x-ray image.

75. The system of claim 74 wherein the instructions operable to process the pre-operative CT image to create the processed CT image include instructions to:project the CT image from three dimensions to two dimensions at a plurality of view angles with respect to a plane to create a plurality of projected CT images; andidentify the processed CT image as a first view angle of the plurality of view angles that corresponds to the view angle of the post-operative x-ray image.

76. The system of claim 75 wherein the CT image of the patient includes a first image of a first feature of the patient, the processed CT image includes a processed first image of the first feature, and wherein the x-ray image of the patient includes a second image of the first feature.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC177. The system of claim 76 wherein the instructions to create the enhanced x-ray image include instructions to include the first processed image of the first feature and not the second image of the first feature.

78. The system of claim 77 wherein the first feature is one selected from a group including a cochlea, a vestibule, a cochlea round window, an apex, and wherein the implantable component of the implantable medical device is one selected from a group including an electrode, an electrode array, and an electrode contact.

79. The system of claim 78 further including instructions to:process the enhanced x-ray image using image processing to assess positioning of the implantable component of the implantable medical device.

80. The system of claim 78 further including instructions to:process the enhanced x-ray image using machine learning to assess positioning of the implantable component of the implantable medical device.

81. The system of claim 73, 74, 75, 76, 77, 78, 79, or 80, wherein the pre-operative image is a pre-operative magnetic resonance imaging (MRI) image and the post-operative image is a post-operative x-ray image, and wherein the processed image is a processed MRI image and the enhanced image is an enhanced x-ray image.

82. The system of claim 73, 74, 75, 76, 77, 78, 79, or 80, wherein the pre-operative image is a pre-operative computed x-ray image and the post-operative image is a post-operative x-ray image, and wherein the processed image is a processed x-ray image and the enhanced image is an enhanced x-ray image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC183. A method comprising:obtaining an original pre-operative image of a subject, the original pre-operative image including at least one marker associated with the subject;obtaining a post-operative image of the subject;creating a projected pre-operative image from the original pre-operative image, wherein creating the projected pre-operative image includes estimating a view angle with respect to a plane, wherein estimating the view angle includes selecting a view angle that provides a match between the projected pre-operative image and the post-operative image, and wherein creating the projected pre-operative image includes extracting the at least one marker and projecting the at least one extracted marker onto the plane at the view angle; and generating an enhanced post-operative image using the projected pre-operative image and the post-operative image, wherein generating the enhanced image includes matching the projected pre-operative image and the post-operative image using an image registration method, and wherein generating the enhanced image further includes emphasizing at least one feature included in the post-operative image and transferring the at least one extracted marker to the enhanced post-operative image.

84. The method of claim 83 wherein the at least one marker is at least one selected from a group including a landmark and a tissue segmentation, and wherein the at least one feature is at least one selected from a group including an electrode array, an electrode, and an electrode contact.

85. The method of claim 83 or 84 wherein the pre-operative image is a CT image and the post-operative image is an x-ray image.

86. The method of claim 83 or 84 further including:applying image processing to the enhanced post-operative image to determine a complexity associated with the at least one feature.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC187. One or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to:obtain an original pre-operative image of a subject, the original pre-operative image including at least one marker associated with the subject;obtain a post-operative image of the subject;create a projected pre-operative image from the original pre-operative image, wherein the instructions operable to create the projected pre-operative image includes estimating a view angle with respect to a plane, wherein estimating the view angle includes selecting a view angle that provides a match between the projected pre-operative image and the postoperative image, and wherein the instructions operable to create the projected pre-operative image include instructions operable to extract the at least one marker and to project the at least one extracted marker onto the plane at the view angle; andgenerate an enhanced post-operative image using the projected pre-operative image and the post-operative image, wherein the instructions operable to generate the enhanced image include instructions operable to match the projected pre-operative image and the postoperative image using an image registration method, and wherein the instructions operable to generate the enhanced image further include instructions operable to emphasize at least one feature included in the post-operative image and to transfer the at least one extracted marker to the enhanced post-operative image.

88. The one or more non-transitory computer readable storage media of claim 87 wherein the at least one marker is at least one selected from a group including a landmark and a tissue segmentation, and wherein the at least one feature is at least one selected from a group including an electrode array, an electrode, and an electrode contact.

89. The one or more non-transitory computer readable storage media of claim 87 wherein the pre-operative image is a CT image and the post-operative image is an x-ray image.

90. The one or more non-transitory computer readable storage media of claim 87, 88, or 89, further including instructions to:apply image processing to the enhanced post-operative image to determine a complexity associated with the at least one feature.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC191. A system, comprising:a memory, wherein instructions are stored in the memory; andat least one processor operably coupled to the memory, wherein the at least one processor is configured to execute the instructions to:obtain an original pre-operative image of a subject, the original pre-operative image including at least one marker associated with the subject;obtain a post-operative image of the subject;create a projected pre-operative image from the original pre-operative image, wherein the instructions operable to create the projected pre-operative image includes estimating a view angle with respect to a plane, wherein estimating the view angle includes selecting a view angle that provides a match between the projected preoperative image and the post-operative image, and wherein the instructions operable to create the projected pre-operative image include instructions operable to extract the at least one marker and to project the at least one extracted marker onto the plane at the view angle; andgenerate an enhanced post-operative image using the projected pre-operative image and the post-operative image, wherein the instructions operable to generate the enhanced image include instructions operable to match the projected pre-operative image and the post-operative image using an image registration method, and wherein the instructions operable to generate the enhanced image further include instructions operable to emphasize at least one feature included in the post-operative image and to transfer the at least one extracted marker to the enhanced post-operative image.

92. The system of claim 91 wherein the at least one marker is at least one selected from a group including a landmark and a tissue segmentation, and wherein the at least one feature is at least one selected from a group including an electrode array, an electrode, and an electrode contact.

93. The system of claim 91 wherein the pre-operative image is a CT image and the postoperative image is an x-ray image.Atty. Docket No. 3065.087 li Client Ref. No. CID04037WOPC194. The system of claim 91, 92, or 93, wherein the at least one processor is further configured to execute instructions to:apply image processing to the enhanced post-operative image to determine a complexity associated with the at least one feature.

Images