Method and system for voice activated display in electrophysiology procedures
Voice-activated commands facilitate efficient operation of electrophysiology systems by translating spoken commands into system interactions, addressing the challenge of managing abundant data with limited screen space and user attention.
Patent Information
- Application Number
- JP2024080799
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-03-21
- Filing Date
- 2024-05-17
- Publication Date
- 2024-11-28
AI Technical Summary
The abundance of information displayed during electrophysiology procedures, such as electrograms and ablation data, poses challenges due to limited screen size and user attention, requiring manual navigation that is cumbersome and time-consuming, especially when the user's hands are occupied with catheter manipulation.
Implementing voice-activated commands to control the electrophysiology system, allowing users to issue commands without manual input, using a microphone to capture speech, analyze it for recognized commands, and translate them into keyboard or pointing device actions.
Enables efficient operation of the electrophysiology system by allowing hands-free command execution, reducing time delays and user burden, and enhancing system interaction efficiency.
Smart Images

Figure 2024166180000001_ABST
Abstract
Description
[Technical field]
[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 63 / 467,344, entitled “METHOD AND SYSTEM FOR VOICE ACTIAVTED DISLAY IN ELECTROPHYSIOLOGY PROCEDURES,” filed May 18, 2023, which is incorporated herein by reference.
[0002] FIELD OF THEINVENTION The present disclosure relates generally to displaying information during an electrophysiology procedure and, in particular, to a method for voice activation of an electrophysiology system. [Background technology]
[0003] Arrhythmia can be caused by problems with the heart's electrical conduction system, particularly electrical activity at one or more points or regions in the walls of the heart chambers.
[0004] To assess a patient's condition and determine a treatment, such as applying one or more ablations, it may be necessary to assess electrical activity at multiple locations on the heart wall. This activity can be obtained by multiple electrodes located at the distal tip of an intracardiac catheter inserted into one or more chambers of the heart.
[0005] A significant amount of information may be displayed to a user, such as a surgeon. Information may include, for example, but is not limited to, an electroanatomical model of the heart chamber, including a graphical representation of multiple signals obtained from the electrodes, information related to the site where ablation occurred and ablation parameters, and other types of information. In addition to manipulating the display, the electrophysiology system allows the user to perform multiple actions during operation, such as marking locations, measuring distances, etc. [Brief description of the drawings]
[0006] A more complete understanding of the present disclosure will be obtained from the following detailed description of the embodiments of the present disclosure when read in conjunction with the drawings. [Figure 1A] 1 is a schematic, depiction of a catheter-based electrophysiological (EP) mapping and ablation system, according to some exemplary embodiments of the present disclosure. [Figure 1B] 1B is an enlarged schematic diagram of the display shown in FIG. 1A according to some exemplary embodiments of the present disclosure. [Figure 1C] 1B is an enlarged schematic diagram of the display shown in FIG. 1A after receiving and implementing a voice command from a user, according to some exemplary embodiments of the present disclosure. [Diagram 2] 1 is a flowchart of steps in a method for operating a voice-activated display of an electrophysiology system, according to some exemplary embodiments of the present disclosure. [Diagram 3] FIG. 1 is a schematic block diagram of a computing platform for a voice-activated display of an electrophysiology system, according to some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0007] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known circuits, control logic, and details of computer program instructions for conventional algorithms and processes have not been shown in detail in order to avoid unnecessarily obscuring the present invention.
[0008] Software programming code embodying aspects of the invention is typically maintained in permanent storage, such as a computer readable medium. In a client-server environment, such software programming code may be stored on a client or a server. The software programming code may be embodied in any of a variety of known media for use with data processing systems, including, but not limited to, magnetic and optical storage devices, such as disk drives, magnetic tapes, compact discs (CDs), digital video discs (DVDs), and computer instruction signals embodied in a transmission medium, with or without a carrier wave upon which the signals are modulated. For example, the transmission medium may include a communications network, such as the Internet. Additionally, while the invention may be embodied in computer software, the functionality necessary to carry out the invention may alternatively be embodied, in part or in whole, using hardware components, such as application specific integrated circuits or other hardware, or some combination of hardware components and software.
[0009] Overview Arrhythmogenic tissue associated with atrial fibrillation can be identified by examining an intracardiac electrogram (IEGM) at one or more locations along the atrial wall, e.g., the inner wall, to detect local electrical potentials caused by depolarization at each of the one or more locations. The IEGM thus records cardiac activity from within the heart, e.g., during catheter manipulation. The IEGM is typically detected with one or more electrodes, e.g., one or more electrode pairs, on the distal tip of the intracardiac catheter. In some exemplary embodiments, the intracardiac catheter further includes a position sensor configured to track the position of the distal tip.
[0010] Modern catheters have multiple electrodes distributed across the catheter's multiple splines, and at any given time, each electrode samples electrical activity at a location. For example, some catheters may have dozens or even a hundred or more electrodes.
[0011] It will be appreciated that multiple electrodes can enable the simultaneous collection of a significant amount of data, providing a good picture of the electrical signals within the heart.
[0012] In addition, other types of information may be available for display to a user, such as the physician performing the procedure, for example information regarding the ablations performed and their parameters, as well as additional information items.
[0013] This wealth of information poses a challenge because not all of it can always be displayed in a comfortable way due to limited screen size and user attention. Thus, the user may need to switch between different views, for example to display a particular electrogram where the electrogram to be displayed may change over time, or to display the characteristics of an ablation that has been performed at a particular location.
[0014] Changing views or performing other activities may require navigating through options provided by the system's user interface (UI), opening menus, making selections within menus, typing keys that translate to hot keys, typing parameter values such as the number of selected electrodes of an electrogram that a user, e.g., a physician, wants to test, etc.
[0015] However, the user's hands are usually busy manipulating the catheter or other equipment and are not free to adjust the display or provide other commands to the system using the input device. Therefore, the user may ask a clinical assistant (CAS) having a computing platform operatively connected to the display that the user can see to adapt the display or perform an action for the user. This configuration also has significant drawbacks, such as the time delay until the view is updated, consisting of the time it takes the user to make a request and the time it takes the CAS to execute it, and the additional burden on the CAS, which occupies them and delays the performance of other necessary tasks.
[0016] In the following disclosure, the terms "command," "user command," or "user input command" are used interchangeably and refer to features or actions of the electrophysiology system that a user can activate using a series of one or more keyboard and / or mouse and / or other pointing device inputs, hotkeys, shortcuts, etc.
[0017] According to some embodiments of the present disclosure, commands may be activated by receiving voice commands from a user and translating them into a series of keyboard and pointing device inputs.
[0018] It should be understood that a user of a system, such as an electrophysiological system, may be required to memorize and use keyboard and / or pointing device actions to operate corresponding user commands, and it should be understood that such systems may include hundreds of user commands, and that a user may only be able to memorize and use a portion of them.
[0019] Thus, according to some exemplary embodiments, the user's voice may be received by a microphone associated with a computing platform executing the electrophysiological system. The microphone may be built into the computing platform or may be an external microphone connected to the computing platform via a wired or wireless connection. An application or device with voice analysis capabilities may be installed on or operably connected to the computing platform executing the electrophysiological system to receive and analyze the captured user's voice. The analyzed speech, including one or more words, may then be recognized to identify one or more user commands enabled for voice activation. Each such user command may be converted into a series of one or more codes equivalent to the code that may be generated when a user strikes a keyboard or performs a pointing device action to execute the user command. The code may be sent to an operating system driver, which generates an event, such as a keyboard or pointing device event, that is added or injected into an event queue (or message queue) of the underlying operating system, thereby triggering the necessary action. Thus, the voice spoken by the user initiates the execution of the user command as if the user had performed a series of keyboard or pointing device activities corresponding to the command.
[0020] In some embodiments, the word spoken to activate a user command may indicate the user command, such as "show electrode number" followed by a number to view the corresponding electrogram, or "mark location" to mark the current location of the catheter.
[0021] In some embodiments, to reduce confusion, voice activation may be enabled for only a subset of available user commands, e.g., the most frequently used ones. Confusion may be caused by a user saying an incorrect command that is similar to the command the user intended, or by misinterpreting words recognized in the audio, thereby executing an incorrect user command. Thus, in some examples, at most about 50%, about 40%, about 25%, or about 10% of the available user commands may be enabled via voice activation so that the user can easily remember and use them.
[0022] Some frequently used commands may include displaying certain data items, e.g., only electrograms acquired by certain electrodes and not other electrodes, electrograms acquired only by bipolar electrodes and not other electrodes, one or more tags indicating the ablation site, certain ablation site information, etc.
[0023] In some embodiments, only user commands that include words from a limited vocabulary may be identified, for example, by using small vocabulary speech recognition. In additional embodiments, user commands may also be provided aloud as free text that may be recognized to determine the closest user command enabled for voice activation.
[0024] In some embodiments, the application may also include a screen capture and image analysis module to extract information presented on the screen. The captured screen may be analyzed to determine system parameters such as the type of catheter used, the location of ablation, etc. Such information may be useful in determining user commands for which voice activation should be enabled. For example, if a catheter with 10 electrodes is used, the user may only ask to see electrograms acquired by any of the 10 electrodes, and a user command to show electrogram number 11 is not enabled.
[0025] By adding a voice activation mechanism to activate commands, especially frequently used commands, users, including physicians and / or CAS, can more efficiently operate the system, such as navigating through menus, without having to use their hands and without having to memorize multiple action sequences.
[0026] Furthermore, voice activation is provided external to the electrophysiology system, thus enabling retrofit of existing systems. Adopting voice activation according to the present disclosure does not require any additional certification, testing, version updates, etc. All options and user commands enabled by the electrophysiology system remain active and unchanged, but in addition, some are also enabled via voice activation.
[0027] Capturing the user's voice and acting on user commands provided as voice commands may continue in parallel with any other electrophysiological and display activity of the system throughout operation, thus allowing the user to always see the data items they need while keeping their hands free.
[0028] System Description Please refer to FIG. 1A, which shows an exemplary catheter-based electrophysiology mapping and ablation system 10. The system 10 may include multiple catheters that are percutaneously inserted by a physician 24 through a patient's vascular system and into a cavity or vasculature of the heart 12. Typically, a delivery sheath catheter is inserted into the left or right atrium near a desired location of the heart 12. One or more catheters may then be inserted into the delivery sheath catheter to reach a desired location within the heart 12. The multiple catheters may include catheters dedicated to sensing Intracardiac Electrogram (IEGM) signals, catheters dedicated to ablation, and / or catheters dedicated to both sensing and ablation. An exemplary catheter 14 configured to sense IEGMs is illustrated herein. However, it is understood that the present disclosure is equally applicable to any other type of catheter.
[0029] The physician 24 can place the distal tip 28 of the catheter 14 in contact with the heart wall to sense a target site in the heart 12. For ablation, the physician 24 can similarly place the distal end of an ablation catheter in contact with a target site for tissue ablation.
[0030] Catheter 14 is an exemplary catheter that includes one, and preferably multiple, electrodes 26 optionally distributed across multiple splines 22 at distal tip 28 and configured to sense IEGM signals. Catheter 14 may further include a position sensor 29 embedded in or near distal tip 28 for tracking the position and orientation of distal tip 28. Optionally and preferably, position sensor 29 is a magnetic-based position sensor including three magnetic coils for sensing three-dimensional (3D) position and orientation.
[0031] The magnetic-based position sensor 29 may operate in conjunction with a location pad 25 that includes a plurality of magnetic coils 32 configured to generate a magnetic field within a predetermined working volume. The real-time position of the distal tip 28 of the catheter 14 may be tracked based on the magnetic fields generated by the location pad 25 and sensed by the magnetic-based position sensor 29. Details of magnetic-based position sensing technology are described in U.S. Patent Nos. 5,5391,199; 5,443,489; 5,558,091; 6,172,499; 6,239,724; 6,332,089; 6,484,118; 6,618,612; 6,690,963; 6,788,967; and 6,892,091, which are incorporated herein by reference.
[0032] The system 10 may include one or more electrode patches 38 positioned for skin contact on the patient 23 to establish a positional reference for the location pads 25 and impedance-based tracking of the electrodes 26. For impedance-based tracking, a current is directed to the electrodes 26 and sensed at the electrode skin patch 38, so that the position of each electrode can be triangulated via the electrode patch 38. Details of impedance-based position tracking technology are described in U.S. Patent Nos. 7,536,218, 7,756,576, 7,848,787, 7,869,865, and 8,456,182.
[0033] Recorder 11 may record multiple electrograms captured by body surface ECG electrodes 18 and intracardiac electrograms (IEGMs) captured by electrodes 26 of catheter 14. Recorder 11 may include pacing capability for pacing the cardiac rhythm and / or may be electrically connected to a stand-alone pacer.
[0034] The system 10 may include an ablation energy generator 50 adapted to deliver ablation energy to one or more electrodes 26 at the distal tip of a catheter configured for ablation. The energy generated by the ablation energy generator 50 may include, but is not limited to, radio frequency (RF) energy or pulsed field ablation (PFA) energy, including unipolar or bipolar high voltage direct current pulses such as may be used to effect irreversible electroporation (IRE), or combinations thereof.
[0035] The patient interface unit (PIU) 30 may be an interface configured to establish electrical communication between the catheters, other electrophysiological equipment, a power source, and a workstation 55 that controls the operation of the system 10. The electrophysiological equipment of the system 10 may include, for example, multiple catheters, location pads 25, body surface ECG electrodes 18, electrode patches 38, an ablation energy generator 50, and a recorder 11. Optionally and preferably, the PIU 30 additionally includes processing capabilities for implementing real-time calculations of catheter positions and performing ECG calculations.
[0036] The workstation 55 may include a processor unit having a memory, a memory or storage device having suitable operating software stored therein, and a user interface capability. The workstation 55 may optionally provide multiple functions, including: (1) modeling the endocardial anatomical structure in three dimensions (3D) and rendering the model or anatomical map 20 for display on the display device 27, (2) displaying activation sequences (or other data) compiled from the recorded electrograms 21 with representative visual indicators or images superimposed on the rendered anatomical map 20 on the display device 27 or the display device 21, optionally in response to user commands provided using a user interface or voice activation, (3) displaying real-time positions and orientations of multiple catheters within the cardiac chambers, and (4) displaying sites of interest on the display device 27, such as locations where ablation energy has been applied. One commercially available product embodying elements of system 10 is available as the CARTO™ 3 system, available from Biosense Webster, Inc., 31A Technology Drive, Irvine, CA 92618.
[0037] The system may include a microphone for capturing speech by physician 24. The microphone may be embedded within the workstation 55. Additionally or alternatively, the microphone may be external to the workstation 55 and connected to the workstation 55 via a wired or wireless connection. The external microphone may be head-mounted on the head of physician 24, fixed in the room, or located elsewhere within the system or its environment.
[0038] The workstation 55 or another computing platform described in more detail below may further include modules for analyzing the captured audio from the physician 24, extracting user commands, displaying information on the display device 27 or 21, or taking other actions in accordance with the user commands.
[0039] Referring now to FIG. 1B, there is shown an expanded schematic diagram of a display shown on display device 27 of FIG. 1A, according to some exemplary embodiments of the present disclosure.
[0040] The diagram generally referenced 70 shows a map 72 of the heart color-coded according to some parameter, such as the latency of electrical activation of the heart. Map 70 also shows a catheter 74 having a number of splines, such as splines 76 and 78, each carrying one or more electrodes, such as electrodes 80 and 82.
[0041] FIG. 70 further includes a frame 84 on the right hand side showing multiple electrograms, such as electrograms 86 and 88, each arising from a different electrode of catheter 74.
[0042] Referring now to FIG. 1C, there is shown an enlarged schematic diagram of a display shown on display device 27 after receiving and executing audio including a user command from a user, in accordance with some exemplary embodiments of the present disclosure.
[0043] The user command indicated that the user only wanted to see the electrogram acquired by electrode A6. The user could then clearly see the electrogram acquired by electrode A6 without being obstructed by multiple electrograms. The user could cause the system to display only this electrogram by selecting a series of menu selections, typing the electrode number, etc. In a system according to the present disclosure, the user could also issue voice commands that could achieve the same effect without the user having to memorize a sequence of actions, while keeping the user's hands free.
[0044] Thus, map 72 is like that of FIG. 1B, but the right-hand box 84 shows only a single electrogram 90 provided by electrode A6.
[0045] It will be understood that a single electrogram as displayed in FIG. 1C is merely an example, and that multiple other user commands can be issued and activated by voice, such as showing one or more electrograms for a shorter period of time, showing only electrograms located on a particular spline, rotating the cardiac map 70 to a predetermined direction and angle, rotating the cardiac map 70 to a required direction and angle, etc.
[0046] Referring now to FIG. 2, there is shown a flowchart of steps in a method for voice activation of an electrophysiology system, according to some exemplary embodiments of the present disclosure.
[0047] It should be understood that this method is performed in addition to all steps associated with activating an electrophysiology system, including manipulation of one or more catheters and display of electrograms, ablation sites, and other data items.
[0048] In step 204, the audio signal may be received through an audio capture device configured to capture the voice of a user, such as a physician or CAS, and thus the audio capture device may output an audio signal including the user's voice throughout the operation.
[0049] At step 208, the audio signal may be analyzed to identify at least one predefined trigger phrase, which may include one or more words.
[0050] For example, whenever the user wants to issue a voice command, the user can say a predefined trigger word or phrase, such as "Carto", "Carto operate", "display please", "start display", etc.
[0051] Once the speech recognition module identifies the trigger phrase, in step 212, the same module or a different voice analysis module may continue to analyze the audio signal to retrieve user commands from a predefined set of user commands for which voice activation is enabled.
[0052] The user command may be identified using a classifier that receives an audio signal as input and outputs an identifier of one of the predefined user commands for which voice activation is enabled. The classifier may be trained on a plurality of audio signals and a label that associates each audio signal with a user command, such that the classifier learns connections between features of the audio signal and the associated user command. Then, during runtime, the classifier may identify the user command that best matches the audio signal. In some embodiments, the classifier may output, for each possible user command, a certainty level that the audio signal relates to the user command. The user command with the highest certainty level may then be selected, and the selected user command may be executed.
[0053] In some embodiments, if two or more commands have similar confidence levels, e.g., the confidence levels differ by at most a predetermined threshold, the two or more commands may be displayed textually on a display device or read aloud to the user using a speaker embedded within or external to the workstation 55, allowing the user to select any one of them or none of them.
[0054] If the user command is recognized, a confirmation tone or audio file may be played by the speaker, and if the user command is not recognized, for example if the highest confidence level associated with any of the user commands is below a predetermined threshold, the system may play an audio file indicating that the captured audio is unclear, not associated with a user command, incorrect, etc., or the user may be asked to repeat or confirm that the command recognized by the system is correct.
[0055] In step 216, the audio signal may be further analyzed to obtain one or more parameters related to the user command. For example, if the user command indicates that the user only wants to see a specific electrogram, the user must provide the number of the corresponding electrode. Thus, if in step 216 the audio signal is classified as a user command intended to see a specific electrogram, the speech may be further analyzed to retrieve the electrode number. The parameter recognition may also use small vocabulary speech recognition, where the vocabulary consists of words related to the possible parameters of the system, such as letters, numbers, etc. In some embodiments, recognizing the parameters may be performed by a corresponding trained classifier.
[0056] One or more codes may be generated corresponding to the identified user commands at step 220. The term code relates to binary codes or signals that may be provided to a driver of the underlying computing platform to indicate user actions such as key presses, mouse events, pointing device events, etc.
[0057] In step 224, the generated code may be provided to an operating system driver of a computing platform, such as workstation 55. The driver may generate a corresponding event and add or insert the event into an event queue. The event may then be processed as any other event on the queue, which is the same as what happens with events generated when a user hits a key or manipulates a pointing device.
[0058] In some embodiments, the application may also include an image capture module or device for capturing an image of the display of the electrophysiology system, and an image analysis module for analyzing the captured image.
[0059] Thus, in step 228, the image capture module may capture a representation of the electrophysiology system, for example as may be displayed on the display device 27, and in step 232 the image may be analyzed to obtain parameters of the electrophysiology system, such as catheter type, number of splines, number of electrodes, etc., without having to interrogate the electrophysiology system.
[0060] In such an embodiment, the identified parameters may be utilized during user command identification step 212 to limit the possibilities of what the physician said to commands and parameters that are relevant to the current way the physician is operating the system. For example, if the physician is using a diagnostic catheter with 20 electrodes and the speech recognition system is unsure as to whether the physician wants to see the signal from electrode 13 or electrode 30, the speech recognition system can use its knowledge that there are only 20 electrodes to assume that the physician was asked to indicate the signal from electrode 13. Speech recognition may be easier and more robust when the options are limited or configured to a predefined list.
[0061] In some embodiments, capturing and analyzing images of the display may be performed one or more times until the required information is obtained with a level of certainty that exceeds a threshold, and then does not need to be repeated as operation continues with the same catheter.
[0062] Step 204 and subsequent steps may be performed continuously as the user may issue further user commands and possibly modify previous commands. Thus, the display may continue to update with new data, for example as the electrogram is continually updated with new readings, and the displayed item(s) or their presentation may change if the user issues a new user command.
[0063] It should be understood that the user may also continue to provide user commands in any other manner, for example, through a user interface or by the CAS.
[0064] Referring now to FIG. 3, a block diagram of a computing platform 300 for operating a voice-activated display of an electrophysiology system is shown, according to some exemplary embodiments of the present disclosure.
[0065] It will be appreciated that the computing platform 300 may be embedded within the console 55, but may also be a stand-alone computing platform, or may be embedded elsewhere and in operative communication with the console 55.
[0066] Computing platform 300 may be implemented as one or more computing platforms that may be operatively connected to one another. For example, one or more remote computing platforms may be implemented, for example, on a cloud computer. The other computing platforms may be part of an associated organizational computer network. In other embodiments, all functionality may be provided by one or more computing platforms that are all part of the organizational network.
[0067] The computing platform 300 may include one or more processors 304, which may or may not be located on the same computing platform, and which may be one or more central processing units (CPUs), microprocessors, electronic circuits, integrated circuits (ICs), etc. The processor 304 may be configured to provide the required functionality, for example, by loading into memory and activating software modules stored in a storage device 312, which will be described in more detail below.
[0068] The computing platform 300 may include one or more I / O devices 306, such as a microphone for capturing the user's voice, a speaker for playing audio, a display that may be embodied as part of the display 21, 27 or another display, a keyboard, a mouse, a touch screen, or another pointing device.
[0069] The computing platform 300 may also include a communications device 308 for communicating with other devices or other computing platforms as needed, for example, to obtain information indicative of position and electrical measurements from a catheter procedure controller, to store data on a remote storage device, to obtain information regarding the ablation being performed, etc. The communications module 308 may be adapted to interface with any communications channel, such as a local area network (LAN), a wide area network (WAN), a cellular network, etc., and to use any associated communications protocol.
[0070] The computing platform 300 may include a storage device 312, such as a hard disk drive, a flash disk, a random access memory (RAM), a memory chip, etc. In some exemplary embodiments, the storage device 312 may hold program code operable to cause the processor 304 to perform operations associated with any of the modules listed below or steps of the method of FIG. 2 above. The program code may include one or more executable units, such as functions, libraries, stand-alone programs, etc., adapted to execute instructions as detailed below.
[0071] Alternatively or additionally, the instructions provided may be stored on a non-transitory, tangible computer readable medium, such as a magnetic, optical, or electronic memory.
[0072] The storage device 312 may include one or more voice analysis and user command determination modules 316. The voice analysis and user command determination module 316 may include a small vocabulary recognition module for recognizing trigger phrases preceding user commands and recognizing certain predefined user commands. The small vocabulary recognition module may also be useful in recognizing parameters associated with one or more user commands, such as electrode identifiers.
[0073] Additionally or alternatively, module 316 may comprise a large vocabulary recognition module, also referred to as a free speech recognition module, for recognizing user commands in free speech that are not limited to a particular vocabulary and determining the highest matching user command available for voice activation.
[0074] If modules 316 include a large vocabulary recognition module, the device may also include text analysis capabilities to parse or otherwise analyze the retrieved speech and recognize user commands.
[0075] The storage device 312 may include a code generation module 320 for generating one or more codes that mimic the code that would be generated if a user performed the action required by the same user command.
[0076] The storage device 312 may include a code transmission module 324 for providing code to the underlying operating system drivers, which generate corresponding events and queue them in an event queue so that they can be processed by the electrophysiology system.
[0077] The storage device 312 can include an image capture and analysis module 328 for capturing and analyzing screens of the electrophysiology system or portions thereof. The analysis can include discovering information such as catheter type, spline or number of electrodes without having to interface with the electrophysiology system. Obtaining these parameters can be useful in determining parameters of commands spoken by the user, such as, for example, excluding displaying electrograms of unavailable electrodes or other impossible combinations.
[0078] The storage device 312 may include a user learning module 332 that may include one or more learning modules associated with one or more users. For example, the user learning module 332 may include a voice signature learning module for acquiring a user's voice signature so that future voice recognition will be more efficient and accurate than before the signature was available. Thus, when a user registers with the system, the user can provide a voice sample on which a voice signature is trained, and the respective voice signature is retrieved when the user identifies with the system. In an alternative embodiment, the associated voice signature is retrieved based on the user's voice captured at the start of an operation. The user's signature may be updated based on the newly acquired voice.
[0079] It will be understood that the steps and modules disclosed above may be in addition to software, hardware, firmware, or other modules required to operate the catheter, display the catheter procedure process, perform other calculations such as signal analysis, particularly complex fractionated electrogram (CFE) analysis, generate cardiac maps, etc. Further details on methods and systems may be found, for example, in U.S. Patent Nos. 8,676,305 and 9,629,567, which are incorporated herein by reference in their entirety for any purpose.
[0080] The present invention may be a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions for causing a processor to perform aspects of the present invention.
[0081] A computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridges in grooves with instructions recorded on them, and any suitable combination of the above. A computer-readable storage medium as used herein should not be construed as being a signal that is itself ephemeral, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a wave guide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electric signal transmitted through a flow line.
[0082] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions to a computer-readable storage medium within the respective computing / processing device for storage.
[0083] The computer readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction set architecture instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, such as Java, C, C++, Python, etc. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to individualize the electronic circuitry to carry out aspects of the present invention.
[0084] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.
[0085] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device to generate a machine such that the instructions are executed by the processor of the computer or other programmable data processing device to create means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium that can cause a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer readable storage medium on which the instructions are stored includes an article of manufacture that includes instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.
[0086] The computer-readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to generate a computer-implemented process such that the instructions executing on the computer, other programmable apparatus, or other device perform the functions / acts defined in the flowchart and / or block diagram blocks.
[0087] The flowcharts and block diagrams in the figures illustrate the structure, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing the illustrated logical function. In some alternative implementations, the functions shown in the blocks may occur in an order other than that shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, may be implemented by a dedicated hardware-based system that performs a particular function or operation, or may operate or execute a combination of dedicated hardware and computer instructions. EXAMPLES
[0088] Example 1 1. A method executed by a computing platform, in an electrophysiology system having an audio capture device, comprising: receiving an audio signal captured by the audio capture device; analyzing the audio signal to extract at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to the user command; and sending the at least one code to a driver of the computing platform, thereby causing execution of the user command by the electrophysiology system.
[0089] Example 2 2. The method of claim 1, further comprising analyzing the audio signal to identify a parameter associated with the user command.
[0090] Example 3 The method of example 1, further comprising obtaining a screen capture of a display of the electrophysiology system, analyzing the screen capture to obtain parameters of the electrophysiology system, and using the parameters to identify a user command or the parameters thereof.
[0091] Example 4 2. The method of claim 1, wherein the user command is selected from a predetermined set of user commands.
[0092] Example 5 The method of example 4, wherein the predetermined set of user commands is a subset of a larger set of user commands available in the electrophysiology system.
[0093] Example 6 6. The method of example 5, wherein the predetermined set of user commands includes at most 25% of a larger set of user commands.
[0094] Example 7 6. The method of example 5, wherein the predetermined set of user commands includes at most 10% of a larger set of user commands.
[0095] Example 8 1. A computerized apparatus having a processor coupled to a memory unit, the processor being adapted to perform the steps of receiving an audio signal captured by an audio capture device; analyzing the audio signal to retrieve at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to the user command; and sending the at least one code to a driver of a computing platform, thereby causing execution of the user command by an electrophysiology system.
[0096] Example 9 The apparatus of Example 8, wherein the audio capture device is a head-mounted microphone.
[0097] Example 10 9. The apparatus of example 8, wherein the processor is further adapted to analyze the audio signal to identify a parameter associated with the user command.
[0098] Example 11 The apparatus of Example 8, wherein the processor is further adapted to obtain a screen capture of a display of the electrophysiology system, analyze the screen capture to obtain parameters of the electrophysiology system, and use the parameters to identify a user command or the parameter.
[0099] Example 12 9. The apparatus of example 8, wherein the user command is selected from a predetermined set of user commands.
[0100] Example 13 13. The apparatus of example 12, wherein the predetermined set of user commands is a subset of a larger set of user commands available in the electrophysiology system.
[0101] Example 14 14. The apparatus of example embodiment 13, wherein the predetermined set of user commands comprises up to 25% of a larger set of user commands.
[0102] Example 15 14. The apparatus of example embodiment 13, wherein the predetermined set of user commands comprises at most 10% of a larger set of user commands.
[0103] (Example 16) 1. A computer program product comprising a non-transitory computer-readable storage medium bearing program instructions configured to cause a processor to perform actions, the program instructions performing the steps of: receiving an audio signal captured by an audio capture device; analyzing the audio signal to retrieve at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to the user command; and sending the at least one code to a driver of a computing platform, thereby causing execution of the user command by an electrophysiology system.
[0104] Although the embodiments described herein primarily address cardiac diagnostic applications, the methods and systems described herein may also be used in other medical applications.
[0105] It will be understood that the embodiments described above are given by way of example, and that the present disclosure is not limited to what is particularly shown and described herein above. Rather, the scope of the present disclosure includes both combinations and subcombinations of the various features described herein above, as well as variations and modifications thereof not disclosed in the prior art that would occur to one skilled in the art upon reading the foregoing description.
[0106] [Embodiment] (1) A method comprising: In an electrophysiology system executed by a computing platform and including an audio capture device, receiving an audio signal captured by an audio capture device; analyzing the audio signal to derive at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to said user command; sending the at least one code to a driver of the computing platform; thereby causing execution of the user command by the electrophysiology system; and A method comprising: (2) The method of embodiment 1, further comprising analyzing the audio signal to identify parameters associated with the user command. (3) obtaining a screen capture of a display of the electrophysiology system; and analyzing the screen capture to obtain parameters of the electrophysiology system; utilizing said parameter to identify said user command or a parameter thereof; 2. The method of embodiment 1, further comprising: (4) The method of claim 1, wherein the user command is selected from a predetermined set of user commands. (5) The method of embodiment 4, wherein the predetermined set of user commands is a subset of a larger set of user commands available in the electrophysiology system.
[0107] (6) The method of claim 5, wherein the predetermined set of user commands includes up to 25% of a larger set of user commands. (7) The method of claim 5, wherein the predetermined set of user commands includes at most 10% of a larger set of user commands. (8) A computerized apparatus having a processor coupled to a memory unit, the processor comprising: receiving an audio signal captured by an audio capture device; analyzing the audio signal to derive at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to said user command; sending the at least one code to a driver of the computing platform; thereby causing execution of the user command by the electrophysiology system; A computerized apparatus adapted to perform the steps of: (9) The apparatus of embodiment 8, wherein the audio capture device is a head-mounted microphone. (10) The apparatus of claim 8, wherein the processor is further adapted to analyze the audio signal to identify parameters associated with the user command.
[0108] (11) The processor, obtaining a screen capture of a display of the electrophysiology system; analyzing the screen capture to obtain parameters of the electrophysiology system; utilizing said parameter to identify said user command or a parameter thereof; 9. The apparatus of embodiment 8, further adapted to perform the steps of: (12) The apparatus of claim 8, wherein the user command is selected from a predetermined set of user commands. (13) The apparatus of embodiment 12, wherein the predetermined set of user commands is a subset of a larger set of user commands available in the electrophysiology system. (14) The apparatus of claim 13, wherein the predetermined set of user commands includes up to 25% of a larger set of the user commands. (15) The apparatus of claim 13, wherein the predetermined set of user commands includes up to 10% of a larger set of the user commands.
[0109] (16) A computer program product comprising a non-transitory computer-readable storage medium bearing program instructions configured to cause a processor to perform an action, the program instructions comprising: receiving an audio signal captured by an audio capture device; analyzing the audio signal to derive at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to said user command; sending the at least one code to a driver of the computing platform; thereby causing execution of said user command by the electrophysiology system; and A computer program product that implements the above.
Claims
1. 1. A method comprising: In an electrophysiology system executed by a computing platform and including an audio capture device, receiving an audio signal captured by an audio capture device; analyzing the audio signal to derive at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to said user command; sending the at least one code to a driver of the computing platform; thereby causing execution of the user command by the electrophysiology system; and A method comprising:
2. The method of claim 1 , further comprising analyzing the audio signal to identify a parameter associated with the user command.
3. obtaining a screen capture of a display of the electrophysiology system; analyzing the screen capture to obtain parameters of the electrophysiology system; utilizing said parameter to identify said user command or a parameter thereof; The method of claim 1 further comprising:
4. The method of claim 1 , wherein the user command is selected from a predetermined set of user commands.
5. The method of claim 4 , wherein the predetermined set of user commands is a subset of a larger set of user commands available in the electrophysiology system.
6. The method of claim 5 , wherein the predetermined collection of user commands includes at most 25% of the larger set of user commands.
7. The method of claim 5 , wherein the predetermined collection of user commands includes at most 10% of the larger set of user commands.
8. 1. A computerized apparatus having a processor coupled to a memory unit, the processor comprising: receiving an audio signal captured by an audio capture device; analyzing the audio signal to derive at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to said user command; sending the at least one code to a driver of the computing platform; thereby causing execution of the user command by the electrophysiology system; A computerized apparatus adapted to perform the steps of:
9. The apparatus of claim 8 , wherein the audio capture device is a head-mounted microphone.
10. The apparatus of claim 8 , wherein the processor is further adapted to analyze the audio signal to identify a parameter associated with the user command.
11. The processor, obtaining a screen capture of a display of the electrophysiology system; analyzing the screen capture to obtain parameters of the electrophysiology system; utilizing said parameter to identify said user command or a parameter thereof; The apparatus of claim 8 , further adapted to:
12. The apparatus of claim 8 , wherein the user command is selected from a predetermined set of user commands.
13. The apparatus of claim 12 , wherein the predetermined set of user commands is a subset of a larger set of user commands available in the electrophysiology system.
14. The apparatus of claim 13 , wherein the predetermined collection of user commands includes up to 25% of the larger set of user commands.
15. The apparatus of claim 13 , wherein the predetermined collection of user commands includes at most 10% of the larger set of user commands.
16. 1. A computer program product comprising a non-transitory computer-readable storage medium bearing program instructions configured to cause a processor to perform an action, the program instructions comprising: receiving an audio signal captured by an audio capture device; analyzing the audio signal to derive at least one trigger phrase; in response to identifying the trigger phrase, further analyzing the audio signal to identify one or more words associated with a user command; generating at least one code corresponding to said user command; sending the at least one code to a driver of the computing platform; thereby causing execution of said user command by the electrophysiology system; and A computer program product that implements the above.
