System and method for a user interface of a medical imaging system

Abstract

The present disclosure provides methods and systems for navigating between a display panel and graphical elements of a user interface of a medical imaging system via controls of a handheld imaging device. In one embodiment, the present disclosure provides a method comprising: in response to an operator of the medical imaging system adjusting one or more controls arranged on a control handle of a handheld ultrasound device of the medical imaging system, adjusting a focus of a user interface (UI) of the medical imaging system between a plurality of graphical control elements displayed in the UI; and in response to the operator selecting, via the one or more controls, a graphical control element of the plurality of graphical control elements at a location of the focus of the UI, performing an action of the medical imaging system associated with the selected graphical control element.

Description

Technical Field

[0001] The embodiments of the subject matter disclosed herein relate to medical imaging, and more specifically to systems and methods for interacting with a user interface of a medical imaging system. Background Technology

[0002] Medical imaging can be used to visualize anatomical features, such as soft tissue features or blood flow, inside a patient's body during an examination. In some types of medical imaging, such as endoscopic ultrasound, the operator of the medical imaging system guides a probe, attached to the distal end of the insertion cannula of a handheld ultrasound device, into the patient's cavity via the handle of the handheld ultrasound device. Signals generated by the transducer of the probe are emitted toward the anatomical feature, and information about the anatomical feature is detected from the signals reflected back from it, thereby producing one or more images of that anatomical feature.

[0003] The insertion cannula can be flexible enough to pass through the body cavity into the desired location, and the probe can be coupled to the insertion cannula via a flexible neck assembly, which can be bent or folded to allow the probe to pass through bends in the body cavity. For example, in transesophageal echocardiography (TEE), the insertion cannula is positioned appropriately along the bends of the patient's esophagus for diagnosing cardiac disease. To advance and manipulate the probe, the handheld ultrasound device can include one or more controls arranged on a handle. In one example, the controls include one or more control wheels that can be rotated by the operator to bend or fold the flexible neck assembly in a left, right, rear, or front direction. By advancing or retracting the probe via the handle and adjusting the one or more control wheels, the probe's transducer can be pointed at and tilted to optimize the acquisition of ultrasound images of the ROI, which can be displayed on the ultrasound system's display device within the user interface (UI). Additionally, one or more buttons arranged on the handle of the handheld ultrasound device allow the operator to interact with the UI. For example, one or more buttons may allow the operator to capture the current image and store it in the image storage area, or switch between different displays of the UI, or adjust one or more settings and / or parameters of the ultrasound system. Summary of the Invention

[0004] In one embodiment, this disclosure provides a method comprising: adjusting the focus of the UI of the medical imaging system among a plurality of graphical control elements displayed in a user interface (UI) in response to an operator of the medical imaging system adjusting one or more controls disposed on a control handle of a handheld ultrasound device of the medical imaging system; and performing an action of the medical imaging system associated with the selected graphical control element in response to the operator selecting a graphical control element at the focus position of the plurality of graphical control elements via the one or more controls.

[0005] The above-described advantages, as well as other advantages and features, of this specification will become apparent, either alone or in connection with the accompanying drawings, from the following detailed description. It should be understood that the above summary is provided to present a simplified version of the selected concepts further described in the detailed description. This is not intended to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims following the detailed description. Furthermore, the claimed subject matter is not limited to embodiments that address any of the disadvantages mentioned above or in any part of this disclosure. Attached Figure Description

[0006] This patent or patent application document contains at least one color-drawn drawing. A published copy of this patent or patent application with the color drawing will be provided by the Patent Office upon request and payment of the necessary fees.

[0007] A better understanding of the various aspects of this disclosure can be achieved by reading the following detailed description and referring to the accompanying drawings, in which:

[0008] Figure 1 A first block diagram of an exemplary implementation of an ultrasound system is shown.

[0009] Figure 2A An exemplary ultrasound probe is shown.

[0010] Figure 2B An exploded view of a portion of the ultrasound probe is shown.

[0011] Figure 3A A first exemplary configuration of a set of controls on the handle of an ultrasound probe is shown.

[0012] Figure 3B The orientation of the first set of controls on the handle of the ultrasound probe is shown when held by the operator of the ultrasound system.

[0013] Figure 3C A second exemplary configuration of a set of controls on the handle of an ultrasound probe is shown.

[0014] Figure 3DThe orientation of the second set of controls on the handle of the ultrasound probe is shown when held by the operator of the ultrasound system.

[0015] Figure 4A An exemplary user interface of an ultrasound system in a first configuration is shown.

[0016] Figure 4B An exemplary user interface of an ultrasound system in a second configuration is shown.

[0017] Figure 4C An exemplary user interface of an ultrasound system in a third configuration is shown.

[0018] Figure 5A An exemplary user interface element is shown that indicates the position of the control wheel of a handheld ultrasound device.

[0019] Figure 5B The first exemplary user interface element is shown, indicating the position of the control wheel of a handheld ultrasound device.

[0020] Figure 5C A second exemplary user interface element is shown, indicating the position of the control wheel of a handheld ultrasound device.

[0021] Figure 5D A third exemplary user interface element is shown, indicating the position of the control wheel of a handheld ultrasound device.

[0022] Figure 6 A flowchart illustrating an exemplary method for displaying and interacting with a user interface of an ultrasound system via controls of a first configuration is shown.

[0023] Figure 7 A flowchart illustrating an exemplary method for displaying and interacting with the user interface of an ultrasound system via controls of a second configuration is shown.

[0024] Figure 8 A flowchart illustrating an exemplary method for indicating the state of the control wheel of an ultrasound system on a user interface of the ultrasound system is shown. Detailed Implementation

[0025] The following description relates to methods and systems for interacting with a medical imaging system, such as an ultrasound imaging system, via one or more operator controls. The description herein is provided with respect to an ultrasound imaging system, but it should be understood that these methods and systems can be adapted to any handheld imaging mode, such as photoacoustic imaging, visible light imaging, optical coherence tomography, etc., without departing from the scope of this disclosure.

[0026] Medical ultrasound imaging typically involves the placement of an ultrasound probe within an ultrasound imaging system, including positioning one or more transducer elements onto or within the imaging subject (such as a patient) at the location of a target anatomical feature (e.g., heart, lungs, etc.). Images are acquired by the ultrasound probe and displayed on a display device in real-time or near real-time (e.g., images are displayed as soon as they are generated without intentional delay). The operator of the ultrasound probe can view the images and adjust the probe's position and / or various acquisition parameters (e.g., emission frequency, emission depth, time gain compensation, beam steering angle, beamforming strategy, etc.) to obtain high-quality images of the target anatomical feature. However, acquiring optimal images (e.g., images of desired quality) can be challenging, thus manipulating the probe may require careful attention to the image on the display device.

[0027] In some types of ultrasound imaging, a probe is inserted into a patient's body cavity, such as in transesophageal echocardiography (TEE), where the patient's heart is imaged via a probe placed in the patient's esophagus. In TEE ultrasound examinations, a handheld ultrasound device is used, comprising a probe coupled to a TEE control handle via a rigid, flexible insertion tube, wherein the insertion tube is inserted into the esophagus using the TEE control handle. One or more controls may be arranged on the TEE control handle, wherein by manipulating these controls, the operator can adjust the orientation and / or position of the probe relative to the insertion tube. In one example, the operator adjusts the rotation of a control wheel to bend or fold a hinged neck assembly that couples the probe to the insertion tube to manipulate the probe in a specific orientation (e.g., left or right, front or back). For example, to advance the probe along the downward and leftward bend of the esophagus, the operator can rotate a first control wheel in a first direction to bend the probe to the left (e.g., the patient's left side) before advancing the probe, and the operator can rotate a second control wheel in a second direction to bend the probe to the back (e.g., towards the patient's back). By adjusting the position and / or orientation of the probe relative to the insertion tube before advancing the probe into the esophagus, the amount of friction between the probe and the soft tissue of the esophagus can be reduced, thereby reducing any damage to the soft tissue and facilitating probe placement. Furthermore, the tilt of the probe lens can be adjusted to facilitate the acquisition of ultrasound images, for example, to ensure that the ultrasound beam of the ultrasound imaging system is perpendicular to the surface of the target object. Additionally, one or more buttons can be arranged on the probe handle, allowing the operator to select one or more functions of the ultrasound imaging system (e.g., shortcut buttons mapped to one or more functions available via the menu of the ultrasound imaging system).

[0028] However, one problem with the controls on the probe handle is that the mapping of one or more buttons on the probe handle to one or more functions of the medical imaging system may not be indicated in the UI. This means the operator may have to remember or recall the mapping. If the operator forgets which button has been selected, their attention may leave the UI and shift to the button to confirm its selection, potentially distracting them from the task being performed during the examination and reducing their efficiency. Furthermore, in some examples, the mapping of one or more buttons on the probe handle to one or more functions of the medical imaging system can be customized between the operator and the imaging system, which may make remembering the mapping even more difficult. Additionally, the buttons may be located on a side of the handle that is difficult to see during ultrasound examinations, requiring the operator to rotate or move the handle to view them, resulting in undesirable changes in the transducer / probe position. However, another problem is that while replacing the mechanical buttons on a handheld ultrasound device with capacitive sensor buttons, resistive touch or force-sensing resistor buttons, or shorting pad buttons can reduce the cost of the probe handle and reduce wall thickness, operators may resist the switch to capacitive sensor buttons due to the loss of tactile feedback when selecting mechanical buttons or spring domes that provide tactile feedback in combination with the button technologies listed above (providing tactile feedback can help operators determine when a button is selected when no visual feedback is provided in the UI).

[0029] Additionally, when the operator rotates the control wheel to adjust the probe position by bending or tilting the flexible neck assembly, the UI may not display an indication of the degree of bending or tilting of the probe's flexible neck assembly. To determine the degree of bending of the flexible neck assembly, the operator relies on sensing the configuration of the wheel with their fingers and / or thumb, which can lead to confusion. Park et al. proposed a partial solution to this problem in U.S. Patent No. 20180085090, in which the probe tilt degree is indicated in text format on the ultrasound system's display device, approximating the ultrasound image, or via audio indication. However, reading text indications may require deliberate attention from the operator, which could reduce the efficiency of the operator's probe manipulation and / or increase the amount of time required to perform the examination, while audio indications may involve deliberate interaction with the user interface or may not be available at the expected time, thus reducing operator efficiency.

[0030] In one embodiment, this disclosure addresses the aforementioned problems at least in part through a method comprising: adjusting the focus of the ultrasound imaging system's UI among a plurality of graphical control elements displayed in the UI in response to an operator adjusting one or more controls arranged on a probe handle. For the purposes of this disclosure, the focus is a predefined bounded area within the UI (e.g., a button or a graphical component of the UI) that can receive input from the operator (e.g., via mouse click or via the one or more controls on the probe handle). The operator can move the focus among the multiple graphical components (e.g., via one or more controls). For example, if the UI comprises a horizontal row of five buttons, the operator can move the focus from a first button to a second button located to the right of the first button by selecting a horizontal arrow key on the keyboard, or move the focus from the second button to a third button located to the right of the second button by selecting a horizontal arrow key, and so on. When the focus is on a button in the horizontal row of five buttons, the operator can select the button on which the focus is set by selecting different controls.

[0031] Therefore, in response to the operator selecting a graphical control element from among the plurality of graphical control elements at the focal point of the UI via one or more controls, the action of the ultrasound imaging system associated with the selected graphical control element is executed. Thus, by allowing the operator to navigate among the plurality of shortcut control elements via one or more controls and select the shortcut control element associated with the desired function, rather than mapping one or more buttons on the control handle to one or more functions, the number of shortcut control elements for quick access functions of the medical imaging system can be increased and / or it can be made easier to customize, thereby improving the efficiency of the operator performing ultrasound examinations. Furthermore, by highlighting the shortcut control element in focus, visual cues can be provided to the operator to help the operator select the shortcut control element with the desired function, and / or visual indications of the current operating mode of the medical imaging system. Due to the visual cues and / or visual indications in the UI, the operator's cognitive load during the examination can be reduced (e.g., due to having to remember the function mapping of one or more buttons on the control handle to one or more functions), and the operator's attentional distraction caused by having to check the one or more controls on the control handle can be reduced, thereby further improving operator efficiency. Another advantage of this method is that the mechanical buttons on the control handle can be replaced with capacitive sensor buttons, thereby reducing the cost of medical imaging systems and / or ultrasound probes, by facilitating more effective interaction with the UI.

[0032] Furthermore, when the operator adjusts the position and / or orientation of the probe during an inspection, the UI display elements can be adjusted to indicate the adjusted position and / or orientation of the ultrasound probe. For example, when the operator manipulates the first control wheel arranged on the handle to move the ultrasound probe in the first free plane, and manipulates the second control wheel to move the ultrasound probe in the second free plane, visual indications of the deflection angles of the ultrasound probe in the first and second free planes can be displayed on the graphical representations of the first and second control wheels in the UI, respectively. By providing visual indications of the deflection angles of the ultrasound probe in the first and second free planes, rather than relying on the operator to sense the configuration of the first and second control wheels with their thumb and fingers, the operator's cognitive load can be reduced, and operator efficiency can be improved.

[0033] As the operator manipulates the probe via the TEE control handle, ultrasound images acquired through the probe's lens are displayed on the ultrasound imaging system's display. These images may be displayed within the ultrasound imaging system's user interface (UI), which may include additional display elements such as indications of the ultrasound system's mode (e.g., B-mode, Doppler, etc.), the duration of the examination, and / or other parameters of the ultrasound imaging system, such as emission frequency, emission depth, time gain compensation, beam steering angle, etc. In some examples, a text description of the probe's orientation and / or position relative to the insertion tube may also be displayed in the UI. For example, on a continuously updated basis, or in response to a tilt angle exceeding one or more threshold angles, the probe's tilt angle in one or more free planes may be displayed in a panel of the UI. However, text descriptions may be difficult to view during the examination or may not be displayed at the time the operator expects. If the text description (or audio indication) is not available to the operator in the desired manner or at the desired time, the operator may use their fingers and / or thumb to sense the rotational position of the first and second control wheels to determine the probe's orientation and / or position, which can increase the operator's cognitive load and reduce operator efficiency during the examination.

[0034] The UI may also include an image storage area where the operator can store one or more selected images (e.g., frames) during the examination. For example, a probe may be placed at a first location in the esophagus to acquire a first desired image of the patient's heart. When the first desired image is acquired, the operator can select and save it for later reference (e.g., for diagnosis, for presentation to the patient, for comparison purposes, etc.). When the operator saves the first desired image, the image may appear in the image storage area of ​​the UI (e.g., a display element in the UI showing the stored image). Subsequently, the probe may be placed at a second location in the esophagus to acquire a second desired image of the patient's heart. When the second desired image is acquired, the operator can select and save it for later reference. When the operator saves the second desired image, the image may appear (e.g., together with the first desired image) in the image storage area of ​​the UI. In this way, the operator can collect desired images during the examination. If the operator wishes to view images from the image storage area, the operator can select images in the UI, whereby the selected images can be displayed.

[0035] In some examples, the additional display elements may be control elements, allowing the operator to interact with and / or change one or more settings of the ultrasound imaging system. These settings may include display settings, such as whether elements are shown or hidden on the user interface screen. These settings may include parameters of the ultrasound system (such as the type of ultrasound examination, e.g., B-mode, Doppler, etc.) or probe configuration parameters of the ultrasound imaging system. The operator may interact with, adjust, or select control elements in the UI via a user input device (such as a mouse, touchpad, etc.), or the operator may interact with the UI via a touchscreen, whereby the operator touches the UI's display screen to interact with the UI. Furthermore, in some examples, the TEE control handle may include one or more controls that allow the operator to interact with the UI. For example, the TEE control handle may include a touchpad and / or one or more buttons. By manipulating the touchpad and / or one or more buttons, the operator can change the display of the UI or the settings of the ultrasound imaging system. For example, the operator can select an image from the image storage area in the UI via a button on the TEE control handle. See below for reference. Figure 2A and Figure 2B as well as Figures 3A to 3D A more detailed example of the TEE handle control is described.

[0036] The one or more buttons can be mechanical buttons with two states: a first unselected state, in which the mechanical button is not pressed; and a second selected state, in which the mechanical button is pressed. Therefore, the operator can (e.g., with a finger or thumb) sense when the mechanical button is selected or unselected based on tactile feedback. In another example, the button is a mechanical button without these two states, in which case the operator cannot (e.g., with a finger or thumb) sense when the mechanical button is selected or unselected based on tactile feedback. However, in some examples, a visual indication that the mechanical button has been selected can be provided (e.g., the button or its outline can be illuminated, etc.), thereby allowing the operator to determine which of the one or more buttons has been selected. Additionally, when the operator selects a mechanical button, the operator can receive tactile feedback indicating that the mechanical button has been pressed, which can indicate to the operator that the mechanical button has been selected.

[0037] In yet another example, the one or more buttons are not mechanical buttons, but capacitive sensor buttons, where pressing the button is recorded electronically by a sensor pad positioned beneath the button's surface. For example, the button may have a surface made of plastic or a similar material arranged above a sensor pad embedded in a softer material (e.g., foam). When the sensor pad is pressed, conductive material is pressed against a printed circuit board (PCB) positioned beneath the sensor pad, thus completing the circuitry that sends a signal that the button has been pressed (e.g., to the ultrasound system). By using capacitive sensor buttons instead of mechanical buttons, fewer moving parts can be used, reducing the cost of the ultrasound imaging system while improving its reliability, cleanliness, and lifespan. Additionally, capacitive sensor buttons offer increased flexibility in button shape and graphical representation.

[0038] However, one problem with using capacitive sensor buttons or similar technologies (such as resistive touch, force-sensing resistor buttons, and shorting pad buttons) is that it can cause operator confusion regarding button states because no tactile feedback is provided to indicate that a capacitive sensor button has been selected, and the operator may not be able to determine which of the one or more buttons has been selected via tactile sensing. Furthermore, the operator may have to remember the mapping of each button to its corresponding function in the ultrasound system.

[0039] For example, the TEE control handle can have three buttons corresponding to three different operating modes, which can be predefined or configured by the user. For instance, the first button could correspond to operation in 2D mode, the second button to operation in 4D mode, and the third button to operation in color mode. The operator can select the first button corresponding to operation in 2D mode, and one or more elements displayed in the UI can be updated based on that selection. At a later point, the operator might want to switch to 4D mode. However, the operator might forget whether the second or third button corresponds to 4D mode, or the operator might forget whether the current 2D mode was activated by pressing the first, second, or third button. Because capacitive sensor buttons do not provide tactile indication of button selection, tactile sensing of the buttons may not help the operator remember which button was pressed. Additionally, the operator might not want to rotate the TEE control handle to check the buttons to determine whether the first, second, or third button is illuminated, as rotating the TEE control handle might change the probe's position (e.g., from a desired position to an undesirable position). Because the operator doesn't remember which button to select to switch to 4D mode, their attention may be diverted from the inspection, and / or they may have to determine which button corresponds to 4D mode through trial and error, thus wasting time and increasing button wear.

[0040] In one example, a medical imaging system addresses the problem of not being able to easily and quickly view the operating modes of an ultrasound imaging system, the rotational positions of a first control wheel and / or a second control wheel, and / or one or more shortcut key options for accessing one or more functions of the ultrasound imaging system. The medical imaging system includes: an ultrasound probe coupled to a TEE control handle via an insertion tube having an articulated neck mechanism; a display device; and a processor communicatively coupled to the TEE control handle and the display device, and including instructions stored in a non-transitory memory that, when executed, cause the processor to: adjust a first set of controls arranged on the TEE control handle in response to the operator of the medical imaging system. The device includes a bendable, hinged neck mechanism for adjusting the position and / or orientation of the ultrasound probe; adjusting the graphical display elements of the medical imaging system's user interface (UI) to indicate the adjusted position and / or orientation of the ultrasound probe; adjusting the focus of the UI among multiple display panels in response to the operator adjusting a second set of controls arranged on the TEE control handle in a first manner; adjusting the focus of the UI among multiple control elements in response to the operator adjusting the second set of controls in a second manner; indicating the position of the adjusted focus of the UI within the UI; and executing the functions of the medical imaging system associated with the control element in response to the operator selecting a control element at the adjusted focus position via the second set of controls. In this way, the operator can maintain attention on the ultrasound image and / or other elements of the UI while receiving visual feedback regarding the selection of available functions, parameters, settings, display options, etc.

[0041] The following text is for reference only. Figure 1 An exemplary ultrasound imaging system is described. The ultrasound imaging system may include a handheld ultrasound device, such as... Figure 2A A handheld ultrasound device 200. The handheld ultrasound device may include one or more controls on the handle of the handheld ultrasound device, the one or more controls including one or more capacitive sensor buttons, such as... Figure 2B As shown. The one or more controls can be constructed using the first exemplary configuration, such as... Figure 3A As shown, one or more controls can be manipulated by the operator's hand, such as... Figure 3B As shown. Figure 3C The image shows a second exemplary configuration of the one or more controls, wherein the one or more controls can be manipulated by an operator's hand, such as... Figure 3D As shown. This one or more controls can be used to select the mode of the ultrasound imaging system, where mode options for selection and the selected mode can be displayed, such as... Figure 4AAn exemplary UI is shown. The one or more controls can be used to save one or more images from an ultrasound image (e.g., for later reference) and to switch between UI display elements (such as an image storage area, where images in the image storage area can be selected via the one or more controls) and mode displays (where one or more modes can be selected via the one or more controls), such as... Figure 4B An exemplary UI is shown. This one or more controls can be used to switch between each of the image storage area, the modal display, and a third display element with optional items, such as... Figure 4C As shown. The UI may also include a graphical representation of the position of one or more control wheels of the handheld ultrasound device 200, such as... Figure 5A As shown, this graphical representation relates to the configuration of the one or more controls indicated in Figure 5B. The graphical representation of the position of the one or more control wheels can be as follows: Figure 5C and Figure 5D The alternative configuration is constructed as shown. It can be based on... Figure 6 The procedure described in method 600, via Figure 3A The first exemplary configuration of one or more controls is used to adjust the operating mode, parameters, and settings of the ultrasound imaging system. This can be done according to... Figure 7 The procedure described in method 700, via Figure 3C The second exemplary configuration of the one or more controls is used to adjust the operating mode, parameters, and settings of the ultrasound imaging system. Figure 5A A graphical representation of the positions of the one or more control wheels of the handheld ultrasound device 200 shown can be derived from... Figure 8 The procedure described in method 800 is used to update.

[0042] See now Figure 1A schematic diagram of an ultrasound imaging system 100 according to an embodiment of the present disclosure is shown. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102 that drives elements (e.g., transducer elements) 104 within a transducer array (referred to herein as probe 106) to transmit pulsed ultrasound signals (referred herein as transmit pulses) into a patient's body (not shown). Probe 106 may be a one-dimensional transducer array probe with a one-dimensional (1D) transducer array structure, or it may be a two-dimensional matrix transducer array probe with a two-dimensional (2D) transducer array structure. Transducer element 104 may be made of a piezoelectric material. When a voltage is applied to a piezoelectric crystal, the crystal physically expands and contracts, thereby emitting ultrasonic sphere waves. In this way, transducer element 104 can convert an electronic emission signal into an acoustic emission beam. In another example, the ultrasound transducer is a capacitive micromachined ultrasound transducer (CMUT). In some examples, probe 106 is a portable, handheld probe that can be manipulated by an operator on a patient's surface (e.g., on the skin). In other examples, probe 106 is coupled to the distal end of a tube of a handheld ultrasound device, wherein the tube can be guided into the patient's body cavity (e.g., during endoscopic surgery) via a handle positioned at the end of the tube opposite to probe 106, as described below. Figure 2A A more detailed description.

[0043] After the element 104 of probe 106 transmits a pulsed ultrasound signal into the patient's body, the pulsed ultrasound signal is backscattered from internal structures (such as blood cells or muscle tissue) to generate an echo returning to element 104. The echo is converted into an electrical signal or ultrasound data by element 104, and the electrical signal is received by receiver 108. The electrical signal representing the received echo passes through receiver beamformer 110, which outputs ultrasound data. Additionally, transducer element 104 can generate one or more ultrasound pulses based on the received echo to form one or more transmit beams.

[0044] According to some embodiments, probe 106 may include all or part of circuitry configured to perform transmit beamforming and / or receive beamforming. For example, all or part of transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be located within probe 106 or within another component of a handheld ultrasound device or medical imaging system. In this disclosure, the terms “scanning” or “under scanning” may also be used to refer to the acquisition of data through the process of transmitting and receiving ultrasound signals.

[0045] User interface 115 can be used to control the operation of ultrasound imaging system 100. User interface 115 may include one or more of the following: a rotary element, a mouse, a keyboard, a trackball, a touchscreen, hard keys linked to specific actions, soft keys configurable to control different functions, and / or a graphical user interface displayed on display device 118. Display device 118 can display ultrasound images and various information generated by ultrasound imaging system 100. In some examples, user interface 115 may be integrated into display device 118, for example, as a touchscreen.

[0046] The ultrasound imaging system 100 includes a processor 116 for controlling a transmitting beamformer 101, a transmitter 102, a receiver 108, and a receiving beamformer 110. The processor 116 is in electronic communication (e.g., communicatively connected) with a probe 106. For the purposes of this disclosure, the term "electronic communication" may be defined to include both wired and wireless communication. The processor 116 can control the operation of the ultrasound imaging system 100 and the signal flow between the internal components of the ultrasound imaging system 100. For example, the processor 116 can control the operation of the ultrasound imaging system 100 by receiving control signals from a user input device 116.

[0047] Processor 116 can control probe 106 to acquire data according to instructions stored on memory 120. As discussed herein, memory includes any non-transitory computer-readable medium in which programming instructions are stored. For the purposes of this disclosure, the term tangible computer-readable medium is explicitly defined as including any type of computer-readable storage device. Exemplary methods and systems can be implemented using coded instructions (e.g., computer-readable instructions) stored on a non-transitory computer-readable medium such as flash memory, read-only memory (ROM), random access memory (RAM), cache, or any other storage medium in which information is stored for any duration (e.g., a prolonged period of time, a permanent, brief instance, for temporary buffering, and / or for caching information). Computer memory as referred to herein may include volatile and non-volatile or removable and non-removable media for storing information in electronic format (such as computer-readable program instructions or modules for storing computer-readable program instructions, data, etc., which may be standalone or as part of a computing device). Examples of computer memory may include any other medium that can be used to store information in a desired electronic format and is accessible by at least a portion of one or more processors or a computing device.

[0048] Processor 116 controls which of the elements 104 are active and the shape of the beam emitted from probe 106. Processor 116 also communicates electronically with display device 118, and processor 116 can process data (e.g., ultrasound data) into images for display on display device 118. Processor 116 may include a central processing unit (CPU) according to one embodiment. According to other embodiments, processor 116 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphics board. According to other embodiments, processor 116 may include multiple electronic components capable of performing processing functions. For example, processor 116 may include two or more electronic components selected from a list of electronic components, including: a CPU, a digital signal processor, a FPGA, and a graphics board. According to another embodiment, processor 116 may also include a composite demodulator (not shown) that demodulates RF data and generates raw data. In another embodiment, demodulation may be performed earlier in the processing chain.

[0049] In one example, probe 106 includes a 2D transducer array, and processor 116 calculates a digital beamforming time delay value relative to one or more subarrays included in the 2D transducer array. For example, processor 116 may calculate an analog beamforming time delay value for each transducer in any of the transducers included in one or more subarrays. In one example, transmitter 102 is an analog transmitter, receiver is a digital receiver, and processor 116 controls analog transmitter 102 and digital receiver 108 to form a transmit signal to be applied to each transducer based on the analog and digital beamforming time delay values. Processor 116 may control analog transmitter 102 to add the signal received from the transducer to each of the one or more subarrays based on the analog beamforming time delay value. Additionally, processor 116 may perform analog-to-digital conversion on the signal added to each subarray. Processor 116 can control digital receiver 108 to generate ultrasound data by adding digitized signals based on the time delay value of digital beamforming, and / or processor 116 can control analog transmitter 102 to classify transducers to be included in the subarray, apply the time delay value for performing analog beamforming, and add a signal for each subarray. Processor 116 can also control analog transmitter 102 to again add the synthesized signal generated by adding a signal for each subarray by applying the time delay value for performing analog beamforming.

[0050] Processor 116 may be adapted to perform one or more processing operations based on multiple selectable ultrasound modalities on the data. In one example, data may be processed in real time during a scanning session as echo signals are received by receiver 108 and transmitted to processor 116. For the purposes of this disclosure, the term "real time" is defined as including procedures executed without any intentional delay. For example, embodiments may acquire images at a real-time rate of 7 frames per second to 20 frames per second. Some embodiments of the invention may include multiple processors (not shown) to handle the processing tasks handled by processor 116 according to the exemplary embodiments described above. For example, a first processor may be used to demodulate and extract RF signals before displaying images, while a second processor may be used to further process the data (e.g., by augmenting the data as further described herein). It should be understood that other embodiments may use different processor arrangements.

[0051] The ultrasound imaging system 100 can continuously acquire data at frame rates, for example, from 10 Hz to 30 Hz (e.g., 10 frames / second to 30 frames / second). Images generated from the data can be refreshed on a display device 118 at a similar frame rate. Other embodiments are capable of acquiring and displaying data at different rates. For example, depending on the frame size and the intended application, some embodiments may acquire data at frame rates less than 10 Hz or greater than 30 Hz. A memory 120 is included for storing frames of processed acquired data. In an exemplary embodiment, the memory 120 has sufficient capacity to store at least several seconds of ultrasound data frames. The data frames are stored in a manner that facilitates retrieval based on their acquisition order or time. The memory 120 may include any known data storage medium.

[0052] In various embodiments of the invention, processor 116 can process data through different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules can generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, and combinations thereof, etc. As an example, one or more modules can process color Doppler data, which may include conventional color flow Doppler, power Doppler, HD flow, etc. Image lines and / or frames are stored in memory and may include timing information indicating the time when image lines and / or frames are stored in memory. These modules may include, for example, a scan conversion module for performing scan conversion operations to convert the acquired images from beam space coordinates to display space coordinates. A video processor module may be provided that reads the acquired images from memory and displays the images in real time while performing procedures on a patient (e.g., ultrasound imaging). The video processor module may include a separate image memory, and ultrasound images may be written to the image memory for reading and display by the display device 118.

[0053] In various embodiments of this disclosure, one or more components of the ultrasound imaging system 100 may be included in a portable handheld ultrasound imaging device. For example, a display device 118 and a user interface 115 may be integrated into the external surface of the handheld ultrasound imaging device, which may also include a processor 116 and a memory 120. A probe 106 may include a handheld and / or endoscopic probe that electronically communicates with the handheld ultrasound imaging device to collect raw ultrasound data. Transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be included in the same or different parts of the ultrasound imaging system 100. For example, transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be included in the handheld ultrasound imaging device, the probe, and combinations thereof.

[0054] After performing a two-dimensional ultrasound scan, a data block containing scan lines and their samples is generated. Following the application of a back-end filter, a process called scan transformation is performed to convert the two-dimensional data block into a displayable bitmap image with additional scan information, such as depth, the angle of each scan line, etc. During scan transformation, interpolation techniques are applied to fill in missing holes (i.e., pixels) in the resulting image. These missing pixels occur because each element of the two-dimensional block should typically cover many pixels in the resulting image. For example, in current medical imaging systems, bicubic interpolation is used, which utilizes the adjacent elements of the two-dimensional block. Therefore, if the two-dimensional block is relatively small compared to the size of the bitmap image, the scan-transformed image will include areas of poor or low resolution, especially for deeper regions.

[0055] It should be understood that Figure 1 The ultrasound imaging system 100 shown is for illustrative purposes only and not for limitation. Another ultrasound imaging system may include more, fewer, or different components.

[0056] Now for reference Figure 2A An exemplary handheld ultrasound device 200 is shown, wherein the handheld ultrasound device 200 is electrically connected to an ultrasound system 240 via a cable 236. In one example, the handheld ultrasound device 200 is... Figure 1 A non-limiting example of the user interface 115 of the ultrasound system. In the depicted example, the handheld ultrasound device 200 is a TEE ultrasound device, which includes a control handle 220 flexibly connected to a probe 230 via an insertion tube 234 on the end of the control handle 220 opposite to the cable 236. The probe 230 may be... Figure 1 A non-limiting example of probe 106. Cable 236 may be coupled to control handle 220 via strain relief member 226, which protects the electrical connection between cable 236 and control handle 220 from mechanical forces applied to the insertion tube cable 236. Probe 230 may be configured to emit ultrasonic signals toward an object and receive ultrasonic echo signals reflected from the object to obtain one or more images of the internal region of the object. Probe 230 may include one or more ultrasonic transducer arrays (... Figure 2A (Not shown in the image).

[0057] The insertion tube 234 may be made of an elastic material, possessing sufficient flexibility to follow a curved path through the esophagus and sufficient rigidity to be advanced by the operator via the control handle 220. In one example, the insertion tube 234 is 100 cm to 110 cm long and 10 mm to 20 mm in diameter. The insertion tube 234 may house a cable for transmitting and receiving signals from the ultrasound transceiver of the ultrasound system 240 to the probe 230 and back from the probe 230 to the ultrasound transceiver. The insertion tube 234 may be coupled to the control handle 220 via a strain relief member 225, which protects the electrical connection between the cable and the control handle 220 from mechanical forces applied to the insertion tube 234.

[0058] The probe 230 can be coupled to the insertion tube 234 via a neck assembly 232, which can be a flexible articulated mechanism disposed between the probe 230 and the insertion tube 234. The neck assembly 232 can be made of metal or another material having similar thermal conductivity and specific strength, and can have an outer surface coated with an elastic material similar to the elastic material of the insertion tube 234. In one example, the flexible articulated mechanism consists of multiple hollow segments connected by one or more wires leading to a control handle 220, allowing the neck assembly 232 to be bent in one or more directions by manipulating the one or more wires (e.g., via controls on the control handle 220). By bending the neck assembly 232 in different configurations, the probe 230 can pass through the esophagus along the curved channel to be positioned at a desired location. In other examples, wires may not be used, and the neck assembly 232 can be bent in different ways.

[0059] The bending of the neck assembly 232 can be controlled by one or more controls arranged on the control handle 220, such as a first control wheel 202 and a second control wheel 204. In one example, the first control wheel 202 is vertically positioned on top of the second control wheel 204 on the top side 205 of the control handle 220, and is arranged coaxially along the vertical axis 203. The first control wheel 202 and the second control wheel 204 can be rotated by the operator via the operator's fingers and / or thumb, wherein the first control wheel 202 controls the bending or deflection of the neck assembly 232 in a first free plane (e.g., in a left or right direction), and the second control wheel controls the bending or deflection of the neck assembly 232 in a second free plane (e.g., in a rear or front direction). In one example, the neck assembly 232 can bend at angles up to 180° in any of the left, right, rear, and / or front directions (e.g., 360° of freedom in each free plane).

[0060] For example, the operator can rotate the first control wheel 202 clockwise to control the neck assembly 232 to deflect toward the patient's right side, or the operator can rotate the first control wheel 202 counterclockwise to control the neck assembly 232 to deflect toward the patient's left side (e.g., move in the first free plane). The operator can rotate the second control wheel 204 clockwise to control the distal end of the probe to deflect toward the patient's front side, or the operator can rotate the second control wheel 204 counterclockwise to control the neck assembly 232 to deflect toward the patient's back side (e.g., move in the second free plane). Therefore, by adjusting the rotational positions of the first control wheel 202 and the second control wheel 204, and by moving the probe 230 forward or backward via the control handle 220 (e.g., move in the third free plane), the operator can manipulate the probe 230 in three degrees of freedom (e.g., left-right, front-back, and up-down). Furthermore, in some examples, the lens 228 of the probe 230 can be rotated by rotating the control handle 220 around the central axis 238, thereby allowing the operator to manipulate the probe 230 with six degrees of freedom (e.g., adding pitch, yaw, and roll).

[0061] The control handle 220 may also include a control wheel lock 206 that can lock the positions of the first control wheel 202 and the second control wheel 204. For example, the control wheel lock 206 can be adjusted from a neutral position to a first position to lock the first control wheel 202 and the second control wheel 204; or the control wheel lock 206 can be adjusted from a neutral position to a first position to lock the first control wheel 202 without locking the second control wheel 204; and can be adjusted to a second position to lock the second control wheel 204 without locking the first control wheel 202; and can be adjusted to a third position to lock both the first control wheel 202 and the second control wheel 204. Both the first control wheel 202 and the second control wheel 204 can be unlocked by adjusting the control wheel lock 206 back to the neutral position.

[0062] The control handle 220 may include one or more control buttons, such as a first button 208, a second button 210, and a third button 212. In one example, the first button 208, the second button 210, and the third button 212 are arranged on a first side 207 of the control handle 220, where the first side 207 is easily accessible to the thumb of a right-handed operator when manipulating the probe 230 via the control handle 220. In another example, the first button 208, the second button 210, and the third button 212 may be arranged on a second side of the control handle 220, opposite the first side of the control handle 220, where the second side is easily accessible to the thumb of a left-handed operator when manipulating the probe 230 via the control handle 220. In still other examples, the first button 208, the second button 210, and the third button 212 may be arranged on different sides of the control handle 220 (e.g., not the first side or the second side).

[0063] In one example, the first button 208, the second button 210, and the third button 212 are arranged below and / or near the control wheels 202 and 204, thereby allowing the operator's thumb to easily move between the control wheels 202 and 204 and the buttons. In other examples, the first button 208, the second button 210, and the third button 212 may not be arranged near the control wheels 202 and 204, and / or may be arranged in different positions on one side of the control handle 202. For example, the first button 208, the second button 210, and the third button 212 may be arranged in a manner that is more... Figure 2A The position depicted as being closer to or further away from the insertion tube 234, or arranged on the first side 207, is more... Figure 2A The higher or lower as depicted. In some examples, control wheels 202 and 204 may also be arranged higher than... Figure 2A The objects depicted are closer to or further away from the insertion tube 234. Furthermore, the spacing between the first button 208, the second button 210, and the third button 212 can be close together, such as... Figure 2A The distances between the buttons (e.g., to allow easy access by the operator's thumb) or between the first button 208, the second button 210 and the third button 212 may be greater (e.g., to allow different buttons to be easily distinguished).

[0064] Figure 2BAn exploded view of a control handle 220 is shown in an example where the first button 208, the second button 210, and the third button 212 are part of a capacitance sensor button. A capacitance sensing plate 252 is shown behind the first button 208, the second button 210, and the third button 212. As described above, when the capacitance sensor button is pressed, the circuitry of the capacitance sensing plate 252 is completed, which sends a signal to the ultrasound imaging system.

[0065] Turn now Figure 3A This illustrates a first exemplary configuration of a set of controls 300 for a TEE control handle 301 of an ultrasound imaging system according to an embodiment. The ultrasound imaging system may be a non-limiting example of the ultrasound imaging system 200 of FIG. 28, and the TEE control handle 301 may be... Figure 2A A non-limiting example of the TEE control handle 220. A first exemplary configuration of a set of controls 300 includes a first button 302, a second button 304, and a third button 306, which are arranged on one side of the TEE control handle 301 and on the first control wheel 314 (e.g., Figure 2A First control wheel 202), second control wheel 316 (e.g., Figure 2A The second control wheel 204) and control wheel lock 318 (e.g., Figure 2A Below the control wheel lock 206, the first control wheel 314 and the second control wheel 316 are coaxially arranged on the top side of the TEE control handle 301. In the depicted example, the control wheel lock 318 is coaxially arranged with the first control wheel 314 and the second control wheel 316, but in other embodiments, the control wheel lock 318 may be arranged at different locations on the TEE control handle 301. For example, the control wheel lock 318 may be placed on different sides of the TEE control handle 301, or in a position that is not coaxial with the first control wheel 314 and the second control wheel 316. In still other examples, the lock may be a button on one side of the TEE control handle 301.

[0066] In a first exemplary configuration of a set of controls 300, the first button 302 is a backward button, as indicated by the backward pointing arrow 308 on the first button 302. The second button 304 is a selection button, as indicated by the OK label 310. The third button 306 is a forward button, as indicated by the forward pointing arrow 312 on the third button 306. In one example, when the operator is using the TEE control handle 301, the operator can hold the TEE control handle 301 by resting the bottom surface of the TEE control handle 301 against the operator's palm, so that the first button 302, the second button 304, and the third button 306 can be operated by the operator's thumb.

[0067] Brief Reference Figure 3B View 320 of a first exemplary configuration of a set of controls 300 shows the orientation of the first button 302, the second button 304, and the third button 306 when an operator is using the TEE control handle 301, wherein the operator's hand 322 grips the TEE control handle 301 with the operator's palm resting against the bottom surface 319 of the TEE control handle 301. When the TEE control handle is gripped as shown in view 320, the operator's thumb 324 can be positioned above the first button 302, the second button 304, and the third button 306, wherein the first button 302, the second button 304, and the third button 306 are within a comfortable range of the thumb 324.

[0068] The first button 302 (e.g., a backward button), the second button 304 (e.g., a select button), and the third button 306 (e.g., a forward button) can be functionally mapped to one or more actions in the user interface of the ultrasound imaging system, as described below. Figure 4A As stated above.

[0069] Turning Figure 4A The graphical UI 400 of the ultrasound imaging system is shown (e.g., Figure 1 The first embodiment of the ultrasound imaging system 100 (UI 115). UI 400 can be used by an operator to examine a patient in a TEE ultrasound device (e.g., Figure 2A The operation of the handheld ultrasound device 200 is displayed on the display device of the ultrasound imaging system.

[0070] UI 400 may include various display panels, including an ultrasound image panel 402. The ultrasound image panel 402 may include an ultrasound image 404 acquired by a probe of an ultrasound imaging system, which can be displayed by the operator via the handle of the TEE ultrasound device (e.g., Figure 2A The TEE control handle 220 is used to operate the TEE ultrasound device and update the image. For example, when the operator moves the probe of the TEE ultrasound device forward or backward, the lens / transducer of the probe can move, causing the view of one or more anatomical features scanned by the lens / transducer to change. When the view of the scanned one or more anatomical features changes, the ultrasound image 404 can be updated. In one example, the probe scans the patient in real time, thereby updating the ultrasound image 404 according to the frame rate of the TEE ultrasound device (e.g., 7 frames / second to 20 frames / second).

[0071] The ultrasound imaging panel 402 may include additional information. For example, the ultrasound imaging panel 402 may include an electrocardiogram (EKG) graph 416, which can indicate the strength and duration of the patient's heartbeat. The ultrasound imaging panel 402 may also include a text information display 406, which includes information such as the duration of the examination, frame rate per second, scan frequency, scan depth, etc., which can be updated by the operator as the probe moves and / or over time. The information panel 408 may also include additional, similar, and / or other textual or graphical information. In one example, the information panel 408 displays one or more settings and / or parameters of the ultrasound imaging system. For example, the information panel 408 may display the frame rate per second, scan frequency, scan depth, dB power, dB gain, scan depth in cm, etc.

[0072] UI 400 may include an image storage panel 410 that can display one or more selected images, such as the exemplary selected ultrasound image 412. During the examination, the operator may wish to capture and save one or more ultrasound images (e.g., frames) for later reference. For example, the operator may detect abnormalities in the patient's anatomical features, such as vegetations or anatomical feature damage, and may wish to save images of the abnormalities to include them in the patient's file, and / or for discussing treatment options with the patient, and / or for consulting a specialist, and / or for different reasons. In one example, the operator uses selection buttons on the handle of the TEE ultrasound device (e.g., Figure 3A and Figure 3B The second selection button 304) is used to capture and store ultrasound images. For example, when an operator encounters an ultrasound image that the operator wishes to save, the operator can use their thumb (e.g., Figure 3B The operator (using thumb 324) selects the selection button to save the ultrasound image to the image storage area. When the ultrasound image is saved to the image storage area, it can be displayed as a thumbnail image within the image storage area panel 410, as shown in the example of selecting ultrasound image 412. In another example, a double-click function can be used to select an ultrasound image. For example, the operator can press the selection button for the first time to select an ultrasound image (e.g., the current frame in the ultrasound image panel 402) (this operation can freeze the image on the screen), and can press the selection button a second time to save the image to the image storage area.

[0073] UI 400 may include a control wheel panel 414 that provides one or more control wheels on the handle of the TEE ultrasound device (e.g., Figure 2AThe handheld ultrasound device 200 displays a graphical indication of the rotational position and / or orientation of the first control wheel 202 and the second control wheel 204. For example, when the operator refers to the above... Figure 2A When the one or more control wheels are manipulated as described, the probe of the TEE ultrasound device can be accessed via a hinged neck assembly (e.g., Figure 2A The articulated neck assembly 232 of the handheld ultrasound device 200 bends in one or more free planes to allow the operator to advance and / or tilt the probe with minimal soft tissue damage to the esophagus to facilitate ultrasound image acquisition. When adjusting the orientation of the probe relative to the insertion tube of the TEE ultrasound device according to the rotation of the one or more control wheels, the control wheel panel 414 can update the graphical representation of the one or more control wheels to indicate their rotational position. Furthermore, the control wheel panel 414 may include an indication of whether one or more of the one or more control wheels are locked. See below for further details. Figures 5A to 5D and Figure 8 The control wheel panel 414 is described in more detail.

[0074] In some examples, UI 400 may additionally include a virtual trackball interface 418, as described below. Figure 4C Describe the virtual trackball interface.

[0075] UI 400 may include a shortcut panel 420, which may include one or more shortcut control elements (also referred to herein as shortcut controls) that can be accessed via one or more buttons arranged on the handle of the TEE ultrasound device (e.g., Figure 3A Access is made via a first exemplary configuration of a set of controls 300 (a first button 302, a second button 304, and a third button 306). In one example, each shortcut element in the one or more shortcut controls replicates an existing function of the ultrasound imaging system (e.g., available via the ultrasound imaging system's menu), thereby allowing the operator to access existing functions more easily and quickly via the one or more buttons on the TEE control handle. Furthermore, accessing existing functions via a menu can involve using a separate input device (e.g., a mouse, touchpad, etc.), which may require using a non-dominant hand or removing the hand from the TEE control handle to manipulate the separate input device, which can be cumbersome and / or distract the operator from performing the examination.

[0076] In the depicted example, the shortcut key panel 420 includes an exemplary shortcut key selection with an angle shortcut key 422, a storage shortcut key 424, a 2D mode shortcut key 426, and a color mode shortcut key 428. In other examples, the shortcut key selection includes additional shortcut keys or a smaller number of shortcut keys. The shortcut key selection can be configured before inspection to include multiple functions expected by the operator, some of whom may expect a larger number of functions and corresponding shortcut keys, while others may expect a smaller number of functions and corresponding shortcut keys. Shortcut keys may correspond to operating modes of the ultrasound system (e.g., 2D mode, 3D mode, 4D mode, color mode, etc.), where selecting a shortcut key associated with a mode causes switching from the current operating mode to the selected operating mode. Shortcut keys may also correspond to parameters or settings of the ultrasound imaging system. In some examples, selecting a shortcut key associated with a parameter or setting of the ultrasound imaging system causes switching between two parameter values ​​or settings of the ultrasound imaging system. In other examples, selecting a shortcut key associated with a parameter or setting of the ultrasound imaging system adjusts the parameter or setting to a pre-selected value. For example, selecting the angle shortcut 422 increases or decreases the divergence angle of the probe lens by a pre-selected angle (e.g., 1°, 5°, 45°, or 90°, depending on the operator's preference). In other examples, selecting a shortcut associated with a parameter or setting of the ultrasound imaging system shifts the focus of UI 400 to a different set of control elements to set or reset the parameter or setting. For example, selecting a shortcut associated with a parameter or setting can bring up a window in UI 400 where the operator can select or enter a new parameter value or setting; or it can shift the focus to another panel of UI 400 (e.g., the virtual trackball interface panel 418) where the operator can select or enter a new parameter value or setting. Shortcuts can also correspond to permitted actions of the ultrasound system. For example, selecting the save shortcut 424 saves the current ultrasound image 404 to the image storage area.

[0077] In one example, the operator navigates between shortcut keys on the shortcut key panel 420 by pressing one or more buttons on the handle of the TEE ultrasound device, allowing the focus of the UI 400 to move forward or backward via shortcut key selection to select the desired shortcut key in the shortcut key selection, as indicated by forward arrow 432 and backward arrow 430, respectively. For example, one of the buttons could be a forward directional button (e.g., Figure 3A The third forward button 306 allows the operator to move forward via a shortcut key, and different buttons among the one or more buttons can be backward directional buttons (e.g., Figure 3AThe first backward button 302 allows the operator to move forward in the opposite direction via a shortcut key. However, another button among these one or more buttons can be a selection button (e.g., Figure 3A The second selection button 304 allows the operator to select a shortcut key from the shortcut key selection. Therefore, if the focus of UI 400 (e.g., the default focus due to the position of angle shortcut key 422, or from a previously selected focus, etc.) is on angle shortcut key 422, the operator can press the forward directional button once to move the focus forward (e.g., toward the operator's right) to the stored shortcut key 424. While the focus is moving forward, a first visual indication can be provided in UI 400 to indicate the current position of the focus (e.g., by illumination, backlighting, outlining, or other forms of highlighting). The operator can press the forward directional button a second time to move the focus forward to the 2D mode shortcut key 426. If the operator wishes to switch to 2D operation mode, they can select 2D operation mode by pressing the selection button while the focus is on 2D mode shortcut key 426. Alternatively, if the focus of UI 400 is on color mode shortcut 428, the operator can press the backward navigation button once to move the focus backward (e.g., to the operator's left) to 2D mode shortcut 426, at which point the operator can select 2D mode shortcut 426. When 2D operating mode is selected, a secondary visual indication can be provided in UI 400 to indicate the selection of 2D mode (e.g., by illuminating, backlighting, outlining, or other forms of highlighting). In some examples, the focus of UI 400 may remain the operator's last selection and be indicated to the operator (e.g., to show the current operating mode), while in other examples, the focus indication may not be maintained (e.g., after changing settings or parameters, saving an image, etc.). In one example, when 2D mode is selected, 2D mode shortcut 426 is changed to indicate a 4D mode shortcut, allowing the user to switch from 2D mode to 4D mode using the same button.

[0078] In one example, a limited number of shortcut controls are displayed in the shortcut panel 420, where the limited number is the number of shortcut controls visible at one time in the shortcut panel 420. In other examples, a larger number of shortcut controls may be displayed in the shortcut panel 420, where some shortcut controls in the selection are visible, while one or more shortcut controls in the selection are invisible. For example, when navigating through the selected shortcut controls in the direction of the forward arrow 432 via the forward directional button, as the operator navigates to the right side of the shortcut panel 420, invisible additional shortcut elements may appear on the right side of the shortcut panel 420, while visible shortcut controls may shift to the left side as indicated by the backward arrow 430, where shortcut elements visible on the left side of the shortcut panel 420 disappear from the view. Alternatively, when navigating through the selected shortcut controls in the direction of the backward arrow 430 via the backward navigation button, as the operator navigates to the left side of the shortcut panel 420, invisible additional shortcut elements can appear on the left side of the shortcut panel 420, while visible shortcut controls can shift to the right side as indicated by the forward arrow 432, where shortcut elements visible on the right side of the shortcut panel 420 disappear from the view. This allows for a number of shortcut controls exceeding the number that can be visible at once in the shortcut panel 420, and the operator can scroll through the shortcut controls using one or more buttons on the control handle.

[0079] For example, an operator may acquire images of a patient's heart in the 2D operating mode of an ultrasound imaging system and may wish to switch to the color operating mode of the ultrasound imaging system. The operator may not want to drop the TEE control handle to switch from 2D to color operating mode using the ultrasound imaging system's menu, as doing so could cause unintended adjustments to the probe's position and / or orientation, resulting in changes to the ultrasound image generated in the ultrasound image display panel 402. The operator may also not want to use their left hand to manipulate the mouse to open menus, as the mouse's button configuration is designed for right-handed use, making it inconvenient to select the appropriate mouse button. Since the operator does not want to drop the TEE control handle and does not want to use their left hand to open menus, they can use one or more buttons on the TEE control handle to switch from 2D to color mode. To switch from 2D to color mode, the operator can press the forward and / or backward navigation buttons to navigate between shortcut controls on the shortcut panel 420 (e.g., depending on the default position or the currently selected shortcut element). As the operator navigates from one shortcut element to the next using the forward and / or backward navigation buttons, the location of the focus is highlighted in a first highlighting mode (illuminating the location, backlighting the location, outlining the location, etc.). When the operator navigates to color mode shortcut 428 (e.g., indicating a switch to the ultrasound imaging system's color operation mode), the operator can press the selection button on the TEE control handle to select color mode shortcut 428. When the operator selects color mode shortcut 428, the location of the focus is highlighted in a second highlighting mode (illuminating the location, backlighting the location, outlining the location, etc.) to indicate the selection of the shortcut (e.g., color mode shortcut 428) at the location of the focus. Simultaneously, the ultrasound imaging system switches from 2D operation mode to color operation mode. By using one or more buttons on the TEE control handle to switch from 2D mode to color mode, the operator can maintain visual focus on the UI400 during the transition from 2D mode to color mode, thereby facilitating faster and smoother examinations.

[0080] One or more buttons on the handle of the TEE ultrasound device can also be configured to perform a function when selected rapidly and sequentially or when selected in a pre-defined mode. In one example, the selection button can be configured to perform an action, such as capturing the current ultrasound image 404 and storing it in the image storage area, when double-clicked. Alternatively, the selection button can be configured to switch between two states of the ultrasound imaging system (e.g., two operating modes, two alternative parameter settings, etc.) when double-clicked. For example, the operator might want to repeatedly compare ultrasound images acquired during 2D operating mode with those acquired during 4D operating mode, or compare ultrasound images acquired according to a first set of parameters with those acquired according to a second set of parameters.

[0081] In other examples, the handle of the TEE ultrasound device may not include multiple buttons and may include an alternative set of controls for navigation within the shortcut panel 420, as referenced below. Figure 3C As stated above.

[0082] Turn now Figure 3C The TEE control handle view 350 illustrates a second exemplary configuration of a set of controls 352 for UI interaction with the ultrasound imaging system via the TEE control handle 360. The ultrasound imaging system can be a non-limiting example of the ultrasound imaging system 200 of FIG. 28, and the TEE control handle 360 ​​can be... Figure 2A A non-limiting example of the TEE control handle 220. According to one embodiment, a second exemplary configuration of a set of controls 352 includes a selection button 356 positioned within a circular radial touchpad 354 arranged on one side of the TEE control handle 360 ​​and within a first control wheel 314 (e.g., Figure 2A First control wheel 202), second control wheel 316 (e.g., Figure 2A The second control wheel 204) and control wheel lock 318 (e.g., Figure 2A Below the control wheel lock 206, the first control wheel 314, the second control wheel 316, and the control wheel lock 318 are coaxially arranged on the top side of the TEE control handle 360. In one example, the selection button 356 (marked with the text "OK" in Figure 3C) is used to indicate user selections in the UI of the ultrasound imaging system, and the radial touchpad 354 is used to scroll through one or more control elements (e.g., ...) in the display panel of the UI. Figure 4A(Shortcut key selection is displayed on the shortcut key display panel 420). When the operator is using the TEE control handle 360, the operator can hold the TEE control handle 360 ​​by resting the bottom surface 358 against the operator's palm, so that the selection button 356 and the radial touchpad 354 of the second exemplary configuration of the set of controls 352 can be operated by the operator's thumb.

[0083] Although Figure 3C An exemplary set of controls 352 includes a radial touchpad 354, but in one example, the radial touchpad 354 is a single-axis touchpad or a single-axis touchpad ( Figure 3C (Not depicted) Instead, the operator can perform a dragging operation on a single-axis touchpad (e.g., along a single axis of the touchpad) with their finger or thumb to scroll through one or more control elements in the UI's display panel. The advantage of using a single-axis touchpad instead of a radial touchpad 354 is that it reduces the amount of space occupied by a set of controls 352, allowing for the use of smaller TEE control handles and / or easier operation with smaller TEE control handles. Therefore, it should be understood that, for the example of replacing the radial touchpad 354 with a single-axis touchpad, as described herein, scrolling the radial touchpad 354 in a first rotational direction and / or a second opposite rotational direction (e.g., opposite to the first rotational direction) can be functionally equivalent to scrolling the single-axis touchpad in the first axial direction and / or the second opposite axial direction. Thus, references to scrolling via the radial touchpad 354 in the first rotational direction and / or the second opposite rotational direction (e.g., dragging the thumb along the radial touchpad 354) can be replaced with references to scrolling the single-axis touchpad in the first axial direction and / or the second opposite axial direction.

[0084] Brief Reference Figure 3D View 370 of a second exemplary configuration of a set of controls 352 shows the orientation of the selection button 356 and the radial touchpad 354 when the operator is using the TEE control handle 360, wherein the operator's hand 372 grips the TEE control handle 360 ​​with the operator's palm resting against the bottom surface 358 of the TEE control handle 360. When the TEE control handle 360 ​​is gripped as shown in view 370, the operator's thumb 374 can be positioned above the second exemplary configuration of the set of controls 352, wherein the selection button 356 and the radial touchpad 354 are within the comfortable range of the thumb 374.

[0085] Selection button 356 and radial touchpad 354 can be functionally mapped to one or more actions in the user interface of the ultrasound imaging system, as described below. Figure 4B As stated above.

[0086] Go to Figure 4B , showed Figure 4A A second embodiment 440 of the graphical user interface 400 includes an ultrasound image panel 402, an information panel 408, a control wheel panel 414, an image storage area panel 410, and a shortcut key panel 420. In the second embodiment 440, the shortcut key panel 420 includes... Figure 4A The expanded shortcut key display shows the element selection, which, in addition to the angle shortcut key 422, 2D mode shortcut key 426, and color mode shortcut key 428, also adds the 4D mode shortcut key 446, freeze shortcut key 452, and depth shortcut key 454. Although Figure 4B Forward and backward arrows are not depicted, but it should be understood that other implementations may include forward and backward arrows and / or similar graphic elements.

[0087] In the depicted implementation, the operator of the ultrasound imaging system can use a TEE control handle (e.g., Figure 3A The operator can use one or more controls on a set of controls 300 (in a first exemplary configuration) to navigate through one or more control or display elements (e.g., shortcut panel 420 or image storage panel 410) within a panel of the graphical UI 440, and can also use one or more controls on the TEE control handle to navigate (e.g., toggle) between the shortcut panel 420 and the image storage panel 410. In one example, the operator uses the forward directional button (e.g., Figure 3A The third forward button 306) is used to scroll through one or more shortcut keys within the shortcut key panel 420 or to scroll through multiple images in the image storage area panel 410, and the operator can use the backward navigation button (e.g., Figure 3A The first back button 302) navigates between the shortcut key panel 420 and the image storage area panel 410.

[0088] For example, an operator might want to scroll right on the shortcut panel 420 to select the 4D mode shortcut 446. The UI focus can initially be on the angle shortcut 422 (e.g., by default, at startup, etc.). By selecting the forward directional button, the focus can be shifted to the 4D mode shortcut 446, whereby the outline 425 of the 4D mode shortcut 446 can be illuminated to indicate the focus position. The operator can press the selection button on the TEE control handle (e.g., ...). Figure 3AThe second selection button 304 is used to select the 4D mode shortcut key 446, thereby switching the ultrasound imaging system from the current or default mode to 4D mode. The color, illumination, or other visual characteristics of the 4D mode shortcut key can be changed to indicate that 4D mode has been selected. When operating in 4D mode, the operator can save the ultrasound image 404 to the image storage area by double-clicking the selection button, thereby displaying the ultrasound image thumbnail 412 in the image storage area panel 410. If the operator wishes to view the saved ultrasound image 404 in the image storage area, the operator can select the backward navigation button to shift the focus to the image storage area panel 410. When the focus is shifted to the image storage area panel, the focus can be placed on the thumbnail 448 by default (e.g., the image positioned on the upper left side of the image storage area panel 410). To view the saved ultrasound image 404, the operator can navigate to the right (e.g., scroll) to the ultrasound image thumbnail 412 by selecting the forward navigation button. When the focus is on image thumbnail 412, the operator can select image thumbnail 412 by pressing the selection button, which will display the saved ultrasound image 404 in the ultrasound image panel 402. If the operator wishes to adjust the probe deflection angle, the operator can navigate (e.g., switch) back to the shortcut panel 420 by selecting the back navigation button, which will shift the focus to the 4D mode shortcut key 446 of the shortcut panel 420 (e.g., the currently or previously selected mode). To navigate to angle shortcut 422, the operator can scroll to the right of shortcut panel 420 (e.g., shift focus) as follows: first select the forward navigation button to focus on 2D mode shortcut 426, second select the forward navigation button to focus on color mode shortcut 428, third select the forward navigation button to focus on freeze shortcut 452, fourth select the forward navigation button to focus on depth shortcut 454, and fifth select the forward navigation button to cycle back to angle shortcut 422 (e.g., since the backward navigation button is used to navigate between shortcut panel 420 and image storage area 410). In this way, the operator can use the forward navigation button, backward navigation button, and select button to navigate within or between shortcut panel 420 and image storage area 410.

[0089] In another example, the one or more controls on the TEE control handle may not include a forward and backward navigation button (e.g., Figure 3A A first exemplary configuration of a set of controls 300), and the one or more controls on the TEE control handle may include a radial touchpad and selection buttons (e.g., Figure 3CThe second exemplary configuration of the set of controls 352 includes a radial touchpad 354 and a selection button 356. An operator can use the radial touchpad to scroll through one or more shortcut keys within the shortcut key panel 420 or scroll through multiple images in the image storage area panel 410. For example, an operator can drag their thumb (or finger) around the radial touchpad in a first rotational direction (e.g., clockwise or counterclockwise) to scroll through one or more shortcut controls in the shortcut key panel 420 and / or multiple images in the image storage area panel 410, and an operator can drag their thumb (or finger) around the radial touchpad in a second opposite rotational direction to navigate between the shortcut key panel 420 and the image storage area panel 410.

[0090] Furthermore, the radial touchpad can be used to navigate between additional panels of the graphical UI 400 (e.g., panels other than the image storage panel 410 and the shortcut key panel 420). For example, Figure 4C It shows Figure 4A A third embodiment 470 of the graphical user interface 400 has three display panels, wherein an operator can navigate between a shortcut panel 420 (labeled box 1), a virtual trackball interface 418 (labeled box 2), and an image storage panel 410 (labeled box 3). In one example, the operator drags their thumb (or finger) around the radial touchpad in a first rotational direction (e.g., clockwise or counterclockwise) to navigate between the shortcut panel 420, the virtual trackball interface 418, and the image storage panel 410 in a first direction, and the operator drags their thumb (or finger) around the radial touchpad in a second opposite rotational direction (e.g., clockwise or counterclockwise) to navigate between control elements displayed within the shortcut panel 420, the virtual trackball interface 418, and the image storage panel 410 in a second direction.

[0091] For example, in the third embodiment 470, the shortcut key panel 420 includes Figure 4BThe extended shortcut control selection is displayed in multiple rows. In one example, the operator can use the radial touchpad to scroll cyclically through the extended shortcut control selection, where dragging the thumb or finger around the radial touchpad in a second opposite rotation direction shifts the focus of the graphical UI 400 in that direction through the shortcut control selection. For example, the operator can drag their thumb around the radial touchpad in a second opposite rotation direction, which causes the UI's focus to shift in that direction from the shortcut control with the default focus to the shortcut control selection. If an operator wishes to shift focus to the image storage panel 410 or the virtual trackball interface 418, the operator can drag their thumb around the radial touchpad in a first rotation direction, which may cause the focus of the UI to shift in the first rotation direction through the image storage panel 410 or the virtual trackball interface 418 (for example, if the first rotation direction is clockwise, the focus is shifted to the image storage panel 410 and then to the virtual trackball interface 418, and if the first rotation direction is counterclockwise, the focus is shifted to the virtual trackball interface 418 and then to the image storage panel 410).

[0092] The virtual trackball interface 418 may include a first button 472, a second button 474, a third button 476, and a fourth button 478 arranged around a center point 480. The first button 472, second button 474, third button 476, and fourth button 478 may be control elements mapped to additional functions, settings, parameters, actions, etc., of the ultrasound imaging system. In one example, the first button 472, second button 474, third button 476, and fourth button 478 are control elements corresponding to functions available via the physical trackball interface of the ultrasound imaging system, which the operator can access by: adjusting the focus of the ultrasound imaging system to the desired button among the first button 472, second button 474, third button 476, and fourth button 478 by performing a drag operation around the radial touchpad in a second opposite rotational direction as described above, and selecting the desired button among the first button 472, second button 474, third button 476, and fourth button 478 via a selection button. By using the virtual trackball interface 418 to access functions available via the physical trackball interface, the operator can access these functions more efficiently. For example, a physical trackball interface may be relatively far from the operator's position during an ultrasound examination, or the physical trackball interface may be ergonomically difficult to manipulate during an ultrasound examination (e.g., due to confined spaces). Another advantage of accessing functions available via the virtual trackball interface 418 is that the risk of cross-contamination can be reduced because the operator does not touch the physical trackball interface.

[0093] In another example, the operator can use the virtual trackball interface 418 to adjust one or more settings of the parameters of the ultrasound imaging system selected in the shortcut panel 420. For example, to adjust the probe's depth parameter, the operator can drag their thumb around the radial touchpad in a first rotational direction to focus the UI on the shortcut panel 420, and can drag their thumb around the radial touchpad in a second opposite rotational direction to focus the UI on the depth shortcut control 454. In one example, the operator can use the selection button to select the shortcut panel 420, while in other examples, the operator can select the shortcut panel 420 without using the selection button, and can switch from dragging their thumb around the radial touchpad in the first rotational direction to dragging their thumb around the radial touchpad in the second opposite rotational direction to select the shortcut panel 420. When the focus is on the depth shortcut control 454, the operator can select the depth shortcut control 454 by pressing the selection button. When the selection button is pressed, the depth shortcut control 454 can be illuminated to indicate that the depth shortcut control 454 has been selected. When the depth shortcut control 454 is selected, the first depth setting of the depth parameter can be displayed in the first rotation element 472, the second depth setting of the depth parameter can be displayed in the second rotation element 474, the third depth setting of the depth parameter can be displayed in the third rotation element 476, and the fourth depth setting of the depth parameter can be displayed in the fourth rotation element 478. The operator can shift the focus of the graphical UI 400 to the virtual trackball interface 418 by dragging their thumb around the radial touchpad in the second opposite rotation direction, and the operator can navigate between the first rotation element 472, the second rotation element 474, the third rotation element 476, and the fourth rotation element 478 by dragging their thumb around the radial touchpad in the first rotation direction, and select the appropriate depth setting of the depth parameter by pressing the selection button. In this way, the operator can use the controls on the TEE control handle to adjust the parameter settings of the ultrasound imaging system without using a separate input device (e.g., mouse, trackball, etc.), while keeping the operator's attention on the ultrasound image displayed in the ultrasound image panel 402.

[0094] It should be understood that the examples provided herein are for illustrative purposes only and not for limitation, wherein other buttons, touchpads or similar controls of the TEE control handle may be combined, interchanged or replaced to guide the focus of the graphical UI 400 between and within the display panels of the graphical UI 400.

[0095] Now for reference Figure 5AAn exemplary graphic control wheel display element 500 is shown, which indicates the rotational position of the first control wheel, the rotational position of the second control wheel, and the state of the control wheel lock of the TEE control handle of the ultrasound imaging system (e.g., Figure 2A The handheld ultrasound device includes a first control wheel 202, a second control wheel 204, and a control wheel lock 206. The graphic control wheel display element 500 can be displayed in the UI 400, for example, displayed on... Figure 4A , Figure 4B and Figure 4C The control wheel display panel 414. When the operator of the TEE control handle manipulates the first and second control wheels, the probe connected to the TEE control handle can be accessed via a hinged neck assembly (e.g., Figure 2A The articulated neck assembly 232 of the handheld ultrasound device 200 is bent or tilted in one or more free planes, and when the position and orientation of the probe relative to the insertion tube of the TEE ultrasound device are adjusted according to the rotation of the first control wheel and the second control wheel, the graphic control wheel display element 500 can update the graphic representation of the first control wheel and the second control wheel to indicate the rotation of the first control wheel and the second control wheel.

[0096] The graphic control wheel display element 500 may include a first graphic control wheel display 502 and may include a second graphic control wheel display 504, the first graphic control wheel display showing the rotational position of the first control wheel of the TEE control handle, and the second graphic control wheel display showing the rotational position of the second control wheel of the TEE control handle. In one example, the first graphic control wheel display 502 and the second graphic control wheel display 504 may have a size difference based on the size difference between the first and second control wheels. For example, if the first control wheel has a smaller diameter than the second control wheel, then the first graphic control wheel display 502 may have a smaller diameter than the second graphic control wheel display 504. Alternatively, if the first control wheel has a larger diameter than the second control wheel, then the first graphic control wheel display 502 may have a larger diameter than the second graphic control wheel display 504.

[0097] In one example, a first control wheel controls the deflection of the probe connected to the TEE control handle in a first free plane, which corresponds to the movement of the probe in a left-hand or right-hand direction (e.g., toward the left or right side of the patient), indicated on the first graphic control wheel display 502 by a left-hand direction indicator 510 and a right-hand direction indicator 512; and a second control wheel controls the deflection of the probe connected to the TEE control handle in a second free plane, which corresponds to the movement of the probe in a front-hand or rear-hand direction (e.g., toward the front or back of the patient), indicated on the first graphic control wheel display 502 by a front-hand direction indicator 522 and a rear-hand direction indicator 524. While for the purposes of this example, the first control wheel indicates movement of the probe in the left / right direction and the second control wheel indicates movement of the probe in the front / back direction, it should be understood that in other examples, the first control wheel may indicate movement of the probe in the front / back direction and the second control wheel may indicate movement of the probe in the left / right direction, or without departing from the scope of this disclosure, the first and / or second control wheels may indicate movement in different dimensions.

[0098] The first graphic control wheel display 502 may include a rotation position indicator 506 that indicates the rotation position of the first control wheel relative to a default neutral position in which the probe coupled to the TEE control handle has a 0° deflection toward either the left or right side of the patient. Similarly, the second graphic control wheel display 504 may include a rotation position indicator 514 that indicates the rotation position of the second control wheel relative to a default neutral position (indicated by the dashed line 515) in which the probe coupled to the TEE control handle has a 0° deflection toward either the front or back of the patient. For example, the rotation position indicator 514 of the second graphic control wheel display 504 may be offset 30° clockwise from the neutral position, as indicated by the text indicator 516 of the second graphic control wheel display 504, which indicates that the second control wheel 504 has rotated 30° from the neutral position.

[0099] exist Figure 5A In a non-limiting example, the rotation position indicator 506 is graphically represented by a triangular arrow pointing in a direction corresponding to the rotational position of the first control wheel relative to the neutral position (e.g., vertically upward). The graphical representation of the rotation position indicator can be further enhanced by a textual representation (e.g., a text indicator 516 such as that of the second control wheel 504). Therefore, according to... Figure 5AThe example depicted in the text suggests that, based on the first graphic control wheel display 502 and the second graphic control wheel display 504, it can be inferred that the probe is not deflected toward the left or right side of the patient, and that the probe is deflected 30° from the neutral position toward the back of the patient.

[0100] The first graphic control wheel display 502 may include a lock indicator 508, and the second graphic control wheel display 504 may include a lock indicator 518. Lock indicators 508 and 518 can indicate the lock status of the first and second control wheels of the TEE control handle. For example, lock indicator 508 can indicate that the first control wheel is not locked by displaying an image of an unlocked padlock, while lock indicator 518 can indicate that the second control wheel is locked by displaying an image of a locked padlock. Furthermore, the locked status can be indicated by a color change or similar appearance change of the locked padlock.

[0101] Figure 5B An alternative configuration 540 of a first graphic control wheel display 502 and a second graphic control wheel display 504 is shown, wherein a lock indicator 508 of the first graphic control wheel display 502 indicates that the first control wheel is in a locked position, while a lock indicator 518 of the second graphic control wheel display 504 indicates that the second control wheel is not in a locked position. In the first alternative configuration 540, the first graphic control wheel display 502 includes a text indicator 546 indicating the degree of rotation of the first control wheel (e.g., 0°) and the corresponding degree of deflection of the probe in the left or right direction (e.g., 0°). Additionally, the rotation position indicator 506 of the first graphic control wheel display 502 is not highlighted because the probe is in a neutral, straight position in a first (e.g., left-right) free plane. However, the rotation position indicator 514 of the second graphic control wheel display 504 is highlighted (e.g., via color change, intensity change, etc.) to indicate that the second control wheel has rotated 30° from the neutral position. In some examples, the ratio of wheel rotation angle to bending angle can vary depending on the probe.

[0102] In another example, the text indicator 546 of the first graphic control wheel display 502 and / or the text indicator 516 of the second graphic control wheel display 504 may not respectively indicate the rotation degree of the first and second control wheels and the corresponding deflection degree of the probe, and may include different text indicators for one or more states of the first graphic control wheel display 502 and / or the second graphic control wheel display 504. Figure 5CIn a second alternative configuration 550 of the first graphic control wheel display 502 and the second graphic control wheel display 504, a text indicator 546 is displayed including text indicating that the first control wheel is in a locked state, and a text indicator 516 of the second graphic control wheel display 504 is displayed including text indicating a higher degree of deflection of the probe in the rearward direction (e.g., 30°). In one example, the text indicator 546 includes a text description of the degree of deflection of the probe in the left or right direction when the first control wheel is unlocked, which changes to a locked state when the first control wheel is locked, and the text indicator 516 includes a text description of the degree of deflection of the probe in the front or rearward direction when the second control wheel is unlocked, which changes to a locked state when the second control wheel is locked.

[0103] Figure 5DA third alternative configuration of the first graphic control wheel display 502 and the second graphic control wheel display 504 is shown, wherein the rotation position indicator 506 and text indicator 546 of the first graphic control wheel display 502 and the rotation indicator 514 and text indicator 516 of the second graphic control wheel display 504 are replaced with a pointer indicator configuration. The first graphic control wheel display 502 includes a pointer indicator 566 that rotates about the center point of the first graphic control wheel display 502 to indicate the rotation position of the first control wheel, wherein the deflection degree of the probe is indicated by the angle between the pointer indicator 566 and the neutral point in the clockwise or counterclockwise direction and according to the position where the point 567 of the pointer indicator 566 falls along the color gradient 562. Similarly, the second graphic control wheel display 504 includes a pointer indicator 576 that rotates around the center point of the second graphic control wheel display 504 to indicate the rotational position of the second control wheel. The degree of probe deflection is indicated by the angle between the pointer indicator 576 and the neutral point in the clockwise or counterclockwise direction, and by the position where the point 577 of the pointer indicator 576 falls along the color gradient 568. For example, if the pointer indicator 566 falls within a light-colored area around the neutral point (indicated by the center line 563), it can be inferred that the probe deflection is lower towards the right or left side of the patient. If the pointer indicator 566 falls within a dark area at the very end 564 of the counterclockwise color gradient 562, it can be inferred that the probe deflection is higher towards the left side of the patient. Alternatively, if the pointer indicator 566 falls within a dark area at the very end 565 of the clockwise color gradient 562, it can be inferred that the probe deflection is higher towards the right side of the patient. Similarly, if pointer 576 falls within the light-colored area around the neutral point (indicated by center line 573), it can be inferred that the probe has a low degree of deflection in the anterior or posterior direction of the patient. If pointer 576 falls within the dark area at the very end 578 of the counterclockwise color gradient 568, it can be inferred that the probe has a high degree of deflection in the anterior direction of the patient. Alternatively, if pointer 576 falls within the dark area at the very end 579 of the clockwise color gradient 568, it can be inferred that the probe has a high degree of deflection in the posterior direction of the patient.

[0104] Furthermore, the shading of color gradients 562 and 568 can indicate one or more desired degrees of bending of the probe. For example, the darkness of the shading of color gradient 562, near either or both of the outermost edges 564 and 565, can indicate the amount of strain on the probe's articulated neck assembly in the left or right direction, respectively, and the darkness of the shading of color gradient 568, near either or both of the outermost edges 564 and 565, can indicate the amount of strain on the probe's articulated neck assembly in the front or rear direction, respectively. By adjusting the shading of color gradients 562 and 568 (including independent adjustments), the operator can be provided with visual guidance for operating the TEE control handle to achieve desired results without causing excessive mechanical strain on the probe.

[0105] Now for reference Figure 6 A flowchart of an exemplary method 600 is shown, which is used for a first configuration with a medical imaging system (such as, via a TEE control handle) controlled by a set of controls. Figure 1 The UI interaction of the ultrasound imaging system 100. The first configuration of this set of controls may include a forward navigation button, a backward navigation button, and a selection button, respectively such as... Figure 3A A first exemplary configuration of a set of controls 300 includes a third button 306, a first button 302, and a second button 304. The TEE control handle may also include a first control wheel and a second control wheel, such as... Figure 2A The first control wheel 202 and the second control wheel 204. See below for reference. Figure 8 Method 800 describes user interaction with the first control wheel and the second control wheel. In one embodiment, during an ultrasound examination of a patient by an operator of a medical imaging system, the operations of method 600 can be stored in non-transitory memory and executed by a processor, such as... Figure 1 The medical imaging system 100 includes a non-transitory memory 120 and a processor 116.

[0106] At 602, method 600 includes a display device of a medical imaging system (e.g., Figure 1 The medical imaging system 100 displays medical images and related information on its display device 118. In one example, the medical image may be an ultrasound image. For example, the medical image may be a 2D, 3D, or 4D ultrasound image, a Doppler ultrasound image, or any other type of ultrasound image. In some examples, the medical image may be a single frame of a sequence of medical images acquired in real time from the patient, for example, during an ultrasound examination performed by a clinician. In other examples, the medical image may be an X-ray image, a CT image, an MRI image, a visible light image, or another type of medical image that includes anatomical features of interest.

[0107] In one example, as referenced above Figure 4A The medical images and related information displayed in the UI are generated by a probe positioned at the distal end of an insertion tube connected to the TEE control handle. When the operator of the medical imaging system begins an examination, they can use the TEE control handle to advance the probe into the patient's body cavity (e.g., the esophagus). As the probe advances into the body cavity, the ultrasound images displayed in the UI can be updated based on probe movement and / or adjustments to the probe's position and / or orientation via the first and second control wheels of the TEE control handle. For example, the operator can advance the probe into the patient's esophagus towards the ROI (region of interest) located behind the esophagus, via the TEE control handle. To navigate through one or more bends in the esophagus, as the operator advances the probe into the esophagus, they can bend the articulated neck mechanism connecting the probe to the insertion tube to angle the probe to the patient's left or right side via the first control wheel and to the patient's front or back side via the second control wheel. When the probe reaches a position in the esophagus close to the ROI, the operator can adjust the rotation position of the first and / or second control wheels to tilt the probe toward the patient's left or right side and / or front or back, respectively. This aligns the probe's lens so that the ultrasound beam is deflected away from the target object of the ROI at a vertical or near-vertical angle, thereby acquiring a medical image. Relevant information may be text information simultaneously displayed on the display device, which may include the frequency of the signal generated by the ultrasound probe, the depth of the ultrasound probe relative to the ROI, the time elapsed since the start of the examination, etc., as a list of non-limiting examples.

[0108] At 604, method 600 includes displaying a shortcut selection in the UI (e.g., the shortcut selection described above with reference to FIG. 4A). In one example, each shortcut in the shortcut selection is associated with a function of the medical imaging system. The function associated with the shortcut may include, for example, switching between operating modes of the medical imaging system (e.g., between 2D and 4D modes, or Doppler mode, or color mode, etc.); performing actions (e.g., showing, hiding, or expanding the display panel of the UI, freezing the medical image stream displayed in the UI to view a single frame, capturing one or more medical images displayed in the UI and / or storing them in an image storage area, etc.); setting, resetting, or adjusting parameters or settings of the medical imaging system (e.g., changing the divergence angle of the probe lens, adjusting the scan depth in centimeters, scan frequency, frame rate per second, dB power, dB gain of the probe, etc.); displaying additional information and / or graphical elements in the UI (e.g., overlays, reference images, annotations, guidance prompts, etc.); and / or other types of functions.

[0109] In one example, shortcut selections are pre-configured by the operator or healthcare organization prior to the examination. For instance, shortcut functions can be linked to one or more menu items on the medical imaging system, allowing the operator to add menu items to the selection or choose shortcut selections from a list of functions compiled by the medical imaging system's manufacturer.

[0110] Keyboard shortcuts can be found in the UI's shortcut panel (as mentioned above). Figures 4A to 4C The shortcut key panel (420) is displayed as an optional control element. In one example, the shortcut key panel is positioned along the bottom edge of the UI below the display of the ultrasound image, where shortcut keys can be displayed horizontally. Displaying shortcut keys may include displaying the name of each shortcut key in the shortcut key selection, or the shortcut keys may be represented as graphical control elements (e.g., buttons) with shapes that may have outlines (e.g., rectangles, circles, etc.). In some examples, names, shapes, and / or outlines may be displayed with various degrees of illumination, shadows, shading, etc., to distinguish one shortcut key from another, or to differentiate between the selected shortcut key and the shortcut key that receives the current focus of the UI. Additionally, the order of shortcut keys in the shortcut key selection can be configured, for example, to extend from the left side of the shortcut key panel to the right side. For example, shortcut keys may be listed in the order in which the operator selects them, or in the order of frequency of use by the operator or operator team. In one example, the operator may configure the order based on operator preferences.

[0111] At 605, method 600 includes setting the focus of the medical imaging system on a default shortcut key. For example, the default shortcut key could be a shortcut key located at the far left of the shortcut key panel (e.g., the first or initial shortcut in the shortcut key selection), or the default shortcut key could be a shortcut key most frequently or recently used by the operator or operator team, which may or may not be located at the far left of the shortcut key panel. In one example, the default focus can be indicated to the operator (e.g., by highlighting the shortcut key's name, shape, or outline). In other examples, the default focus may not be indicated to the operator, or the focus may be indicated in response to the operator adjusting the focus via a shortcut key button on the TEE control handle.

[0112] At 606, method 600 includes determining whether user input has been received from the set of controls on the TEE control handle (e.g., by the operator selecting a forward directional button, a backward directional button, or a select button, instead of the first or second control wheel). If it is determined at 606 that user input has not yet been received from the set of controls on the TEE control handle, method 600 proceeds to 607. At 607, method 600 includes updating any relevant information in the ultrasound image generated by the probe and / or the UI until it is determined that user input has been received. If it is determined at 607 that user input has been received from the set of controls on the TEE control handle, method 600 proceeds to 608.

[0113] At 608, method 600 includes determining whether a selection button has been selected (e.g., Figure 3A (Second selection button 304). If it is determined at 608 that the selection button has not been selected, method 600 proceeds to 610. At 610, method 600 includes determining whether the forward navigation button has been selected. If it is determined at 610 that the forward navigation button has been selected, method 600 proceeds to 612. At 612, method 600 includes moving the focus of the UI forward from the shortcut key where the focus is located to the adjacent shortcut key. In one example, the forward direction moves from the left side of the UI to the right side of the UI (e.g., the left and right sides of the operator). If it is determined at 610 that the forward navigation button has not been selected, method 600 infers that the backward navigation button has been selected, whereby method 600 proceeds to 614. At 614, method 600 includes moving the focus of the UI in the opposite direction from the shortcut key where the focus is located (referred to herein as the focus shortcut key) to the adjacent shortcut key. In one example, the opposite direction moves from the right side of the UI to the left side of the UI.

[0114] As the focus moves forward or backward, the UI can highlight the focus shortcut. In one example, highlighting the shortcut where the focus is located could include adjusting the color of the focus shortcut. For example, other shortcuts in the shortcut selection could be displayed in a first color, while the focus shortcut could be displayed in a different second color. In another example, the brightness of the focus shortcut could be adjusted (e.g., the focus shortcut is brightened, and other shortcuts in the shortcut selection are brightened less). Alternatively, the visual characteristics of the focus shortcut can be adjusted; for example, the text of the shortcut's label could be displayed in bold or underlined, or its brightness could be higher or lower than other shortcuts in the shortcut selection, or the shape or outline of the shortcut could be adjusted.

[0115] If it is determined at 608 that the selection button has been selected, method 600 continues to 616. At 616, method 600 includes determining whether the selection button has been double-clicked. If it is determined at 616 that the selection button has not been double-clicked, method 600 continues to 618. At 618, method 600 includes selecting the shortcut key for the currently focused object (e.g., to execute the function associated with the selected shortcut key), and method 600 ends. If it is determined at 616 that the selection button has been double-clicked, method 600 continues to 620. At 620, method 600 includes saving the medical image displayed in the UI in the image storage area of ​​the medical imaging system (e.g., for later reference in diagnosis and / or treatment). In one example, the UI's image storage area panel (e.g., Figure 4A The image storage panel 410 can display thumbnail images of images saved to the image storage area, thus, when a medical image is saved to the image storage area, a thumbnail of the medical image appears on the image storage panel. Method 600 ends after the medical image is saved to the image storage area.

[0116] Now for reference Figure 7 A flowchart of an exemplary method 700 is shown, which is used for a second configuration of a medical imaging system (such as, via a set of controls on a TEE control handle) with a TEE control handle. Figure 1 The UI interaction of the ultrasound imaging system 100. A second configuration of this set of controls may not include forward and backward navigation buttons, but may include a radial touchpad and selection buttons, such as... Figure 3C A second exemplary configuration of a set of controls 352 includes a radial touchpad 354 and a selection button 356. The TEE control handle may also include a first control wheel and a second control wheel, such as... Figure 2A The first control wheel 202 and the second control wheel 204. In one embodiment, during an ultrasound examination of a patient by an operator of a medical imaging system, the operations of method 700 can be stored in a non-transitory memory and executed by a processor, such as... Figure 1 The medical imaging system 100 includes a non-transitory memory 120 and a processor 116.

[0117] Steps 702, 704, 705, 706, and 707 of method 700 are the same as steps 602, 604, 605, 606, and 607 of method 6 described above, wherein the ultrasound image and related information, along with the shortcut control element selection in the shortcut key panel, the image storage area panel, and the virtual trackball interface panel, are displayed in the UI described above, with the focus of the UI placed on the default shortcut key in the shortcut key panel. Method 700 updates the ultrasound image and related information until user input is received.

[0118] At 708, method 700 includes determining whether a selection button has been selected (e.g., Figure 3C (Select button 356). If it is determined at 708 that the selection button has not been selected, method 700 continues to 710. At 710, method 700 includes determining whether the operator is scrolling in the first rotational direction using the radial touchpad (e.g., by dragging the operator's thumb or finger around the radial touchpad). If it is determined at 710 that the operator is scrolling in the first rotational direction using the radial touchpad, method 700 continues to 712. At 712, method 700 includes moving the focus of the UI in the first direction from an element of the panel where the focus is located (e.g., shortcuts in the shortcut panel, thumbnails in the image storage panel, control elements of the virtual trackball interface panel, etc.) to an adjacent element of that panel. For example, if the operator scrolls clockwise using the touchpad when the focus is on a shortcut in the shortcut panel, the focus can move clockwise (e.g., from left to right along a single row or top row of shortcuts in the shortcut panel, and from right to left along the bottom row of shortcuts in the shortcut panel).

[0119] If it is determined at 710 that the forward navigation button has not been selected, then method 700 infers that the backward navigation button has been selected, and method 700 continues to 714. At 714, method 700 includes moving the focus of the UI from the panel where the focus is located to an adjacent panel in the opposite direction (e.g., the direction opposite to forward). For example, if the operator is scrolling counterclockwise with the touchpad while the focus is on a shortcut key in the shortcut key panel, the focus can move counterclockwise from the shortcut key panel to an adjacent panel (e.g., the virtual trackball panel). If the operator continues to scroll counterclockwise with the touchpad while the focus is on the virtual trackball panel, the focus can move from the virtual trackball panel to the image storage area panel (e.g., the adjacent panel in the counterclockwise direction). Thus, by using the radial touchpad in one direction (e.g., a first rotation direction), the operator can scroll through the UI's controls or display elements arranged within the display panel, and by using the radial touchpad in the opposite direction (e.g., a second opposite rotation direction), the operator can scroll through the UI's display panel.

[0120] If it is determined at 708 that the selection button has been selected, method 700 continues to 716. At 716, method 700 includes determining whether the selection button has been double-clicked. If it is determined at 716 that the selection button has not been double-clicked, method 700 continues to 718. At 718, method 700 includes selecting the element of the currently focused object (e.g., a shortcut key in the shortcut panel, a thumbnail image in the image storage panel, or a control element in the virtual trackball panel), and method 700 ends. If it is determined at 716 that the selection button has been double-clicked, method 700 continues to 720. At 720, method 700 includes saving the medical image displayed in the UI to the image storage area of ​​the medical imaging system. After saving the medical image to the image storage area, method 700 ends.

[0121] Now for reference Figure 8 A flowchart of an exemplary method 800 is shown, which is used to control the first and second control wheels (such as, via the TEE control handle of a medical imaging system) Figure 2A The handheld ultrasound device 200 with its first control wheel 202 and second control wheel 204 is connected to a medical imaging system (such as...). Figure 1 The UI interaction of the ultrasound imaging system 100. In one embodiment, during an ultrasound examination of a patient by an operator of the medical imaging system, the operations of method 800 can be stored in non-transitory memory and executed by a processor, such as... Figure 1 The medical imaging system 100 includes a nontransitory memory 120 and a processor 116.

[0122] At 802, method 800 includes a display device of a medical imaging system (e.g., Figure 1 The medical imaging system 100 displays medical images and related information on its display device 118. In one example, the medical image may be an ultrasound image. For example, the medical image may be a 2D, 3D, or 4D ultrasound image, a Doppler ultrasound image, or any other type of ultrasound image. In some examples, the medical image may be a single frame of a sequence of medical images acquired in real time from the patient, for example, during an ultrasound examination performed by a clinician. In other examples, the medical image may be an X-ray image, a CT image, an MRI image, a visible light image, or another type of medical image that includes anatomical features of interest.

[0123] In one example, as referenced above Figure 4A Graphical UI 400 and Figure 6The medical images and related information are displayed in the UI as described in method 600, wherein the medical images are generated by a probe disposed at the distal end of an insertion tube connected to the TEE control handle of the medical imaging system. When the operator of the medical imaging system begins an examination, the operator can use the TEE control handle to advance the probe into the patient's body cavity (e.g., esophagus, etc.). As the probe advances into the body cavity, the ultrasound images displayed in the UI can be updated based on the movement of the probe and / or the adjustment of the probe's position and / or orientation via the first and second control wheels of the TEE control handle. For example, the operator can advance the probe into the patient's esophagus toward the ROI of the patient's heart located behind the esophagus via the TEE control handle. To navigate through one or more bends in the esophagus, as the operator advances the probe into the esophagus, the operator can bend the articulated neck mechanism that connects the probe to the insertion tube to angle the probe toward the patient's left or right side via the first control wheel and toward the patient's front or back side via the second control wheel. When the probe reaches a position in the esophagus close to the ROI, the operator can adjust the rotation position of the first and / or second control wheels to tilt the probe to the left or right and / or front or back of the patient, respectively. This aligns the probe lens so that the ultrasound beam is deflected away from the target object of the ROI at a vertical or near-vertical angle, thereby acquiring a medical image. Relevant information may be text information simultaneously displayed on the display device, which may include the frequency of the signal generated by the ultrasound probe, the depth of the ultrasound probe relative to the ROI, the time elapsed since the start of the examination, etc., as a list of non-limiting examples.

[0124] At 803, method 800 includes displaying a control wheel panel in the UI, such as Figure 4A The graphical user interface 400 has a control wheel panel 414. In one example, the control wheel panel could be... Figure 5A A non-limiting example of a graphic control wheel display element 500 includes a first graphic control wheel display and a second graphic control wheel display. The first graphic control wheel display shows the rotational position of a first control wheel of the TEE control handle, and the second graphic control wheel display shows the rotational position of a second control wheel of the TEE control handle, as referenced above. Figures 5A to 5D The first and second graphic control wheel displays can also indicate the control wheel lock of the TEE control handle (e.g., ...). Figure 2A The state of the control wheel lock 206 of the handheld ultrasonic device.

[0125] At 804, method 800 includes determining whether user input has been received by the first control wheel, second control wheel, and / or control wheel lock of the TEE control handle (e.g., by adjusting the rotational position of the first control wheel, second control wheel, or control wheel lock). If it is determined at 804 that user input has not been received from the first control wheel, second control wheel, and / or control wheel lock, method 800 continues to 805. At 805, method 800 includes updating any relevant information in the ultrasound image generated by the probe and / or the UI until it is determined that user input has been received. If it is determined at 805 that user input has been received from the first control wheel, second control wheel, and / or control wheel lock, method 800 continues to 806.

[0126] At method 806, method 800 includes determining whether the rotational position of the first control wheel has been adjusted by the operator. For example, the operator can adjust the first control wheel by manipulating it with their fingers and thumb to rotate it in a first direction (e.g., clockwise) or in a second opposite direction (e.g., counterclockwise). If it is determined at 806 that the rotational position of the first control wheel has been adjusted, method 800 proceeds to 808. At 808, method 800 includes adjusting a first deflection indicator displayed by the first graphical control wheel in the UI. In one example, the first deflection indicator displayed by the first graphical control wheel is a reference... Figure 5A , Figure 5B and Figure 5C The deflection indicator 506 described herein is a non-limiting example. In another example, the first deflection indicator displayed by the first graphical control wheel is a reference... Figure 5D Non-limiting examples of the pointer indicator 566. In one example, adjusting the first deflection indicator includes adjusting the rotational position of the first deflection indicator around the circumference displayed on the first graphic control wheel in a first direction (e.g., clockwise) or a second opposite direction (e.g., counterclockwise), such that the indication point of the first deflection indicator extends outward from the center point of the first graphic control wheel display (e.g., as described above regarding...). Figure 5A , Figure 5B , Figure 5C and Figure 5D The above).

[0127] If it is determined at 806 that the first control wheel has not been adjusted, or once the first deflection indicator displayed by the first graphical control wheel in the UI has been adjusted at 808, method 800 continues to 810. At 810, method 800 includes determining whether the rotational position of the second control wheel has been adjusted by the operator (e.g., in a first direction or a second opposite direction). If it is determined at 810 that the rotational position of the second control wheel has been adjusted, method 800 continues to 812. At 812, method 800 includes adjusting the second deflection indicator displayed by the second graphical control wheel in the UI. In one example, the second deflection indicator displayed by the second graphical control wheel is a reference... Figure 5A , Figure 5B and Figure 5C The deflection indicator 514 described herein is a non-limiting example. In another example, the second deflection indicator displayed by the second graphical control wheel is a reference... Figure 5D Non-limiting examples of the pointer indicator 576. In one example, adjusting the second deflection indicator includes adjusting the rotational position of the second deflection indicator around the circumference of the second graphic control wheel display in a first direction or a second opposite direction, such that the indicator point of the deflection indicator extends outward from the center point of the second graphic control wheel display (e.g., as described above regarding...). Figure 5A , Figure 5B , Figure 5C and Figure 5D The above).

[0128] If it is determined at 810 that the second control wheel has not been adjusted, or once the second deflection indicator displayed on the second graphical control wheel in the UI has been adjusted at 812, method 800 continues to 814. At 814, method 800 includes determining whether the first control wheel has been locked by a control wheel lock. If it is determined at 814 that the first control wheel has been locked, method 800 continues to 816. At 816, method 800 includes adjusting the lock indicator displayed on the first graphical control wheel in the UI (e.g., ...). Figure 5A , Figure 5B , Figure 5C and Figure 5D (Lock indicator 508). For example, adjusting the lock indicator displayed on the first graphical control wheel in the UI may include changing the graphical representation of the padlock to that of an unlocked padlock, or changing the color of the graphical representation of the padlock, etc.

[0129] If it is determined at 814 that the first control wheel is not locked, or once the lock indicator displayed on the first graphical control wheel in the UI is adjusted at 816, method 800 continues to 818. At 818, method 800 includes determining whether the second control wheel has been locked by a control wheel lock. If it is determined at 818 that the second control wheel has been locked, method 800 continues to 820. At 820, method 800 includes adjusting the lock indicator displayed on the second graphical control wheel in the UI (e.g., Figure 5A, ...). Figure 5B and Figure 5C Lock indicator 518).

[0130] At 822, method 800 includes adjusting and / or updating a first text label displayed by a first graphical control wheel, and adjusting and / or updating a second text label displayed by a second graphical control wheel in the UI. For example, as referenced above. Figure 5A , 5B As described in 5C, the first text label and / or the second text label may respectively indicate that the first control wheel and / or the second control wheel have been locked, or the first text label and / or the second text label may respectively indicate the deflection degree of the first control wheel and / or the second control wheel, or the first text label and / or the second text label may include different indications for the operator (e.g., a higher deflection degree, etc.). It should be understood that the examples provided herein are for illustrative purposes only, and other types of graphic or text indicators may be included without departing from the scope of this disclosure. Method 800 ends after adjusting and / or updating the text on the first control wheel display and the second control display.

[0131] Therefore, by allowing operators of medical imaging systems to navigate through multiple shortcut control elements visible in one or more display panels of the medical imaging system's UI via one or more controls arranged on the control handle of the medical imaging system (such as the TEE control handle described above), and to select the shortcut control element associated with the desired function, the efficiency of operator performance of examinations can be improved. Actions and functions available via the medical imaging system's menus can be accessed more quickly and directly, and operators can seamlessly switch between modes of the medical imaging system without having to look at the control handle or adjust its position and / or orientation to view one or more controls. Providing visual cues in the UI indicating functions being performed or that can be performed during the examination reduces operator cognitive load (e.g., because they do not have to remember the function mapping of one or more buttons on the control handle to one or more functions). It also reduces operator distraction and can reduce the time spent on the examination, thereby improving patient experience and / or examination results. Furthermore, by facilitating more efficient interaction with the UI, the mechanical buttons on the control handle can be replaced with capacitive sensor buttons, thereby reducing the cost of the medical imaging system and / or ultrasound probe. In addition, by adjusting the UI display elements to provide visual indication of the deflection angle of the probe of the medical imaging system in one or more free planes, instead of relying on the operator to sense the configuration of the first and second control wheels with the operator's thumb and fingers, the operator's cognitive load can be further reduced and the operator's efficiency can be improved.

[0132] The technology that allows operators to navigate between multiple display panels in the UI and access one or more quick control elements mapped to one or more functions of the medical imaging system can improve operator efficiency while reducing operator cognitive load and / or distraction during examinations.

[0133] This disclosure also provides support for a method for a medical imaging system, the method comprising: adjusting the focus of the medical imaging system's UI among a plurality of graphical control elements displayed in a user interface (UI) in response to an operator of the medical imaging system adjusting one or more controls disposed on a control handle of a handheld ultrasound device of the medical imaging system; and performing an action of the medical imaging system associated with the selected graphical control element in response to the operator selecting a graphical control element at the focus position of the UI via the one or more controls. In a first example of the method, the one or more controls include a selection button, and at least one of a forward directional button, a backward directional button, and a radial touchpad and a single-axis touchpad. In a second example of the method, the second example optionally includes the first example, wherein at least one of the selection button, the forward directional button, and the backward directional button is a capacitive sensor button, a resistive touch button, a shorting pad button, and a force-sensing resistor. In a third example of the method, the third example optionally includes one or both of the first and second examples. Adjusting the one or more controls arranged on the control handle includes one of the following operations: the operator selects a forward directional button or a backward directional button; the operator drags a thumb or finger around the radial touchpad in a first rotational direction or a second opposite rotational direction; and the operator drags a thumb or finger along the single-axis touchpad in a first direction or a second opposite direction. In a fourth example of the method, the fourth example optionally includes one or more examples or each of the first to third examples. The plurality of graphical control elements are arranged in a plurality of display panels of the UI, and the method further includes: adjusting the focus of the UI among the plurality of graphical control elements displayed in the selected display panel in response to the operator adjusting the one or more controls arranged on the control handle in a first manner; selecting a graphical control element of the selected display panel in response to the operator selecting a selection button while adjusting the one or more controls in the first manner; adjusting the focus of the UI among the plurality of display panels in response to the operator adjusting the one or more controls arranged on the control handle in a second manner; and selecting a display panel in response to the operator selecting a selection button while adjusting the one or more controls in the second manner.In a fifth example of the method, optionally including one or more examples from the first to fourth examples, adjusting the one or more controls arranged on the control handle in a first manner includes one of the following operations: the operator selects a first directional button, which is one of a backward directional button and a forward directional button, and the operator drags their thumb or finger around the touchpad in a first direction; and adjusting the one or more controls arranged on the control handle in a second manner includes one of the following operations: the operator selects a second directional button, which indicates a direction opposite to the first directional button, and the operator drags their thumb or finger around the touchpad in a second opposite direction. In a sixth example of the method, optionally including one or more examples from the first to fifth examples, the plurality of graphical control elements includes one or more shortcut control elements, and the action of the medical imaging system associated with each of the one or more shortcut control elements is a function accessible via the menu of the medical imaging system. In a seventh example of the method, which optionally includes one or more examples or each of the first to sixth examples, one of the following can be activated via a function accessible through the menu of the medical imaging system: a 2D operation mode of the medical imaging system, a 3D operation mode of the medical imaging system, a 4D operation mode of the medical imaging system, a color operation mode of the medical imaging system, and a Doppler operation mode of the medical imaging system. In an eighth example of the method, which optionally includes one or more examples or each of the first to seventh examples, a predetermined shortcut control element is selected in response to a double-click of a selection button by the operator. In a ninth example of the method, which optionally includes one or more examples or each of the first to eighth examples, the predetermined shortcut control element saves the medical image displayed in the UI to the image storage area of ​​the medical imaging system, and displays a thumbnail of the medical image in the image storage area panel of the UI. In a tenth example of the method, which optionally includes one or more examples or each of the first to ninth examples, the handheld ultrasound device is a transesophageal echocardiography (TEE) device, and the control handle is a TEE control handle.

[0134] This disclosure also provides support for a medical imaging system comprising: an ultrasound probe coupled to a transesophageal echocardiography (TEE) control handle via an insertion tube having an articulated neck mechanism; a display device; and a processor communicatively coupled to the TEE control handle and the display device, and including instructions stored in a non-transitory memory that, when executed, cause the processor to: adjust a first set of controls arranged on the TEE control handle in response to an operator of the medical imaging system by: bending the articulated neck mechanism to adjust the position and / or orientation of the ultrasound probe; adjusting graphical display elements of the user interface (UI) of the medical imaging system to indicate the adjusted position and / or orientation of the ultrasound probe; adjusting the focus of the UI among multiple display panels of the UI in response to an operator adjusting a second set of controls arranged on the TEE control handle in a first manner; adjusting the focus of the UI among multiple control elements of the UI in a second manner in response to an operator adjusting the second set of controls in a second manner, indicating the position of the adjusted focus of the UI in the UI; and performing a function of the medical imaging system associated with the control element in response to an operator selecting a control element at the position of the adjusted focus via the second set of controls. In a first example of the system, the first set of controls includes a first control wheel and a second control wheel arranged on a first side of the TEE control handle, and wherein adjusting the graphical display elements of the medical imaging system's UI to indicate the adjusted position and / or orientation of the ultrasound probe includes: indicating the rotational position of the first control wheel on the graphical representation of the first control wheel; and indicating the rotational position of the second control wheel on the graphical representation of the second control wheel. In a second example of the system, the second example optionally includes the first example, with additional instructions stored in non-transitory memory. These additional instructions, when executed, cause the processor to indicate the wheel lock state of the first and second control wheels, the wheel lock state being one of the following: the first control wheel is locked and the second control wheel is unlocked, the first control wheel is unlocked and the second control wheel is locked, the first control wheel is locked and the second control wheel is locked, and the first control wheel is unlocked and the second control wheel is unlocked. In a third example of the system, the third example optionally includes one or both of the first and second examples, and indicating the state of the wheel locks of the first and second control wheels includes at least one of the following operations: in response to the first control wheel being locked, displaying a graphic image of a padlock in a locked configuration on the graphic representation of the first control wheel; in response to the first control wheel being unlocked, displaying a graphic image of a padlock in an unlocked configuration on the graphic representation of the first control wheel; in response to the second control wheel being locked, displaying a graphic image of a padlock in a locked configuration on the graphic representation of the second control wheel; in response to the second control wheel being unlocked, displaying a graphic image of a padlock in an unlocked configuration on the graphic representation of the second control wheel.In a fourth example of the system, optionally including one or more examples of the first to third examples, additional instructions are stored in non-transitory memory. When executed, these additional instructions cause the processor to display text on one of the graphical representations of the first and second control wheels, wherein the text is one of an indication that the corresponding control wheel is locked and the degree of deflection of the ultrasound probe in the free plane corresponding to the corresponding control wheel. In a fifth example of the system, optionally including one or more examples of the first to fourth examples, adjusting the graphical display elements of the medical imaging system's UI to indicate the adjusted position and / or orientation of the ultrasound probe includes: adjusting a first rotational position indicator on the graphical representation of the first control wheel to indicate the degree of deflection of the probe in the first free plane; and adjusting a second rotational position indicator on the graphical representation of the second control wheel to indicate the degree of deflection of the probe in the second free plane. In a sixth example of the system, optionally including one or more examples from the first to the fifth examples, the second set of controls includes a selection button and a radial touchpad on a second side of the TEE control handle, the second side being different from the first side. Adjusting the second set of controls in a first manner includes scrolling with the radial touchpad in a first rotational direction, and adjusting the second set of controls in a second manner includes scrolling with the radial touchpad in a second rotational direction, the second rotational direction being opposite to the first rotational direction. In a seventh example of the system, optionally including one or more examples from the first to the sixth examples, the second set of controls includes a capacitive sensor selection button, a capacitive sensor forward directional button, and a capacitive sensor backward directional button arranged on a second side of the TEE control handle, the second side being different from the first side. Adjusting the second set of controls in a first manner includes selecting a first directional button, the first directional button being one of the backward directional button and the forward directional button, and adjusting the second set of controls in a second manner includes selecting a second directional button, the second directional button being one of the backward directional button and the forward directional button and different from the first directional button.

[0135] This disclosure also provides support for a method for a medical imaging system, the method comprising: in response to an operator of the medical imaging system selecting one or more shortcut control elements disposed on a control handle of a handheld ultrasound device of the medical imaging system, highlighting the selected shortcut control element in the graphical user interface (UI) of the medical imaging system, and in response to a second selection of the selected shortcut control element by the operator, performing a function of the medical imaging system associated with the selected shortcut control element.

[0136] When describing elements of various embodiments of this disclosure, the terms “an,” “a,” and “the” are intended to refer to one or more of these elements. The terms “first,” “second,” etc., do not indicate any order, quantity, or importance, but are used to distinguish one element from another. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may exist in addition to the listed elements. As used herein, the terms “connected to,” “linked to,” etc., indicate that an object (e.g., a material, element, structure, component, etc.) may be connected to or linked to another object, regardless of whether the object is directly connected to or linked to the other object, or whether one or more intervening objects exist between the object and the other. Furthermore, it should be understood that references to “an embodiment” or “an embodiment” of this disclosure are not intended to be construed as excluding the existence of additional embodiments that also include the referenced features.

[0137] In addition to any modifications previously indicated, those skilled in the art can devise many other variations and alternative arrangements without departing from the spirit and scope of this description, and the appended claims are intended to cover such modifications and arrangements. Therefore, although the information has been described in particular and detail above in conjunction with what is currently considered to be the most practical and preferred aspects, it will be apparent to those skilled in the art that many modifications can be made without departing from the principles and concepts set forth herein, including but not limited to changes in form, function, mode of operation, and use. Likewise, as used herein, in all respects, examples and embodiments are intended to be illustrative only and should not be construed as restrictive in any way.

Claims

1. A method for a medical imaging system, the method comprising: In response to the operator of the medical imaging system adjusting a first set of controls arranged on the control handle of the handheld ultrasound device of the medical imaging system, the position and / or orientation of the probe of the handheld ultrasound device is adjusted. The first set of controls includes a first control wheel and a second control wheel. The first control wheel is configured to adjust the deflection of the neck assembly, including the probe, in a first free plane, and the second control wheel is configured to adjust the deflection of the neck assembly in a second free plane. Adjust one or more graphical display elements of the user interface (UI) of the medical imaging system to indicate the adjusted position and / or orientation of the probe, the one or more graphical display elements including a representation of the first control wheel indicating the rotational position of the first control wheel and a representation of the second control wheel indicating the rotational position of the second control wheel; In response to the operator of the medical imaging system adjusting a second set of controls arranged on the control handle of the handheld ultrasound device of the medical imaging system, the focus of the UI of the medical imaging system is adjusted among multiple graphical control elements displayed in the user interface (UI); as well as In response to the operator selecting a graphical control element at the focus position of the plurality of graphical control elements in the UI via the second set of controls, the action of the medical imaging system associated with the selected graphical control element is executed.

2. The method of claim 1, wherein the second set of controls includes a selection button, and at least one of a forward directional button, a backward directional button, a radial touchpad, and a single-axis touchpad.

3. The method of claim 2, wherein at least one of the selection button, the forward directional button, and the backward directional button is one of a capacitive sensor button, a resistive touch button, a shorting pad button, and a force-sensing resistor.

4. The method of claim 2, wherein adjusting the second set of controls arranged on the control handle comprises one of the following operations: The operator selects either the forward navigation button or the backward navigation button; The operator drags their thumb or finger around the radial touchpad in a first rotational direction or a second opposite rotational direction; and The operator drags their thumb or finger along the single-axis touchpad in a first direction or a second opposite direction.

5. The method of claim 2, wherein the plurality of graphical control elements are arranged in a plurality of display panels of the UI, and the method further comprises: In response to the operator adjusting the second set of controls arranged on the control handle in a first manner, the focus of the UI is adjusted among the plurality of graphical control elements displayed in the selected display panel; In response to the operator selecting the selection button while adjusting the second group of controls in the first manner, the operator selects a graphical control element of the selected display panel; In response to the operator adjusting the second set of controls arranged on the control handle in a second manner, the focus of the UI is adjusted among the plurality of display panels; as well as In response to the operator selecting the selection button while adjusting the second group of controls in the second manner, the display panel is selected.

6. The method according to claim 5, wherein: Adjusting the second set of controls arranged on the control handle in the first manner includes one of the following operations: The operator selects a first directional button, which is one of the backward directional button and the forward directional button; and The operator drags their thumb or finger around the touchpad in a first direction; and Adjusting the second set of controls arranged on the control handle in the second manner includes one of the following operations: The operator selects a second directional button, which indicates the opposite direction to the first directional button; and The operator drags the thumb or the finger around the touchpad in a second opposite direction.

7. The method of claim 2, wherein the plurality of graphical control elements includes one or more shortcut control elements, and the action of the medical imaging system associated with each of the one or more shortcut control elements is a function accessible via a menu of the medical imaging system.

8. The method of claim 7, wherein the function accessible via the menu of the medical imaging system activates one of the following: The 2D operation mode of the medical imaging system; The 3D operation mode of the medical imaging system; The 4D operation mode of the medical imaging system; The color operation mode of the medical imaging system; and The Doppler operating mode of the medical imaging system.

9. The method of claim 7, wherein in response to the operator double-clicking the selection button, a predetermined shortcut control element is selected from the one or more shortcut control elements.

10. The method of claim 9, wherein the predetermined shortcut control element saves the medical image displayed in the UI to the image storage area of ​​the medical imaging system, and displays a thumbnail of the medical image in the image storage area panel of the UI.

11. The method of claim 1, wherein the handheld ultrasound device is a transesophageal echocardiography (TEE) device, and the control handle is a TEE control handle.

12. A medical imaging system, comprising: An ultrasound probe, which is connected to a transesophageal echocardiography (TEE) control handle via an insertion tube having a hinged neck mechanism. Display devices; A processor, communicatively coupled to the TEE control handle and the display device, and including instructions stored in non-transitory memory, which, when executed, cause the processor to: In response to an operator adjusting a first set of controls on the TEE control handle of the medical imaging system, the first set of controls includes a first control wheel and a second control wheel arranged on a first side of the TEE control handle. The first control wheel is configured to adjust the deflection of the articulated neck mechanism in a first free plane, and the second control wheel is configured to adjust the deflection of the articulated neck mechanism in a second free plane. The hinged neck mechanism is bent to adjust the position and / or orientation of the ultrasound probe; Adjust one or more graphical display elements of the user interface (UI) of the medical imaging system to indicate the adjusted position and / or orientation of the ultrasound probe, the one or more graphical display elements including a representation of the first control wheel indicating the rotational position of the first control wheel and a representation of the second control wheel indicating the rotational position of the second control wheel; In response to the operator adjusting the second set of controls arranged on the TEE control handle in a first manner, the focus of the UI is adjusted among the multiple display panels of the UI; In response to the operator adjusting the second set of controls in a second manner, the focus of the UI is adjusted among the multiple control elements of the UI; The UI indicates the position of the adjusted focus. In response to the operator selecting a control element at the location of the adjusted focus via the second set of controls, the function of the medical imaging system associated with the control element is executed.

13. The system of claim 12, wherein adjusting one or more graphical display elements of the UI of the medical imaging system to indicate the adjusted position and / or orientation of the ultrasound probe comprises: The rotational position of the first control wheel is indicated on the graphical representation of the first control wheel; as well as The rotational position of the second control wheel is indicated on the graphical representation of the second control wheel.

14. The system of claim 13, wherein additional instructions are stored in the non-transitory memory, said additional instructions, when executed, cause the processor to indicate the state of the wheel locks of the first control wheel and the second control wheel, said state of the wheel locks being one of the following: The first control wheel is locked and the second control wheel is unlocked; The first control wheel is unlocked and the second control wheel is locked; The first control wheel is locked and the second control wheel is locked; as well as The first control wheel is unlocked and the second control wheel is unlocked.

15. The system of claim 14, wherein indicating the state of the wheel lock of the first control wheel and the second control wheel includes at least one of the following operations: In response to the first control wheel being locked, a graphic image of a padlock in the locked configuration is displayed on the graphic representation of the first control wheel; In response to the first control wheel being unlocked, a graphic image of the padlock in the unlocked configuration is displayed on the graphic representation of the first control wheel; In response to the second control wheel being locked, a graphic image of the padlock in the locked configuration is displayed on the graphic representation of the second control wheel; In response to the second control wheel being unlocked, a graphic image of the padlock in the unlocked configuration is displayed on the graphic representation of the second control wheel.

16. The system of claim 13, wherein additional instructions are stored in the non-transitory memory, which, when executed, cause the processor to display text on one of the graphical representations of the first control wheel and the second control wheel, wherein the text is one of an indication that the corresponding control wheel is locked and a deflection degree of the ultrasound probe in a free plane corresponding to the corresponding control wheel.

17. The system of claim 12, wherein adjusting the graphical display elements of the UI of the medical imaging system to indicate the adjusted position and / or orientation of the ultrasound probe comprises: Adjust the first rotation position indicator of the first control wheel in a graphical representation to indicate the degree of deflection of the probe in the first free plane; as well as Adjust the second rotation position indicator on the graphic representation of the second control wheel to indicate the degree of deflection of the probe in the second free plane.

18. The system of claim 12, wherein the second set of controls includes a selection button and a radial touchpad on a second side of the TEE control handle, the second side being different from the first side, and adjusting the second set of controls in the first manner includes scrolling with the radial touchpad in a first rotational direction, and adjusting the second set of controls in the second manner includes scrolling with the radial touchpad in a second rotational direction, the second rotational direction being opposite to the first rotational direction.

19. The system of claim 12, wherein the second set of controls includes a capacitive sensor selection button, a capacitive sensor forward orientation button, and a capacitive sensor backward orientation button disposed on a second side of the TEE control handle, the second side being different from the first side, and adjusting the second set of controls in the first manner includes selecting a first orientation button, the first orientation button being one of the backward orientation button and the forward orientation button, and adjusting the second set of controls in the second manner includes selecting a second orientation button, the second orientation button being one of the backward orientation button and the forward orientation button and being different from the first orientation button.

Images