User input system and method for a computer-assisted medical system

Abstract

A user input system guides a computer-assisted medical system comprising a set of controls, including a first control and a second control configured to receive user input for operation in a manipulation input mode. In this mode, user input to the first control manipulates a first manipulator of the computer-assisted medical system, and user input to the second control manipulates a second manipulator. The user input system also guides the computer-assisted medical system for operation in a mixed input mode, where user input to the first control manipulates the first manipulator, and user input to the second control adjusts parameter settings associated with the medical procedure.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 039,753, filed June 16, 2020, the contents of which are incorporated herein by reference in their entirety. Background Technology

[0003] Various technologies, including computing, robotics, medical technology, and extended reality (e.g., augmented reality, virtual reality), enable users (such as surgeons) to perform and be trained to perform a wide range of medical procedures. For example, users can perform and be trained to perform minimally invasive medical procedures, such as computer-assisted medical procedures, in clinical settings (e.g., procedures performed on living or animal patients), non-clinical settings (e.g., procedures performed on human or animal cadavers, tissues removed from human or animal anatomy), and training settings (e.g., procedures performed on physical anatomical training models, virtual anatomical models in extended reality environments), etc.

[0004] In any program with such a setup, when a user controls the instruments of a computer-assisted medical system to perform a procedure, the user can view images of anatomical spaces related to the body (e.g., internal body regions). The user can control the instruments using various input mechanisms on or connected to the computer-assisted medical system. These input mechanisms can affect the efficiency and / or effectiveness of the procedure the user is able to perform. Summary of the Invention

[0005] The following description presents a brief overview of one or more aspects of the systems and methods described herein. This overview is not a comprehensive review of all anticipated aspects and is neither intended to identify key or important elements of all aspects, nor to depict the scope of any or all aspects. Its sole purpose is to present one or more aspects of the systems and methods described herein as a prelude to the detailed description presented below.

[0006] An illustrative system includes: a memory storing instructions; and a processor communicatively coupled to the memory and configured to execute instructions to: direct a computer-assisted medical system including a set of controls, the set of controls including a first control and a second control configured to receive user input for operation in a manipulation input mode, wherein user input to the first control manipulates a first manipulator of the computer-assisted medical system, and user input to the second control manipulates a second manipulator of the computer-assisted medical system; and direct the computer-assisted medical system to operate in a mixed input mode, wherein user input to the first control manipulates the first manipulator, and user input to the second control adjusts parameter settings associated with a medical procedure.

[0007] An illustrative method includes a processor (e.g., a processor of a user input system) that directs a computer-assisted medical system comprising a set of controls, the set of controls including a first control and a second control configured to receive user input for operation in a manipulation input mode, wherein user input to the first control manipulates a first manipulator of the computer-assisted medical system, and user input to the second control manipulates a second manipulator of the computer-assisted medical system; and directs the computer-assisted medical system for operation in a mixed input mode, wherein user input to the first control manipulates the first manipulator, and user input to the second control adjusts parameter settings associated with a medical procedure.

[0008] An illustrative computer-readable medium includes instructions that, when executed by a processor, cause the processor to direct a computer-assisted medical system comprising a set of controls, the set of controls including a first control and a second control configured to receive user input for operation in a manipulation input mode, wherein user input to the first control manipulates a first manipulator of the computer-assisted medical system, and user input to the second control manipulates a second manipulator of the computer-assisted medical system; and directs the computer-assisted medical system to operate in a mixed input mode, wherein user input to the first control manipulates the first manipulator, and user input to the second control adjusts parameter settings associated with a medical procedure.

[0009] An illustrative system includes: a memory storing instructions, and a processor communicatively connected to the memory and configured to execute the instructions to: direct a computer-assisted medical system including a set of controls, the set of controls including a first control and a second control configured to receive user input for operation in a manipulation input mode, wherein the first control is configured to receive a first input to manipulate a first manipulator of the computer-assisted medical system, and the second control is configured to receive a second input to manipulate a second manipulator of the computer-assisted medical system; and direct the computer-assisted medical system to operation in a mixed input mode, wherein the first control is configured to receive a third input to manipulate the first manipulator, and the second control is configured to receive a fourth input to adjust parameter settings associated with a medical procedure.

[0010] An illustrative method includes a processor (e.g., a processor of a user input system) that directs a computer-assisted medical system comprising a set of controls, the set of controls including a first control and a second control configured to receive user input for operation in a manipulation input mode, wherein the first control is configured to receive a first input to manipulate a first manipulator of the computer-assisted medical system, and the second control is configured to receive a second input to manipulate a second manipulator of the computer-assisted medical system; and directs the computer-assisted medical system for operation in a mixed input mode, wherein the first control is configured to receive a third input to manipulate the first manipulator, and the second control is configured to receive a fourth input to adjust parameter settings associated with a medical procedure.

[0011] An illustrative computer-readable medium includes instructions that, when executed by a processor, cause the processor to direct a computer-assisted medical system comprising a set of controls, the set of controls including a first control and a second control configured to receive user input for operation in a manipulation input mode, wherein the first control is configured to receive a first input to manipulate a first manipulator of the computer-assisted medical system, and the second control is configured to receive a second input to manipulate a second manipulator of the computer-assisted medical system; and directs the computer-assisted medical system to operation in a mixed input mode, wherein the first control is configured to receive a third input to manipulate the first manipulator, and the second control is configured to receive a fourth input to adjust parameter settings associated with a medical procedure. Attached Figure Description

[0012] The accompanying drawings illustrate various embodiments and are part of the specification. The illustrated embodiments are merely examples and do not limit the scope of this disclosure. Throughout the drawings, the same or similar reference numerals denote the same or similar elements.

[0013] Figure 1 An example user input system based on the principles described in this article is illustrated.

[0014] Figure 2A and 2B An example configuration of a user input system for a computer-aided medical system based on the principles described herein is illustrated.

[0015] Figure 3 Example images illustrating a computer-aided medical system based on the principles described in this article.

[0016] Figure 4A-5B An example interface for the user input system of a computer-aided medical system based on the principles described herein is presented.

[0017] Figure 6This paper illustrates an example user input method for a computer-aided medical system based on the principles described herein.

[0018] Figure 7 This paper illustrates an example user input method for a computer-aided medical system based on the principles described herein.

[0019] Figure 8 This paper illustrates an example user input method for a computer-aided medical system based on the principles described herein.

[0020] Figure 9 An example computer-aided medical system based on the principles described in this article is illustrated.

[0021] Figure 10 An example computing device based on the principles described herein is illustrated. Detailed Implementation

[0022] This document describes a user input system and method for computer-assisted medical systems. During a computer-assisted medical procedure, a user (e.g., a surgeon) can control (e.g., remotely operate) instruments via a set of controls that receive input configured to manipulate the instruments through the computer-assisted medical system. For example, user input received through this set of controls can cause elements of the computer-assisted medical system, such as manipulators, to be manipulated or controlled in a manner that manipulates instruments connected to those manipulators.

[0023] This set of controls can be configured to operate in a manipulation input mode, where user input to the controls (e.g., through user manipulation of the controls) can be translated into manipulation of instruments and / or manipulators by the computer-assisted medical system. As described herein, the set of controls can also be configured to operate in a mixed input mode, where user input to one control causes the computer-assisted medical system to manipulate instruments and / or manipulators, while user input to another control causes one or more parameter settings associated with the computer-assisted medical procedure to be adjusted. For example, when operating in mixed input mode, user input to one control can be translated into manipulation of instruments and / or manipulators (e.g., movement), while user input to another control can be translated into adjustment of parameter settings associated with the medical procedure. An example of such parameter settings is described herein.

[0024] The systems and methods described herein offer a variety of advantages and benefits. For example, the systems and methods described herein provide a hybrid input mode in which a set of user controls is configured to receive input for manipulating an instrument while simultaneously adjusting parameter settings associated with a medical procedure, such as instrument parameter settings. The hybrid input mode improves the efficiency and / or effectiveness of receiving input from computer-assisted medical systems compared to conventional user input systems used in computer-assisted medical systems (e.g., conventional user input systems where instrument operation and parameter setting adjustments must be performed at different times and / or in independent input modes, or where independent input mechanisms must be used instead of a set of controls for manipulating the instrument to adjust parameter settings, etc.). This improvement in the efficiency and / or effectiveness of receiving input can lead to more efficient and / or more effective medical procedures, such as by facilitating more efficient and / or more effective operation of medical instruments. These and other advantages and benefits of the systems and methods described herein will become apparent herein.

[0025] Various embodiments will now be described in more detail with reference to the accompanying drawings. The disclosed systems and methods may provide one or more of the above-described benefits and / or various additional and / or alternative benefits, which will become apparent herein.

[0026] Figure 1 An example user input system 100 (“System 100”) for a computer-assisted medical system is described. System 100 may be included in, implemented by, or connected to one or more components of a computer-assisted medical system, such as those described below. Figure 9 The description illustrates a computer-aided medical system. For example, system 100 may be implemented by one or more components of a computer-aided medical system, such as an operating system, a user control system, or an auxiliary system. As another example, system 100 may be implemented by a separate computing system communicatively coupled to the computer-aided medical system.

[0027] like Figure 1 As shown, system 100 may include, but is not limited to, storage facility 102 and processing facility 104 selectively and communicatively interconnected. Facilities 102 and 104 may each include or be implemented by one or more physical computing devices, which include hardware and / or software components such as processors, memory, storage drives, communication interfaces, instructions stored in memory for execution by the processor, etc. Although facilities 102 and 104 are... Figure 1Facilities 102 and 104 are shown as separate facilities, but they can be combined into fewer facilities, such as a single facility, or divided into more facilities to serve a specific implementation. In some examples, each of facilities 102 and 104 can be distributed among multiple devices and / or multiple locations to serve a specific implementation.

[0028] Storage facility 102 may maintain (e.g., store) executable data used by processing facility 104 to perform any of the functionalities described herein. For example, storage facility 102 may store instructions 106 that can be executed by processing facility 104 to perform one or more of the operations described herein. Instructions 106 may be implemented by any suitable application, software, code, and / or other instance of executable data. Storage facility 102 may also maintain any data received, generated, managed, used, and / or transmitted by processing facility 104.

[0029] Processing facility 104 may be configured to perform (e.g., execute instructions 106 stored in storage facility 102) various operations associated with a user input system for a computer-aided medical system. Examples of such operations that may be performed by system 100 (e.g., by processing facility 104 of system 100) are described herein. In the following description, any reference to functions performed by system 100 shall be understood as being performed by processing facility 104 based on instructions 106 stored in storage facility 102.

[0030] Figure 2A An example configuration 200 for the user input system 100 used in a computer-assisted medical system 202 is described. Although Figure 2A A user input system 100 separate from the computer-assisted medical system 202 is shown, but in some examples, the user input system 100 may be a component of the computer-assisted medical system 202. As shown, the computer-assisted medical system 202 includes a set of controls 204, including a first control 204-1 and a second control 204-2. The computer-assisted medical system 202 also includes manipulators 206 (e.g., a first manipulator 206-1 and a second manipulator 206-2).

[0031] Control 204 can be implemented in any suitable manner to receive user input for manipulating manipulator 206. For example, control 204 can be implemented by a set of main controls, regarding... Figure 9Examples of such examples are described. Additionally or alternatively, control 204 may include any other hand-driven control that can be moved, rotated, clamped, and released by a hand-driven control (e.g., a joystick, steering pad, or other handheld control), foot-driven control (e.g., a pedal or other control configured to be operated by one or more feet), head-driven control (e.g., a helmet or sensor configured to detect head movement to manipulate manipulator 206), gaze-based control (e.g., a control configured to detect eye movement and / or gaze to manipulate manipulator 206), audio-based control (e.g., a control configured to detect voice commands to manipulate manipulator 206), and / or any other control configured to receive user input to manipulate manipulator 206.

[0032] Manipulator 206 may be implemented in any suitable manner to manipulate and / or control a medical device based on user input received from control 204. The medical device may include any apparatus, tool, or other instrument that can be used in a medical procedure, such as a medical device, non-medical device, imaging device, etc. Manipulator 206 may include any suitable mechanism that translates input from control 204 into operation of the medical device coupled to manipulator 206. For example, manipulator 206 may include a manipulator arm, a motorized joint, and / or other manipulator components configured to move based on user input received from control 204. The medical device may be coupled to the manipulator arm such that movement of the manipulator arm causes movement of the medical device.

[0033] Configuration 200 describes how user input system 100 configures computer-assisted medical system 202 to operate in manipulation input mode 208-1. In manipulation input mode 208-1, computer-assisted medical system 202 can be configured to translate user input received via control 204 into manipulation (e.g., movement) of manipulator 206 and / or an instrument connected to manipulator 206. For example, computer-assisted medical system 202 can receive a first input on a first control 204-1 and a second input on a second control 204-2. User input system 100 can translate the first input into manipulation of the first manipulator 206-1 and the second input into manipulation of the second manipulator 206-2. As an example, the first input can be a right-to-left movement of the first control 204-1. In response, user input system 100 can translate the movement of the first control 204-1 into a corresponding movement of the instrument connected to the first manipulator 206-1 (e.g., a corresponding amount of right-to-left movement of the instrument based on the input movement). The second input can be a counterclockwise rotation of the second control 204-2. In response, the user input system 100 can convert the rotation of the second control 204-2 into a corresponding rotation of another instrument connected to the second manipulator 206-2 (e.g., a counterclockwise rotation corresponding to the amount of the counterclockwise rotation input). Any other suitable manipulation of the control 204 by user input (e.g., movement of the hand, wrist, arm, etc., movement of the finger portion attached to the control 204, activation of a button or other input mechanism to send a control signal, etc.) (e.g., any change in posture, such as a change in position and / or orientation) can be considered as input received by the control 204 and can be converted by the user input system 100, which operates in the manipulation input mode 208-1, into a corresponding manipulation of the manipulator 206 and / or the instrument connected to the manipulator 206.

[0034] As used herein, manipulation of controls, manipulators, and / or instruments typically refers to the physical movement of the controls, manipulators, and / or instruments. For example, in some implementations, user input may be provided to a control, which causes the control to move within space. User input system 100 can translate user input into manipulation of a manipulator and / or an instrument coupled to the manipulator.

[0035] Figure 2BAnother example configuration 210 for the user input system 100 for a computer-assisted medical system 202 is described. In configuration 210, the user input system 100 configures the computer-assisted medical system 202 to operate in a mixed input mode 208-2. In mixed input mode 208-2, the computer-assisted medical system 202 can be configured to translate input received on one of the controls 204 into manipulation of one of the manipulators 206, while input received on the other control 204 can adjust parameter settings associated with a medical procedure. For example, the computer-assisted medical system 202 can receive a third input on a first control 204-1 and a fourth input on a second control 204-2. The user input system 100 can translate the third input into manipulation of the first manipulator 206-1, as in manipulation input mode 208-1. However, instead of translating the fourth input into manipulation of the second manipulator 206-2, the user input system 100 can adjust parameter settings based on the fourth input received on the second control 204-2. In some cases, a third and a fourth input can be received simultaneously (e.g., simultaneously, or within a relatively short period of time). Therefore, the user input system 100 can enable a user to manipulate the medical device (via the third input on the first control 204-1) and simultaneously adjust one or more parameter settings associated with the medical procedure (via the fourth input on the second control 204-2). For example, a user can use the first control 204-1 to move a cauterization device connected to the first manipulator 206-1 while using the second control 204-2 to adjust the power output of the cauterization device.

[0036] Parameter settings associated with a medical procedure may include any adjustable attributes and / or characteristics of the computer-assisted medical system, adjustable attributes and / or characteristics of any medical devices involved in the medical procedure, and adjustable attributes and / or characteristics of any data provided to or provided by the computer-assisted medical system (e.g., display attributes, user interface attributes, etc.). For example, parameter settings associated with a medical procedure may include display parameter settings of the display device associated with the medical procedure (e.g., brightness, color, saturation, hue, image layout, image format, etc., associated with the display device and / or the content displayed by the display device, such as a three-dimensional (3D) model or any other content), imaging parameter settings of the imaging device and / or image processing procedure associated with the medical procedure (e.g., one or more automatic exposure settings of the imaging device), operating parameter settings of the medical devices associated with the medical procedure, user interface parameter settings of the user interface associated with the computer-assisted medical system, and / or any other adjustable parameter settings of the computer-assisted medical system and / or the medical devices used in or otherwise associated with the medical procedure.

[0037] To illustrate, users can operate computer-aided medical systems to control medical devices, such as ultrasound probes located in the anatomical space associated with a medical procedure. Images of the anatomical space can be captured and presented in conjunction with the medical procedure. Figure 3 An illustrative image 300 provided by a computer-assisted medical system is shown. Image 300 may be an image of an anatomical space provided by an imaging device (e.g., an imaging device included in or coupled to a computer-assisted medical system). Image 300 shows an anatomical object 302, which may be an organ on which a user is performing a medical procedure. Image 300 also shows a first instrument 304-1 and a second instrument 304-2. The first instrument 304-1 may be coupled to a first manipulator of the computer-assisted medical system, and the second instrument 304-2 may be coupled to a second manipulator of the computer-assisted medical system. The first instrument 304-1 is also coupled to an ultrasound probe 306 (e.g., by grasping or otherwise engaging an embedded ultrasound probe). The user can control the ultrasound probe 306 through the computer-assisted medical system.

[0038] When system 100 configures the computer-assisted medical system to operate in a manipulation input mode, the first instrument 304-1 and the second instrument 304-2 can be controlled by first and second manipulators, which are in turn controlled by a user through input provided by a set of controls within the computer-assisted medical system. For example, the user can provide input to a first control in this set of controls to manipulate the first manipulator, causing the first instrument 304-1 to move within the anatomical space (e.g., by changing the position and / or orientation of the ultrasound probe 306) (e.g., to capture ultrasound images at different locations within the anatomical space, such as different locations on the anatomical object 302). The user can also provide input to a second control in this set of controls to manipulate the second manipulator, causing the second instrument 304-2 to move within the anatomical space (e.g., to perform any suitable task within the anatomical space).

[0039] When system 100 configures the computer-aided medical system to operate in a mixed input mode, the user can manipulate the first instrument 304-1 by providing input to the first control to operate the first manipulator. Thus, during a medical procedure, after system 100 switches from manipulator input mode to mixed input mode, the user can continue to move the position and / or orientation of the ultrasound probe 306. In mixed input mode, the user can also adjust one or more parameter settings associated with the medical procedure by providing input to the second control. For this purpose, any user input provided to the second control for manipulating the second manipulator during operation in manipulator input mode can alternatively be translated by system 100 into commands for adjusting parameter settings during operation in mixed input mode. For example, in manipulator input mode, clockwise rotation of the second control causes clockwise movement of the second manipulator, and thus clockwise movement of the second instrument 304-2. In mixed input mode, clockwise rotation of the second control can alternatively be translated into adjustments to parameter settings associated with the medical procedure, such as increasing the value of a parameter setting associated with the medical procedure.

[0040] In mixed input mode, system 100 can be configured to convert any user input received by the second control into adjustments to parameter settings in any suitable manner. As an example, a specific user input received by the second control can be converted into a specific corresponding adjustment to the parameter setting (e.g., clockwise rotation input is converted into an increase in the parameter setting value, counterclockwise rotation input is converted into a decrease in the parameter setting value, compression input is converted into an adjustment of the parameter setting, etc.). As another example, a specific user input received by the second control can be converted into operations in a user interface (e.g., a graphical user interface) that can be performed to adjust the parameter settings. For example, user input received by the second control can cause the cursor to move in the graphical user interface and / or be used in other ways to adjust the parameter settings. An example of a user interface that can be provided by system 100 and used to facilitate adjustments to parameter settings in mixed input mode will now be described.

[0041] Figures 4A-4EAn example interface 400 for a hybrid input mode for a computer-assisted medical system is illustrated. Interface 400 includes an image 402 of an anatomical space (e.g., images 402-1 to 402-5) (e.g., endoscopic images of an anatomical scene) and ultrasound images 404 (ultrasound images 404-1 to 404-5), which can be captured by an ultrasound probe 306 manipulated by a first instrument 304-1 coupled to a first manipulator of the computer-assisted medical system. In this example, image 402 may resemble image 300 (e.g., an anatomical space depicted in image 300 at different time points). Ultrasound image 404 can be displayed in interface 400 along with image 402 in any suitable configuration. In some configurations, ultrasound image 404 may overlap and / or partially obscure elements of image 402 from the view. While user interface 400 shows ultrasound image 404 centered in the view and obscuring most of image 402, the example shown is illustrative and not limiting. Ultrasound image 404 can be positioned within the user interface at any suitable location relative to image 402, including locations configured to reduce, minimize, or eliminate any overlap between ultrasound image 404 and image 402. For example, ultrasound image 404 may be smaller than shown in user interface 400 and may be located in a corner area of ​​user interface 400 to minimize obstruction or avoid obstructing areas of interest and / or elements (e.g., medical devices) depicted in image 402. For example, ultrasound image 404 may be configured to allow ultrasound probe 306 to remain unobstructed and within the user's field of vision, allowing the user to simultaneously use a first control to move ultrasound probe 306 in anatomical space and a second control to adjust one or more parameter settings associated with ultrasound probe 306.

[0042] Interface 400 also includes user interface elements 406, which include icons 408-414 indicating parameter settings that can be adjusted by the user via a second control in mixed input mode. In this example, icons 408-414 include a depth icon 408, a brightness icon 410, a Doppler icon 412, and a snapshot icon 414. The depth icon 408 allows the user to adjust the depth of focus of the ultrasound image 404 captured by the ultrasound probe 306. The brightness icon 410 allows the user to adjust the display brightness of the ultrasound image 404. The Doppler icon 412 allows the user to change the ultrasound mode (e.g., turn Doppler mode on or off). The snapshot icon 414 allows the user to save a still image or a portion of the ultrasound image 404 received from the ultrasound probe 306. Therefore, while the user adjusts any of these (or any other suitable) parameter settings using the second control, the user can also use the first control to move the ultrasound probe 306 to capture ultrasound images from different locations in anatomical space, such as different portions and / or angles of the anatomical object 302.

[0043] For example, Figure 4B This describes how a user adjusts the brightness of ultrasound image 404-2 in interface 400 in mixed input mode. The user can adjust the brightness by interacting with brightness icon 410 using a second control. Mixed input mode allows the user to select and / or interact with brightness icon 410 in any suitable manner. For example, mixed input mode allows the user to control the cursor on interface 400 via a second control, which the user can use to indicate and make selections on interface 400. Mixed input mode allows the user to move the cursor by moving the second control and to make selections via buttons or finger input (e.g., clenching gestures) or any other suitable input mechanism on the second control. Additionally or alternatively, mixed mode allows the user to cycle through and select from a predetermined set of options (e.g., icons 408-414 or any other parameter settings provided for adjustment) to adjust parameter settings. For example, mixed input mode allows the user to select the next option by moving the second control in one direction and to select the previous option by moving the second control in the opposite direction or any other suitable input mechanism on the second control. Users can use this input mechanism to cycle through options such as icons 408-414 and select icons to adjust the parameter settings associated with the selected icons.

[0044] like Figure 4B As shown, the user selects brightness icon 410. Based on this selection, user interface element 406 can display brightness slider 416 to allow the user to adjust the brightness of ultrasound image 404-2. The user can interact with brightness slider 416 in a manner similar to the user selecting brightness icon 410. For example, a hybrid input mode allows the user to use a second control to control the cursor and select brightness settings on brightness slider 416. Additionally or alternatively, when brightness icon 410 is selected, the user can use the second control to raise and lower brightness slider 416 by moving the second control up and down, rotating the second control clockwise and counterclockwise, pressing a button on the second control, and / or any other suitable input mechanism of the second control. The brightness of ultrasound image 404-2 can be adjusted based on the adjustment of the brightness parameter settings via slider 416.

[0045] Figure 4C This illustrates an example of a user adjusting the ultrasound image capture depth in mixed input mode. When the user adjusts the depth of the ultrasound scan in interface 400, one or more depth scan parameters of the ultrasound probe can be adjusted to change the ultrasound image displayed in ultrasound image 404-3. Adjustments and changes can occur in real time.

[0046] and Figure 4C The interaction of the interface 400 shown can be similar to Figure 4BThe interaction described herein. Similar to the brightness icon 410, the computer-assisted medical system allows the user to select the depth icon 408 using a second control in a mixed input mode. Based on the selection of the depth icon 408, the user interface element 406 can display a depth slider 418, which allows the user to select the focusing depth of the ultrasound probe 306. Based on the depth selection, the ultrasound image 404-3 can be a deeper or shallower ultrasound image within the tissue of the target (e.g., anatomical object 302) from the ultrasound probe 306. As described herein, the mixed input mode allows the user to interact with the depth icon 408 and the depth slider 418 in any suitable manner.

[0047] Figure 4D This illustrates an alternative example of interface 400 for users to adjust the depth of ultrasound images 404-4 in mixed input mode. Figure 4D Instead of the depth slider 418, a depth arrow 420 is shown. Using the depth arrow 420, the user can select a deeper or shallower focusing depth for the ultrasound probe 306. Interacting with the depth arrow 420 using the second control is similar to interacting with other icons and parameter setting elements. When the user uses the second control to select the upward arrow of the depth arrow 420, ultrasound image 404-4 displays an ultrasound image from the anatomical object 302 at increasingly shallower depths. When the user uses the second control to select the downward arrow of the depth arrow 420, ultrasound image 404-4 displays an ultrasound image from the anatomical object 302 at increasingly deeper depths. In this way, the mixed input mode allows for efficient observation of different parts of the anatomical object 302, allowing the user to simultaneously adjust the ultrasound scan position on the anatomical object 302 and the focusing depth within the anatomical object 302.

[0048] As another example, Figure 4E This illustrates that in mixed input mode, the user selects the Doppler mode for ultrasound imaging in interface 400. As shown, the user has selected the Doppler icon 412, which can be indicated by the highlight of the Doppler icon 412, the outline of the Doppler icon 412, the color change of the Doppler icon 412, or any other suitable indicator. The selection of the Doppler icon 412 can be described in the same way as other icons (e.g., depth icon 408 and brightness icon 410). Based on the selection of the Doppler icon 412, ultrasound images 404-5 can be ultrasound images showing blood flow direction and / or any other information that can be captured by the Doppler mode of ultrasound imaging.

[0049] Although user interface element 406 displays four adjustable parameter settings, in mixed input mode, more, fewer, or different parameters can be adjusted. For example, in addition to or instead of the Doppler mode, other ultrasound modes (e.g., modalities) can be adjusted. Furthermore, any other suitable parameter settings associated with ultrasound imaging can be provided for adjustment, such as gain, frequency, image width, image processing, color, etc.

[0050] In some examples, the hybrid input mode allows a user to adjust the overall brightness of interface 400, the brightness of one or more portions of interface 400 (e.g., image 402, image 402 and ultrasound image 404, etc.), the size, position, and / or features (e.g., transparency, color, visibility, etc.) of ultrasound image 404 shown in interface 400. For example, a user might want to see more of image 402, such as seeing the position of ultrasound probe 306 more clearly in hybrid input mode. The hybrid input mode provides adjustable parameters for setting the size, position, transparency, visibility, or any other suitable characteristics of ultrasound image 404, such that ultrasound image 404 occupies less space on interface 400 and / or otherwise allows image 402 or elements of interest depicted in image 402 to be visible.

[0051] Figure 5A and 5B Another example interface 500 illustrating the input mode of a computer-assisted medical system is provided. Interface 500 shows an illustrative image 502 provided by the computer-assisted medical system. Image 502 may be an image of an anatomical space provided by an imaging device. Image 502 shows an anatomical object 504, which may be an organ on which a user is performing a medical procedure. Image 502 also shows a first instrument 506-1 and a second instrument 506-2. As described above, the first instrument 506-1 may be coupled to a first manipulator of the computer-assisted medical system, and the second instrument 506-2 may be coupled to a second manipulator of the computer-assisted medical system. Interface 500 also shows an inserted image 508 (e.g., in a picture-in-picture window) showing a 3D model 510, such as a 3D model of the anatomical object 504 (e.g., a preoperative or intraoperative 3D model of the anatomical object 504).

[0052] In manipulation mode, the user can provide input to the first control in the set of controls to manipulate the first manipulator, causing the first instrument 506-1 to move within the anatomical space. Similarly, the user can provide input to the second control in the set of controls to manipulate the second manipulator, causing the second instrument 506-2 to move within the anatomical space. The user can also provide input to control the imaging device to change the view of image 502. For example, the user can move their head, which can cause a corresponding movement of the imaging device to display a change in the angle or field of view of image 502. Additionally or alternatively, the imaging device can change the view of image 502 to follow the movement of the first instrument 506-1 and / or the second instrument 506-2. Any other suitable input can be provided to change image 502.

[0053] In manipulation mode, in response to changes in image 502 and / or the imaging device, image 508 can show a corresponding change in the view of 3D model 510. For example, if the user zooms in on the imaging device to display a magnified view of image 502, then image 508 can show a similar magnified view of 3D model 510. If the user zooms in on the imaging device to show a portion of the right side currently shown in image 502, then image 508 can similarly show a portion of the right side of 3D model 510 currently shown in image 508. Therefore, in manipulation mode, image 508 can track 3D model 510 based on the anatomical object 504 shown in image 502.

[0054] When system 100 configures the computer-assisted medical system to operate in mixed input mode, the user can continue to operate the first instrument 506-1 by providing input to the first control to manipulate the first manipulator. Input provided by the user to the second control can be converted into adjusting display parameter settings associated with the medical procedure.

[0055] For example, Figure 5B This describes the interface 500 in mixed input mode. Figure 5B An interface 500 with image 502 is shown, which shows an anatomical object 504, a first instrument 506-1, and a second instrument 508-2. Figure 5BAn inserted image 512 is also shown, which shows different views of the 3D model 510 (e.g., zoomed in to the top of the 3D model 510). In mixed input mode, the user can adjust the display parameter settings of the inserted image 512 by providing input to the second control to adjust the view of the 3D model 510. Therefore, the user can use the input provided to the second control to zoom in, rotate, pan, or otherwise adjust the 3D model 510 and / or the camera view of the 3D model 510. Simultaneously, the input provided to the first control can continue to manipulate the first manipulator to control the first instrument 506-1. In some examples, when the system 100 exits the mixed input mode and returns to the manipulation input mode, image 512 may return to the image of the tracking image 502 (e.g., image 508). In other examples, image 512 may be held while the user manipulates and / or displays a combination of manipulated and tracking images (e.g., maintaining the zoomed-in but tracking image 502's motion and / or angle).

[0056] In some examples, the parameters provided for adjustment may depend on the context of the medical procedure. For instance, the parameters provided for adjustment may be based on the instrument controlled by the first control in mixed input mode. For example, when an ultrasound probe is controlled by user input to the first control in mixed input mode, parameters related to the ultrasound and / or the ultrasound probe may be provided by the second control for adjustment. If different instruments (e.g., suture tools) are manipulated by the first control, then different parameters related to the different instruments (e.g., suture type, suture technique, etc.) may be presented for adjustment.

[0057] In some examples, system 100 may be configured to provide various mechanisms to assist users of the computer-assisted medical system in operating in a mixed input mode. Such mechanisms may include those configured to mitigate risks and / or help users manage cognitive loads that are increased or different from those associated with using a manipulative input mode. For example, user input system 100 may include mechanisms that are sufficiently clear when a user enters and exits a mixed input mode. For example, user input system 100 may be configured to provide notifications, such as alarms, upon entering and / or exiting a mixed input mode. Notifications may include one or more notifications and / or notification types, such as audio notifications, visual notifications, haptic notifications, etc.

[0058] In some examples, the user input system 100 may require relatively fine and / or deliberate input to enter and / or exit a mixed input mode. For example, the input may include a series of inputs in a specified order, inputs from an input mechanism that is not easily triggered, multiple consecutive inputs on an input mechanism (e.g., double-click, triple-click, etc.), the duration of the input (e.g., holding a button down for a predetermined amount of time, etc.), or any other suitable input that can mitigate unintentional entry and / or exit from the mixed input mode.

[0059] In some examples, the user input system 100 may include mechanisms to adequately inform the user that the user input system 100 is operating in a mixed input mode. For example, the user input system 100 may provide a visualization of the user interface and / or images presented to the user in mixed input mode that differs from the visualization of the user interface and / or images presented to the user in manipulation input mode and / or another operating mode. Such different visualizations may include any suitable visualization mechanisms, such as adding prominent parameter setting user interface elements, changing visualization parameters (e.g., changing colors, changing grayscale, changing visualization style, inverting images, changing image size, etc.), etc. Additionally or alternatively, such mechanisms may include audio indicators indicating that the user is in mixed mode, such as periodic beeps or other alarms, sound playback for the duration of the mixed mode, or any other suitable audio indicator.

[0060] In some examples, the user input system 100 may include a mechanism for transitioning a user into or out of a mixed input mode. For example, upon entering and / or exiting a mixed input mode, the user input system 100 may filter input received on controls over a predetermined amount of time. For example, upon entering and / or exiting a mixed input mode, the user input system 100 may filter out control movements exceeding a threshold speed and / or distance for a period of time (e.g., one second, a portion of a second, several seconds, etc.). Additionally or alternatively, the user input system 100 may filter all movements within a predetermined amount of time upon entering and / or exiting a mixed input mode (e.g., preventing all movements of controls and / or tools). Additionally or alternatively, the user input system 100 may temporarily constrain the movement of controls, manipulators, and / or instruments, for example, by setting the speed of controls, manipulators, and / or instruments upon entering and / or exiting a mixed input mode. For example, upon entering and / or exiting a mixed input mode, the user input system 100 may initially set the speed of controls, manipulators, and / or instruments to zero for a predetermined amount of time. The user input system 100 can constrain the movement of controls, manipulators, and / or instruments in any other suitable manner.

[0061] In some examples, the user input system 100 may include a mechanism to restrict the simultaneous operation of two controls in a mixed input mode. For example, the user input system 100 may filter input received on controls such that when user input is received on one control, the user input system 100 may avoid converting user input received on another control. In this way, the user input system 100 may allow the user to manipulate the manipulator or adjust parameter settings one at a time (but not simultaneously). The user input system 100 may implement such constraints in any suitable manner.

[0062] User input system 100 may include any of the above-described example mechanisms, any combination or sub-combination thereof, and / or any other suitable mechanisms to clearly distinguish operation under mixed input modes, thereby enabling the user to fully understand the effect of input provided on the controls.

[0063] Figure 6 An example user input method 600 for a computer-aided medical system is described. Although Figure 6 The description illustrates example operations based on one embodiment, but other embodiments may omit, add, reorder, combine, and / or modify these practices. Figure 6 Any operation shown. Figure 6 One or more operations shown can be performed by a user input system, such as user input system 100, any components included therein, and / or any implementation thereof.

[0064] In operation 602, the user input system can instruct a computer-assisted medical system, which includes a set of controls, including a first control and a second control configured to receive user input for operation in a manipulation input mode, wherein user input to the first control manipulates a first manipulator of the computer-assisted medical system, and user input to the second control manipulates a second manipulator of the computer-assisted medical system. Operation 602 can be performed in any of the manner described herein.

[0065] In operation 604, the user input system can instruct the computer-aided medical system to operate in a mixed input mode, wherein user input to a first control manipulates a first manipulator, and user input to a second control adjusts parameter settings associated with the medical procedure. Operation 604 can be performed in any of the manner described herein.

[0066] Figure 7 An example user input method 700 for a computer-aided medical system is described. Although Figure 7 The description illustrates example operations based on one embodiment, but other embodiments may omit, add, reorder, combine, and / or modify these practices. Figure 7 Any operation shown. Figure 7 One or more operations shown can be performed by a user input system, such as user input system 100, any components included therein, and / or any implementation thereof.

[0067] In operation 702, the user input system operates in a manipulation input mode. For example, the user input system may configure a computer-assisted medical system to operate based on a manipulation input mode. Operation 702 may be performed in any of the manner described herein.

[0068] In operation 704, the user input system determines whether to change to another input mode. This determination may be based on user input received by the user input system and may include the user input system determining whether the received user input matches a predefined command for changing the mode. If not, the user input system may return to operation 702 and continue operating in the manipulation input mode. If yes, the user input system may move to operation 706 to change the valid input mode from the manipulation input mode to a mixed input mode.

[0069] In operation 706, the user input system operates in a mixed input mode. For example, the user input system may configure a computer-assisted medical system to operate based on a mixed input mode. Operation 706 may be performed in any of the manner described herein.

[0070] In operation 708, the user input system determines whether to change to another input mode. Similar to operation 704, this determination may be based on user input received by the user input system and may include the user input system determining whether the received user input matches a predefined command for changing the mode. If not, the user input system may return to operation 706 and continue operating in the mixed input mode. If yes, the user input system may move to operation 702 to change the valid input mode from the mixed input mode to the manipulation input mode.

[0071] Any suitable user input can be defined and used to trigger transitions from manipulation input mode to mixed input mode and / or from mixed input mode to manipulation input mode. Each transition can be triggered using the same or different user input.

[0072] Figure 8 This describes an example user input method 800 for a computer-aided medical system. Although Figure 8 The description illustrates example operations based on one embodiment, but other embodiments may omit, add, reorder, combine, and / or modify these practices. Figure 8 Any operation shown. Figure 8 One or more operations shown may be performed by a user input system (such as user input system 100), any of its components and / or any implementation thereof.

[0073] In operation 802, when operating in the manipulation input mode, the user input system receives user input to the controls. Operation 802 can be performed in any of the ways described herein.

[0074] In operation 804, the user input system manipulates the instrument based on user input to the controls received in the manipulation input mode. Operation 804 can be performed in any of the manner described herein.

[0075] In operation 806, when operating in mixed input mode, the user input system receives user input to controls. In some examples, the received input may include the same type of input received in manipulation input mode in operation 802 (e.g., manipulation, button activation, etc.). Operation 806 may be performed in any of the manner described herein.

[0076] In operation 808, the user input system adjusts parameter settings based on user input to controls received in mixed input mode. Operation 808 can be performed in any of the ways described herein.

[0077] Figure 9 An illustrative computer-aided medical system 900 (“Medical System 900”) is shown. System 100 may be implemented by Medical System 900, connected to Medical System 900, and / or used in conjunction with Medical System 900 in other ways.

[0078] As shown in the figure, the medical system 900 may include an operating system 902, a user control system 904, and an auxiliary system 906, which are interconnected in a communicative manner. The medical system 900 can be used by a medical team to perform computer-aided medical procedures on a patient 908. As shown in the figure, the medical team may include a surgeon 910-1, an assistant 910-2, a nurse 910-3, and an anesthesiologist 910-4, all of whom can be collectively referred to as "medical team members 910". Additional or alternative medical team members may be present during a medical session as they may contribute to the implementation.

[0079] although Figure 9 This describes an ongoing minimally invasive procedure; however, it should be understood that Medical System 900 can be similarly used to perform open surgical procedures or other types of medical procedures that can similarly benefit from the accuracy and convenience of Medical System 900. Furthermore, it should be understood that the medical sessions in which Medical System 900 can be used throughout can include not only, but also, other, minimally invasive procedures. Figure 9 The described medical procedure refers to the operational phases of the procedure, but may also include pre-operative, post-operative, and / or other appropriate phases of the medical procedure.

[0080] like Figure 9As shown, the manipulation system 902 may include a plurality of manipulator arms 912 (e.g., manipulator arms 912-1 to 912-4), to which multiple medical devices may be coupled. Each medical device may be implemented by any suitable therapeutic instrument (e.g., a tool with tissue interaction capabilities), medical instrument, imaging device (e.g., an endoscope, ultrasound probe, etc.), diagnostic instrument, etc., that can be used to perform computer-assisted medical procedures on a patient 908 (e.g., by being at least partially inserted into and manipulated to perform computer-assisted medical procedures on the patient 908). In some examples, one or more medical devices may include force sensing and / or other sensing capabilities. Although the manipulation system 902 is depicted and described herein as including four manipulator arms 912, it will be appreciated that the manipulation system 902 may include only a single manipulator arm 912 or any other number of manipulator arms that may be used in a particular implementation.

[0081] The manipulator arm 912 and / or the medical device attached to the manipulator arm 912 may include one or more displacement sensors, orientation sensors, and / or position sensors for generating raw (i.e., uncorrected) kinematic information. One or more components of the medical system 900 may be configured to use the kinematic information to track (e.g., determine the position of the medical device) and / or control the medical device.

[0082] User control system 904 may be configured to facilitate surgeon 910-1's control of manipulator arm 912 and medical instruments attached to manipulator arm 912. For example, surgeon 910-1 may interact with user control system 904 to remotely manipulate manipulator arm 912 and medical instruments. To this end, user control system 904 may provide surgeon 910-1 with images (e.g., high-resolution 3D images) of the surgical area associated with patient 908 captured by an imaging system (such as any medical imaging system described herein). In some examples, user control system 904 may include a stereoscopic viewer with two displays, where surgeon 910-1 can view stereoscopic images of the anatomical space associated with patient 908 and generated by a stereoscopic imaging system. Surgeon 910-1 may utilize the images to perform one or more procedures, wherein one or more medical instruments are attached to manipulator arm 912.

[0083] To facilitate control of the medical device, the user control system 904 may include a set of master controls. These master controls can be manipulated by the surgeon 910-1 to control the movement of the medical device (e.g., by utilizing robotic and / or remote operation technologies). The master controls can be configured to detect various movements of the surgeon 910-1's hand, wrist, and fingers. The user control system 904 can be configured to receive information from the master controls, such as position, movement, rotation, interaction, etc., to track the movement of the master controls. The user control system 904 can translate the received information into corresponding movements and / or controls of the manipulator arm 912. In this way, the surgeon 910-1 can use the master controls to manipulate the medical device attached to the manipulator arm 912. In this manner, the surgeon 910-1 can intuitively perform procedures using one or more medical devices.

[0084] The auxiliary system 906 may include one or more computing devices configured to perform primary processing operations of the medical system 900. In such a configuration, the one or more computing devices included in the auxiliary system 906 may control and / or coordinate operations performed by various other components of the medical system 900, such as the operating system 902 and the user control system 904. For example, the computing devices included in the user control system 904 may send instructions 906 to the operating system 902 via one or more computing devices included in the auxiliary system. As another example, the auxiliary system 906 may receive and process image data representing images captured by an imaging device attached to one of the manipulators 912 from the operating system 902.

[0085] In some examples, the assistive system 906 may be configured to present visual content to a medical team member 910 who may not have access to the images provided to the surgeon 910-1 at the user control system 904. For this purpose, the assistive system 906 may include a display monitor 914 configured to display one or more user interfaces, such as images of the surgical area (e.g., 2D images, 3D images), information associated with the patient 908 and / or medical procedures, and / or any other visual content that may be useful for the specific implementation. For example, the display monitor 914 may display an image of the surgical area along with additional content displayed concurrently with the image (e.g., graphical content, contextual information, etc.). In some embodiments, the display monitor 914 is implemented by a touchscreen display that the medical team member 910 can interact with (e.g., via touch gestures) to provide user input to the medical system 900.

[0086] The operating system 902, the user control system 904, and the auxiliary system 906 can be interconnected in any suitable manner. For example, Figure 9As shown, the operating system 902, the user control system 904, and the auxiliary system 906 can be communicatively connected via control line 916, which can represent any wired or wireless communication link that can serve a specific implementation. Therefore, the operating system 902, the user control system 904, and the auxiliary system 906 can each include one or more wired or wireless communication interfaces, such as one or more local area network interfaces, Wi-Fi network interfaces, cellular interfaces, etc.

[0087] Some of the examples described herein relate to the implementation of System 100 with a computer-aided medical system such as Medical System 900. In such implementations, System 100 may be configured to selectively instruct Medical System 900 to operate in either a manipulation input mode or a mixed input mode as described herein. The effective manipulation input mode manages how user input received by the master control is translated into operations of Medical System 900. In other examples, System 100 may be similarly implemented with other computer-aided systems (e.g., surgical systems), robotic systems, etc.

[0088] In some examples, a non-transitory computer-readable medium may be provided for storing computer-readable instructions, based on the principles described herein. When executed by a processor of a computing device, the instructions may direct the processor and / or computing device to perform one or more operations, including one or more operations described herein. Such instructions may be stored and / or transmitted using any of a variety of known computer-readable media.

[0089] As used herein, non-transitory computer-readable media may include any non-transitory storage medium that contributes to providing data (e.g., instructions) that can be read and / or executed by a computing device (e.g., a computer's processor). For example, non-transitory computer-readable media may include, but is not limited to, any combination of non-volatile storage media and / or volatile storage media. Illustrative non-volatile storage media include, but are not limited to, read-only memory, flash memory, solid-state drives, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), ferroelectric random access memory (“RAM”), and optical discs (e.g., optical discs, digital video discs, Blu-ray discs, etc.). Illustrative volatile storage media include, but are not limited to, RAM (e.g., dynamic RAM).

[0090] Figure 10 An example computing device 1000 is described, which may be specifically configured to perform one or more processes described herein. Any system, unit, computing device, and / or other component described herein may be implemented by computing device 1000.

[0091] like Figure 10As shown, computing device 1000 may include a communication interface 1002, a processor 1004, a storage device 1006, and an input / output (“I / O”) module 1008, which are communicatively connected to each other via communication infrastructure 1010. Although the example computing device 1000 is... Figure 10 As shown in the image, but Figure 10 The components shown are not intended to be limiting. Additional or alternative components may be used in other embodiments. A more detailed description will follow. Figure 10 The components of the computing device 1000 shown.

[0092] The communication interface 1002 can be configured to communicate with one or more computing devices. Examples of the communication interface 1002 include, but are not limited to, wired network interfaces (e.g., network interface cards), wireless network interfaces (e.g., wireless network interface cards), modems, audio / video connections, and any other suitable interfaces.

[0093] Processor 1004 generally refers to any type or form of processing unit capable of processing data and / or interpreting, executing, and / or directing the execution of one or more of the instructions, procedures, and / or operations described herein. Processor 1004 can perform operations by executing computer-executable instructions 1012 (e.g., application programs, software, code, and / or other executable data instances) stored in storage device 1006.

[0094] Storage device 1006 may include one or more data storage media, devices, or configurations, and may take any type, form, and combination of data storage media and / or devices. For example, storage device 1006 may include, but is not limited to, any combination of non-volatile media and / or volatile media as described herein. Electronic data, including the data described herein, may be temporarily and / or permanently stored in storage device 1006. For example, data representing computer-executable instructions 1012 configured to bootstrap processor 1004 to perform any of the operations described herein may be stored within storage device 1006. In some examples, data may be arranged in one or more databases residing within storage device 1006.

[0095] I / O module 1008 may include one or more I / O modules configured to receive user input and provide user output. I / O module 1008 may include any hardware, firmware, software, or a combination thereof that supports input and output capabilities. For example, I / O module 1008 may include hardware and / or software for capturing user input, including but not limited to a keyboard or keypad, a touchscreen component (e.g., a touchscreen display), a receiver (e.g., an RF or infrared receiver), a motion sensor, and / or one or more input buttons.

[0096] I / O module 1008 may include one or more means for presenting output to a user, including but not limited to a graphics engine, a display (e.g., a screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In some embodiments, I / O module 1008 is configured to provide graphical data to the display for presentation to the user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content that may be servicing a particular implementation.

[0097] In some examples, any of the facilities described herein may be implemented by or within one or more components of computing device 1000. For example, one or more applications 1012 residing in storage device 1006 may be configured to direct the implementation of processor 1004 to perform one or more operations or functions associated with the processing facility 104 of system 100. Similarly, storage facility 102 of system 100 may be implemented by or within the implementation of storage device 1006.

[0098] One or more operations described in this document can be performed in real time. As used herein, an operation performed "in real time" will be understood as being performed immediately without any inappropriate delay, even if there is absolutely zero delay.

[0099] Any system, apparatus, and / or component thereof may be implemented in any suitable combination or sub-combination. For example, any system, apparatus, and / or component thereof may be implemented as a device configured to perform one or more of the operations described herein.

[0100] In the foregoing description, various illustrative embodiments have been described with reference to the accompanying drawings. However, it will be apparent that various modifications and changes can be made thereto, and other embodiments can be implemented without departing from the scope of the invention as set forth in the appended claims. For example, certain features of one embodiment described herein may be combined with or substituted for features of another embodiment described herein. Therefore, the specification and drawings are to be considered illustrative rather than restrictive.

Claims

1. A system comprising: Memory, which stores instructions; A processor, communicatively connected to the memory and configured to execute instructions to: Instructing a computer-assisted medical system to operate in a manipulation input mode, the computer-assisted medical system including a set of controls, the set of controls including a first control and a second control configured to receive user input, in the manipulation input mode. The user input to the first control manipulates the first manipulator of the computer-assisted medical system, and The second controller manipulates the computer-assisted medical system based on user input to the second control; and Instruct the computer-assisted medical system to operate in mixed input mode, wherein The user input to the first control manipulates the first manipulator, and The user input to the second control adjusts the parameter settings associated with the medical procedure.

2. The system of claim 1, wherein the user input to the second control for adjusting the parameter settings in the mixed input mode includes the same type of input as the user input to the second control for manipulating the second manipulator in the manipulation input mode.

3. The system of claim 1, wherein the parameter settings associated with the medical procedure include display parameter settings associated with the medical procedure.

4. The system of claim 3, wherein the display parameter settings associated with the medical procedure include display parameters of a three-dimensional (3D) model associated with the medical procedure.

5. The system of claim 1, wherein the parameter settings associated with the medical procedure include device parameter settings associated with the device controlled by the first manipulator.

6. The system according to claim 1, wherein: The first manipulator is configured to manipulate the ultrasound probe; and The parameter settings include at least one of the following: Gain of the ultrasound image from the ultrasound probe, The depth of the ultrasound image, or The modalities of the ultrasonic probe.

7. The system of claim 1, wherein the processor is further configured to execute the instructions to instruct the computer-assisted medical system to provide notification when entering or exiting at least one of the mixed input modes.

8. The system of claim 1, wherein the processor is further configured to execute the instructions to: Receive a command to exit the mixed input mode; Based on the command, the computer-assisted medical system is instructed to exit the mixed input mode; and When exiting the mixed input mode, the computer-assisted medical system is instructed to temporarily restrict the movement of the manipulator.

9. The system of claim 1, wherein the user input to the first control, which manipulates the first manipulator in the mixed input mode, and the user input to the second control, which adjusts the parameter settings in the mixed input mode, are simultaneously received.

10. A method comprising: The computer-assisted medical system, guided by a processor, operates in a manipulation input mode. The computer-assisted medical system includes a set of controls, comprising a first control and a second control configured to receive user input in the manipulation input mode. The user input to the first control manipulates the first manipulator of the computer-assisted medical system, and The second controller manipulates the computer-assisted medical system based on user input to the second control; and The processor guides the computer-assisted medical system to operate in a mixed input mode, wherein... The user input to the first control manipulates the first manipulator, and The user input to the second control adjusts the parameter settings associated with the medical procedure.

11. The method of claim 10, wherein the user input to the second control for adjusting the parameter settings in the mixed input mode includes the same type of input to the second control as the user input for manipulating the second manipulator in the manipulation input mode.

12. The method of claim 10, wherein the parameter settings associated with the medical procedure include display parameter settings associated with the medical procedure.

13. The method of claim 10, wherein the parameter settings associated with the medical procedure include device parameter settings associated with the device controlled by the first manipulator.

14. The method of claim 10, wherein: The first manipulator is configured to manipulate the ultrasound probe; and The parameter settings include at least one of the following: Gain of the ultrasound image from the ultrasound probe, The depth of the ultrasound image, or The modalities of the ultrasonic probe.

15. The method of claim 10, further comprising instructing the computer-assisted medical system via the processor to provide notification when entering or exiting at least one of the mixed input modes.

16. The method of claim 10, further comprising: The processor receives a command to exit the mixed input mode; The processor, based on the command, guides the computer-assisted medical system to exit the mixed input mode; and The processor, when exiting the mixed input mode, instructs the computer-assisted medical system to temporarily restrain the movement of the manipulator.

17. The method of claim 10, wherein the user input to the first control, which manipulates the first manipulator in the mixed input mode, and the user input to the second control, which adjusts the parameter settings in the mixed input mode, are simultaneously received.

18. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to: Instructing a computer-assisted medical system to operate in a manipulation input mode, the computer-assisted medical system including a set of controls, the set of controls including a first control and a second control configured to receive user input, in the manipulation input mode. The user input to the first control manipulates the first manipulator of the computer-assisted medical system, and The second controller manipulates the computer-assisted medical system based on user input to the second control; and The computer-aided medical system is instructed to operate in a mixed input mode, wherein The user input to the first control manipulates the first manipulator, and The user input to the second control adjusts the parameter settings associated with the medical procedure.

19. The non-transitory computer-readable medium of claim 18, wherein the user input adjusting the parameter settings in the mixed input mode includes the same type of input as the user input to the second control used to manipulate the second manipulator in the manipulation input mode.

20. The non-transitory computer-readable medium of claim 18, wherein the parameter settings associated with the medical procedure include display parameter settings associated with the medical procedure.

21. The non-transitory computer-readable medium of claim 18, wherein the parameter settings associated with the medical procedure include device parameter settings associated with a device controlled by the first manipulator.

22. The non-transitory computer-readable medium according to claim 18, wherein: The first manipulator is configured to manipulate the ultrasound probe; and The parameter settings include at least one of the following: Gain of the ultrasound image from the ultrasound probe, The depth of the ultrasound image, or The modalities of the ultrasonic probe.

23. The non-transitory computer-readable medium of claim 18, wherein when the instructions are executed, the instructions further cause the processor to instruct the computer-assisted medical system to provide notification when entering or exiting at least one of the mixed input modes.

24. The non-transitory computer-readable medium of claim 18, wherein when the instructions are executed, the instructions further cause the processor to: Receive a command to exit the mixed input mode; Based on the command, the computer-assisted medical system is instructed to exit the mixed input mode; and When exiting the mixed input mode, the computer-assisted medical system is instructed to temporarily restrict the movement of the manipulator.

25. The non-transitory computer-readable medium of claim 18, wherein the user input to the first control, which manipulates the first manipulator in the mixed input mode, and the user input to the second control, which adjusts the parameter settings in the mixed input mode, are simultaneously received.

Images