Horizontal image alignment in rotatable imaging system

Abstract

A surgical system includes a handheld cable housing having a user input device positioned thereon. A surgical system includes an endoscope shaft having a distal end and a proximal end. A proximal end of the endoscope shaft is coupled with the handheld cable housing, wherein a distal end of the endoscope shaft includes an image sensor. The endoscope shaft is rotatable relative to the handheld cable housing. The one or more angular position sensors are configured to measure an angular offset relative to a defined image horizon. Upon receiving angular offset data indicating an angular offset with respect to a defined image horizontal line, the image processor generates rotated image data based on the angular offset data and causes the rotated image data to be displayed.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 382,967, filed on November 9, 2022, which is incorporated herein by reference in its entirety. Background Art

[0003] Conventional endoscopes include an image sensor in the handle of the endoscope and a rotatable rod - lens system that conveys a scene from the field of view of the rotatable rod - lens system to the image sensor. When the rod - lens system rotates to view different fields of view, the rod - lens optics rotate relative to the fixed image sensor. Thus, the image produced by the image sensor remains oriented relative to the fixed image sensor within the handle of the endoscope. Summary of the Invention

[0004] A first aspect of the present disclosure includes a surgical system that includes a hand - held cable housing on which a user input device is positioned. The surgical system further includes an endoscope shaft having a distal end and a proximal end. The proximal end of the endoscope shaft is coupled to the hand - held cable housing. The distal end of the endoscope shaft includes an image sensor. The endoscope shaft is rotatable relative to the hand - held cable housing. The surgical system further includes one or more angular position sensors configured to measure an angular offset relative to a defined image horizon.

[0005] In some embodiments of the first aspect of the surgical system, the one or more angular position sensors are positioned at a coupling portion between the endoscope shaft and the hand - held cable housing.

[0006] In any of the above - described embodiments of the first aspect of the surgical system, the defined image horizon is relative to the hand - held cable housing.

[0007] In any of the above - described embodiments of the first aspect of the surgical system, the angular offset is a measurement of the angular rotation of the endoscope shaft relative to the hand - held cable housing.

[0008] In any of the above - described embodiments of the first aspect of the surgical system, the angular offset is a measurement of the angular rotation between the defined image horizon and the field - of - view direction of the endoscope shaft.

[0009] In any of the above - described embodiments of the first aspect of the surgical system, the defined image horizon is the horizontal mid - plane of the hand - held cable housing at the coupling portion.

[0010] In any of the above-described embodiments of the first aspect of the surgical system, the defined image horizontal line is orthogonal to the vertical median plane of the handheld cable housing and parallel to the longitudinal axis of the handheld cable housing.

[0011] In any of the above-described embodiments of the first aspect of the surgical system, the defined image horizontal line is relative to the endoscope axis.

[0012] In any of the above-described embodiments of the first aspect of the surgical system, the angular offset is a measure of the angular rotation of the handheld cable housing relative to the endoscope axis.

[0013] In any of the above-described embodiments of the first aspect of the surgical system, the defined image horizontal line is the horizontal median plane of the endoscope axis at the coupling portion.

[0014] In any of the above-described embodiments of the first aspect of the surgical system, the defined image horizontal line is orthogonal to the vertical median plane of the endoscope axis and parallel to the longitudinal axis of the endoscope axis.

[0015] In any of the above-described embodiments of the first aspect of the surgical system, the defined image horizontal line is based on the sensed direction of gravity.

[0016] In any of the above-described embodiments of the first aspect of the surgical system, the one or more angular position sensors are positioned at the distal end of the endoscope axis.

[0017] In any of the above-described embodiments of the first aspect of the surgical system, the one or more angular position sensors are positioned at the handheld cable housing.

[0018] In any of the above-described embodiments of the first aspect of the surgical system, the one or more angular position sensors include one or more sensors selected from the group consisting of Hall effect sensors, mechanical encoders, optical encoders, magnetic encoders, electromagnetic induction encoders, encoders, rotary potentiometers, resolvers, gravity sensors, gyroscopes, magnetometers, and linear acceleration sensors.

[0019] In any of the above-described embodiments of the first aspect of the surgical system, the user input device is positioned on the control surface of the handheld cable housing.

[0020] In any of the above-described embodiments of the first aspect of the surgical system, the control surface is the top surface of the handheld cable housing.

[0021] In any of the above-described embodiments of the first aspect of the surgical system, the user input device is selected from the group consisting of physical buttons, capacitive sensing buttons, soft buttons on a touch screen, switches, touch pads, rollers, and directional keys.

[0022] In any of the above-described embodiments of the first aspect of the surgical system, the handheld cable housing includes a socket sized and configured to receive the proximal end of the endoscopic shaft.

[0023] In any of the above-described embodiments of the first aspect of the surgical system, the handheld cable housing includes a lock configured to maintain the proximal end of the endoscopic shaft within the socket of the handheld cable housing.

[0024] In any of the above-described embodiments of the first aspect of the surgical system, the lock is biased in a locked configuration.

[0025] In any of the above-described embodiments of the first aspect of the surgical system, the handheld cable housing includes a release lever that selectively configures the lock to an unlocked configuration for releasing the proximal end of the endoscopic shaft within the socket of the handheld cable housing.

[0026] In any of the above-described embodiments of the first aspect of the surgical system, the handheld cable housing includes a data interface configured to receive image data from an image sensor and angular offsets from the one or more angular position sensors.

[0027] In any of the above-described embodiments of the first aspect of the surgical system, the proximal end of the endoscopic shaft includes a corresponding data interface configured to supply image data from an image sensor and angular offsets from the one or more angular position sensors to the handheld cable housing.

[0028] In any of the above-described embodiments of the first aspect of the surgical system, the handheld cable housing includes a connector cable configured to transmit image data and angular offset data to an external device.

[0029] In any of the above-described embodiments of the first aspect of the surgical system, the proximal end of the endoscopic shaft includes a housing configured to remain fixed relative to the handheld cable housing. The proximal end of the endoscopic shaft further includes a rotatable interface configured to facilitate rotation of the endoscopic shaft relative to the handheld cable housing.

[0030] In any of the above-described embodiments of the first aspect of the surgical system, the distal end of the endoscopic shaft includes an optical assembly positioned to receive light incident on the distal face of the endoscopic shaft.

[0031] In any of the above-described embodiments of the first aspect of the surgical system, the distal face is oriented at an angle to the endoscope axis.

[0032] In any of the above-described embodiments of the first aspect of the surgical system, the angle is any angle between 0° and 90°.

[0033] In any of the above-described embodiments of the first aspect of the surgical system, the optical assembly includes one or more lenses for guiding light incident on the distal face along an optical path to an image sensor.

[0034] In any of the above-described embodiments of the first aspect of the surgical system, the field of view direction of the endoscope axis is configured to change in response to rotation of the endoscope axis relative to the handheld cable housing.

[0035] A second aspect of the present disclosure includes a method that includes defining an image horizontal line relative to the orientation of a surgical system. The surgical system includes a handheld cable housing and an axis, with a user input device positioned on the handheld cable housing and an image sensor positioned at a distal end of the axis. The axis is coupled to the handheld cable housing such that the axis is rotatable relative to the handheld cable housing. The method includes determining an angular offset between the image horizontal line and the field of view direction of the axis. The method includes transmitting image data captured by the image sensor and transmitting angular offset data indicative of the angular offset, the angular offset data being for rotation of the image data.

[0036] In some embodiments of the second aspect of the present disclosure, the angular offset is measured by one or more angular position sensors of the surgical system.

[0037] In any of the above-described embodiments of the second aspect of the method, the one or more angular position sensors are positioned at a coupling portion between the axis and the handheld cable housing.

[0038] In any of the above-described embodiments of the second aspect of the method, the image horizontal line is defined as the horizontal midplane of the handheld cable housing at the coupling portion.

[0039] In any of the above-described embodiments of the second aspect of the method, the image horizontal line is defined relative to the handheld cable housing.

[0040] In any of the above-described embodiments of the second aspect of the method, the angular offset is a measure of the angular rotation of the axis relative to the handheld cable housing.

[0041] In any of the above embodiments of the second aspect of the method, the angular offset is a measure of the angular rotation between the horizontal image line and the viewing direction of the image sensor.

[0042] In any of the above embodiments of the second aspect of the method, the horizontal image line is defined as being orthogonal to the vertical midplane of the handheld cable housing and parallel to the longitudinal axis of the handheld cable housing.

[0043] In any of the above embodiments of the second aspect of the method, the horizontal image line is defined relative to an axis.

[0044] In any of the above embodiments of the second aspect of the method, the angular offset is a measure of the angular rotation of the handheld cable housing relative to the axis.

[0045] In any of the above embodiments of the second aspect of the method, the horizontal image line is defined as the horizontal midplane of the axis at the coupling portion.

[0046] In any of the above embodiments of the second aspect of the method, the horizontal image line is defined as being orthogonal to the vertical midplane of the axis and parallel to the longitudinal axis of the axis.

[0047] In any of the above embodiments of the second aspect of the method, the horizontal image line is defined based on the sensed direction of gravity.

[0048] In any of the above embodiments of the second aspect of the method, the one or more angular position sensors are positioned on the axis.

[0049] In any of the above embodiments of the second aspect of the method, the one or more angular position sensors are positioned at the handheld cable housing.

[0050] In any of the above embodiments of the second aspect of the method, the one or more angular position sensors include one or more sensors selected from the group consisting of Hall effect sensors, mechanical encoders, optical encoders, magnetic encoders, electromagnetic induction encoders, encoders, rotary potentiometers, resolvers, gravity sensors, gyroscopes, magnetometers, and linear acceleration sensors.

[0051] In any of the above embodiments of the second aspect of the method, the user input device is positioned on the control surface of the handheld cable housing.

[0052] In any of the above embodiments of the second aspect of the method, the control surface is the top surface of the handheld cable housing.

[0053] In any of the above-described embodiments of the second aspect of the method, the user input device is selected from the group consisting of physical buttons, capacitive sensing buttons, soft buttons on a touch screen, switches, touch pads, rollers, and directional keys.

[0054] In any of the above-described embodiments of the second aspect of the method, the handheld cable housing includes a socket sized and configured to receive the proximal end of the shaft.

[0055] In any of the above-described embodiments of the second aspect of the method, the handheld cable housing includes a lock configured to maintain the proximal end of the shaft within the socket of the handheld cable housing.

[0056] In any of the above-described embodiments of the second aspect of the method, the lock is biased in a locked configuration.

[0057] In any of the above-described embodiments of the second aspect of the method, the handheld cable housing includes a release lever that selectively configures the lock to an unlocked configuration for releasing the proximal end of the shaft within the socket of the handheld cable housing.

[0058] In any of the above-described embodiments of the second aspect of the method, the handheld cable housing includes a data interface configured to receive image data and angular offset data.

[0059] In any of the above-described embodiments of the second aspect of the method, the proximal end of the shaft includes a corresponding data interface. The method includes transmitting image data and angular offset data from the shaft to the handheld cable housing.

[0060] In any of the above-described embodiments of the second aspect of the method, the handheld cable housing includes a connector cable. The method includes transmitting image data and angular offset data from the handheld cable housing to an external device.

[0061] A third aspect of the present disclosure includes a method that includes receiving image data captured by an image sensor positioned at a distal end of a shaft. The shaft is coupled to a handheld cable housing. A user input device is positioned on the handheld cable housing. The shaft is rotatable relative to the handheld cable housing. The method includes receiving angular offset data indicative of an angular offset relative to a defined image horizontal line. The method includes generating rotated image data based on the angular offset data. The method includes causing the rotated image data to be displayed.

[0062] In some embodiments of the third aspect of the present disclosure, the method includes receiving a control signal in response to a selection of the user input device. The control signal sets the defined image horizontal line to one of a plurality of image horizontal lines.

[0063] In any of the above-described embodiments of the third aspect of the present disclosure, the method includes supplying light from a lighting source to a handheld cable housing.

[0064] In any of the above-described embodiments of the third aspect of the present disclosure, the method includes receiving a second control signal in response to a selection of a user input device. The second control signal causes the light from the lighting source to change.

[0065] In any of the above-described embodiments of the third aspect of the present disclosure, the second control signal causes the light from the lighting source to turn off or change frequency.

[0066] In any of the above-described embodiments of the third aspect of the present disclosure, causing the display of the rotated image data includes transmitting the rotated image data to an external display.

[0067] In any of the above-described embodiments of the third aspect of the present disclosure, the defined image horizontal line is with respect to the handheld cable housing.

[0068] In any of the above-described embodiments of the third aspect of the present disclosure, the angular offset is a measurement of the angular rotation of the axis with respect to the handheld cable housing.

[0069] In any of the above-described embodiments of the third aspect of the present disclosure, the angular offset is a measurement of the angular rotation between the defined image horizontal line and the field of view direction of the image sensor.

[0070] In any of the above-described embodiments of the third aspect of the present disclosure, the defined image horizontal line is the horizontal midline plane of the handheld cable housing at the coupling portion between the axis and the handheld cable housing.

[0071] In any of the above-described embodiments of the third aspect of the present disclosure, the defined image horizontal line is orthogonal to the vertical midline plane of the handheld cable housing and parallel to the longitudinal axis of the handheld cable housing.

[0072] In any of the above-described embodiments of the third aspect of the present disclosure, the defined image horizontal line is with respect to the axis.

[0073] In any of the above-described embodiments of the third aspect of the present disclosure, the angular offset is a measurement of the angular rotation of the handheld cable housing with respect to the axis.

[0074] In any of the above-described embodiments of the third aspect of the present disclosure, the defined image horizontal line is the horizontal midline plane of the axis at the coupling portion between the axis and the handheld cable housing.

[0075] In any of the above-described embodiments of the third aspect of the present disclosure, the defined image horizontal line is orthogonal to the vertical median plane of the axis and parallel to the longitudinal axis of the axis.

[0076] In any of the above-described embodiments of the third aspect of the present disclosure, the defined image horizontal line is based on the sensed direction of gravity.

[0077] In any of the above-described embodiments of the third aspect of the present disclosure, one or more angular position sensors for measuring angular offset are positioned at the axis.

[0078] In any of the above-described embodiments of the third aspect of the present disclosure, one or more angular position sensors for measuring angular offset are positioned at the handheld cable housing.

[0079] In any of the above-described embodiments of the third aspect of the present disclosure, one or more angular position sensors configured to measure angular offset are selected from the group consisting of Hall effect sensors, mechanical encoders, optical encoders, magnetic encoders, electromagnetic induction encoders, encoders, rotary potentiometers, resolvers, gravity sensors, gyroscopes, magnetometers, and linear acceleration sensors.

[0080] In any of the above-described embodiments of the third aspect of the present disclosure, the user input device is positioned on the control surface of the handheld cable housing.

[0081] In any of the above-described embodiments of the third aspect of the present disclosure, the control surface is the top surface of the handheld cable housing.

[0082] In any of the above-described embodiments of the third aspect of the present disclosure, the user input device is selected from the group of user input devices consisting of physical buttons, capacitive sensing buttons, soft buttons on a touch screen, switches, touch pads, rollers, and direction keys.

[0083] These and other features will be more clearly understood from the following detailed description in conjunction with the drawings and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] To more fully understand the present disclosure, reference is now made to the following drawings, which are incorporated herein by reference and in which like reference numerals represent like parts in conjunction with the detailed description.

[0085] Figure 1 is a plan view of a minimally invasive remote operation surgical system.

[0086] Figure 2 is a perspective view of a user control system.

[0087] Figure 3It is a perspective view of an electronic device cart.

[0088] Figure 4 It is a schematic example of a remote operation surgical system.

[0089] Figure 5 It is a cross-sectional view of a chip-in-tip (CIT) endoscope image capture device in a tip.

[0090] Figures 6A - 6B It shows an endoscope image capture device with a rotatable endoscope assembly, which is positioned to capture images from different fields of view.

[0091] Figure 7 It is a block diagram of an endoscope element with a rotatable endoscope assembly connected to a fixed hand-held cable housing.

[0092] Figures 8A - 8B It shows an endoscope image capture device with different options for defining an image horizontal line.

[0093] Figure 9 It is a diagram of an image processing operation for maintaining horizontal image alignment.

[0094] Figure 10A It is a cross-sectional view of an exemplary endoscope, which shows a coupling part between a fixed hand-held cable housing and a rotatable endoscope assembly.

[0095] Figure 10B It is Figure 10A an exploded view of a rotatable endoscope assembly of

[0096] Figure 10C It is Figure 10A a perspective view of a rotatable endoscope assembly of , which shows details of an angular position sensor for measuring rotation between the rotatable endoscope assembly and the fixed hand-held cable housing.

[0097] Figure 11 It is an operation flow chart of an image capture device according to various embodiments described herein.

[0098] Figure 12 It is an operation flow chart of an image processor according to various embodiments described herein.

[0099] Figure 13 It illustrates an exemplary computer system. Detailed Description

[0100] First, it should be understood that although exemplary embodiments of one or more embodiments are illustrated below, the disclosed systems and methods can be implemented using many different techniques, whether currently known or existing. The present disclosure should in no way be limited to the exemplary embodiments, drawings, and techniques illustrated below, but may be modified within the scope of the appended claims and the full scope of their equivalents. The use of the phrase "and / or" indicates that any one or any combination from a list of options can be used. For example, "A, B, and / or C" means "A" or "B" or "C" or "A and B" or "A and C" or "B and C" or "A and B and C".

[0101] Elements described in detail with reference to one embodiment, implementation, or application may in actuality be included in other embodiments, implementations, or applications in which they are not specifically shown or described. For example, if an element is described in detail with reference to one embodiment and not described with reference to a second embodiment, the element may still be required to be included in the second embodiment. Thus, to avoid unnecessary repetition in the following description, one or more elements shown and described in connection with one embodiment, implementation, or application may be incorporated into other embodiments, implementations, or aspects, unless otherwise specifically described, unless the one or more elements render the implementation or embodiment inoperative, or unless two or more of the elements provide conflicting functions.

[0102] Some embodiments are described using the da Vinci TM surgical system (such as the da Vinci TM Xi TM surgical system) commercialized by Intuitive Surgical, Inc. of Sunnyvale, California. However, those skilled in the art will understand that the inventive aspects disclosed herein can be embodied and implemented in various ways - including robotic as well as (where applicable) non-robotic embodiments and implementations. Embodiments on the Da Vinci TM surgical system are merely exemplary and should not be considered as limiting the scope of the inventive aspects disclosed herein.

[0103] According to various aspects, the present disclosure describes a system for maintaining image level in a chip-in-the-tip (CIT) endoscopic image capture device in a tip of a rotatable endoscopic assembly having a rigid or flexible shaft (e.g., a rotatable endoscope shaft). The distal end of the shaft of the rotatable endoscopic assembly includes a camera tip with imaging optics and one or more image sensors, collectively or individually referred to as image sensors. The rotatable endoscopic assembly is (removably or fixedly) coupled to a handheld cable housing and is rotatable relative thereto.

[0104] The rotatable endoscope assembly is moved by an alignment wheel or a joystick attached to the shaft. For example, the rotatable endoscope assembly can rotate at an angle of + / -180° relative to a midpoint position, rotate at an angle of +360° relative to a starting position, rotate at an angle of -360° relative to an end position, or rotate relative to any other intermediate position between the starting position and the end position or at any subset of angles. In some embodiments, the rotatable endoscope assembly does not include a starting or an end position, such that the rotatable endoscope assembly can rotate indefinitely in either direction (e.g., can rotate continuously).

[0105] When the rotatable endoscope assembly rotates relative to the handheld cable housing, one or more angular position sensors (collectively or individually referred to as angular position sensors) measure the angular offset relative to a defined horizontal line. The horizontal line can be defined relative to the orientation of the handheld cable housing, relative to the gravity sensed at the distal tip of the rotatable endoscope assembly (e.g., at the camera tip), relative to the gravity sensed at the fixed proximal end of the rotatable endoscope assembly connected to the handheld cable housing, relative to the gravity sensed by the handheld cable housing, and / or relative to a user-defined horizontal line.

[0106] The handheld cable housing includes one or more control buttons, collectively or individually referred to as control buttons. The control buttons are physical buttons, capacitive sensing buttons, soft buttons on a touch screen, switches, touch pads, rollers, arrow keys, or any other user input device. The control buttons are positioned on a control surface of the handheld cable housing such that the control buttons can be easily accessed even when the rotatable endoscope assembly rotates relative to the handheld cable housing. This is in contrast to an endoscopic image capture device in which one or more control buttons may be positioned at the proximal end of the rotatable endoscope assembly. In such a system, the control buttons rotate with the rotatable endoscope assembly, such that access to the control buttons changes as the rotatable endoscope assembly rotates, and additional flexibility is required to activate the control buttons when the position of the control buttons changes over time.

[0107] Thus, according to the present disclosure, access to the control buttons remains unchanged even when the direction-of-view (DOV) of the rotatable endoscope assembly changes after rotation. For example, when the control surface is positioned on the top surface of the handheld cable housing, the handheld cable housing can be held in a vertical orientation to achieve optimal visibility of the control surface and accessibility to the control buttons, thereby supporting single-handed control without wrist torsion, which has ergonomic advantages. In other embodiments, the control surface is positioned on one or more other surfaces of the handheld cable housing, such as a side surface, a gripping portion of the handheld cable housing, a surface extending from the top of the gripping portion of the handheld cable housing, or any other surface on or extending from the handheld cable housing.

[0108] The rotatable endoscope assembly receives power and illumination from a hand-held cable housing. The rotatable endoscope assembly includes an optical fiber bundle having one or more optical fibers configured to transmit light received from the hand-held cable housing to a camera tip to illuminate a scene being imaged by an image sensor, such as in a diagnostic or surgical procedure. Optionally, the scene may be illuminated by light provided by a single optical fiber, a phosphor conversion layer at the camera tip, multiple single optical fibers transmitting individual or combined light wavelengths, or one or more illumination sources (e.g., light emitting diodes (e.g., white or color multiplexed) positioned at the camera tip). The rotatable endoscope shaft supplies still or video images captured by the image sensor and one or more signals indicating angular offset measurements by an angular position sensor to the hand-held cable housing.

[0109] The hand-held cable housing includes a flexible cable with a second optical fiber bundle having one or more optical fibers. The cable includes a connector configured to couple the second optical fiber bundle to a light source. In some embodiments, the light source is positioned external to the hand-held cable housing. In some embodiments, the light source is positioned within the hand-held cable housing. Images captured by the image sensor in the camera tip are transmitted to the hand-held cable housing via a wired or wireless connection or an optical connection and, in turn, to the connector via a wired or wireless connection or an optical connection in the flexible cable.

[0110] A control surface with control buttons is positioned between the flexible cable and the rotatable endoscope assembly. Upon selection of a control button, a control signal is transmitted to the connector via a wired or wireless connection or an optical connection in the flexible cable. The control signal provides instructions to turn on or off the illumination source, capture a still image from the image sensor, start / stop video recording, define an image horizontal line, turn on or off horizontal image alignment, or perform any other control function for operating the endoscope image capture device.

[0111] A controller system (such as an electronic equipment cart) includes a socket configured to receive the connector. The controller system includes a light source coupled to the socket and configured to provide light to the second optical fiber bundle in the flexible cable. Optionally, the controller system includes a power source for generating illumination in the hand-held cable housing or at the camera tip, such as via one or more light emitting diodes. The controller system also includes an image processor coupled to the socket and configured to receive the images and angular offset measurements transmitted via an electrical or optical connection in the flexible cable. In some embodiments, the angular offset is encoded as metadata within the video source or still image. In some embodiments, the angular offset is transmitted as a separate file that may include a timestamp associated with the angular offset or a reference to a video frame or still image.

[0112] The controller system is coupled to a local and / or remote monitor and is configured to display images processed by an image processor. The controller system is further configured to receive and process control signals, such as via the image processor or another processor. For example, the controller system operates to turn on / off a light source, store still images from an image sensor, store video data, and / or store defined horizontal lines.

[0113] The image processor is configured to rotate the received image based on the received angle offset measurement to maintain a constant image level in the image displayed on the monitor (e.g., perform horizontal image alignment). Thus, even if the image provided by the image sensor rotates as the DOV of the rotatable endoscope assembly changes, the image processor aligns the image to a defined image level before displaying it on the monitor.

[0114] Although the various examples provided herein are described with respect to an endoscopic image capture device, the present disclosure is not limited thereto but is intended to cover any device coupled to a controller system that is configured to rotate a received image based on a received angle offset measurement to maintain a constant image horizontal line in the image displayed on the monitor. Similarly, the present disclosure is intended to cover any image capture device coupled to the controller system, and the control system is used to process the images captured by the image capture device. For example, the present disclosure may equally apply to a borescope or other similar inspection cameras.

[0115] Now referring to the drawings, in which like reference numerals represent like parts throughout the several views, Figure 1 is a plan view of a minimally invasive remote operating surgical system 10, which is generally used to perform minimally invasive diagnostic or surgical procedures on a patient 12 lying on a mobile operating table 14. The system includes a user control system 16, such as a mobile surgeon's console used by a surgeon 18 during the procedure. One or more assistants 20 may also participate in the procedure. The minimally invasive remote operating surgical system 10 further includes a manipulation system 22, such as a mobile patient-side cart and a mobile electronics cart 24. In some embodiments, the mobile operating table 14, the user control system 16, the manipulation system 22, and the electronics cart 24 are all mounted on wheels to provide mobility.

[0116] The manipulation system 22 or other such manipulation systems include a plurality of segmented mechanical support arms 72, each mechanical support arm having an end rotatably mounted to a vertical support structure 74 and having another end for removably coupling a surgical instrument 26. In some embodiments, each mechanical support arm 72 includes a first section 72-1, a second section 72-2, and a third section 72-3. During procedure preparation, the multiple sections of at least one support arm 72 are moved to position the surgical instrument for insertion into a minimally invasive incision in the patient 12.

[0117] During the procedure, while the instrument is inserted into the patient's body cavity, the surgeon 18 views the surgical site through the user control system 16. Images of the surgical site can be obtained through an endoscope 28, such as a stereoscopic endoscope, and the endoscope 28 can be manipulated by a manipulation system 22 to orient the endoscope 28. In some embodiments, the manipulation system 22 can manipulate a rotatable endoscope assembly (e.g., a rotatable endoscope shaft) to change the viewing direction of the endoscope 28.

[0118] In some embodiments, the endoscope 28 can be implemented as the above-described endoscope image capture device having a rotatable endoscope assembly having a distal end including a CIT image sensor and a proximal end (releasably or fixedly) coupled to a handheld cable housing such that the rotatable endoscope assembly can rotate relative to the handheld cable housing. When the rotatable endoscope assembly rotates relative to the handheld cable housing, the endoscope 28 captures video or still image data and uses one or more angular position sensors to measure the angular offset relative to a defined horizontal line.

[0119] In some embodiments, the surgeon 18 manually manipulates the endoscope 28 within the patient's body cavity. The surgeon 18 views the surgical site through a monitor, such as a monitor on the electronic equipment cart 24 or another monitor external to the electronic equipment cart 24. In such an embodiment, the surgeon 18 manually manipulates the rotatable endoscope assembly (e.g., via an alignment wheel or joystick attached thereto) to change the viewing direction of the endoscope 28 while maintaining ergonomic access to the control buttons on the handheld cable housing. The surgeon 18 can also manipulate the control functions on the endoscope 28 via the manipulation control buttons to change one or more operating functions of the electronic equipment cart 24 (e.g., turning on or off the illumination, changing the light source, etc.) or the image processing functions performed by the electronic equipment cart 24 (e.g., rotating the received image to the image horizontal line, capturing a still image, etc.).

[0120] The (one or more) computer processors located on the electronic equipment cart 24 can be used to process the images of the surgical site for subsequent display to the surgeon 18 through the user control system 16 or another display (such as another display on the electronic equipment cart 24). The (one or more) computer processors can alternatively be referred to herein as an image processor or a video processor. More generally, the image processor or video processor referred to throughout this disclosure refers to any processor capable of performing the image or video processing functions described herein, such as including a general-purpose processor, a graphics processor, a video processor, an image processor, or an application-specific integrated circuit. The image processor is configured to rotate the received image based on the angular offset measurements received from the endoscope 28 to maintain a constant image horizontal line in the image displayed on the user control system 16 or another display.

[0121] One or more light sources or illuminators may also be provided on the electronic device cart 24 to provide light for illuminating the surgical site used by the endoscope 28. The illuminator may include a white light source, a colored light source (e.g., red, green, blue, cyan, magenta, yellow, etc.), an infrared light source, a laser light source, or any other type of light source or a combination thereof. Different illuminators may be used at different time points during a surgical or diagnostic procedure. For example, the electronic device cart 24 may be controlled via the user control system 16 or a selection on the endoscope 28 (e.g., via a control button) to provide light from a first set of one or more illuminators at a first time and light from a second set of one or more illuminators at a second time.

[0122] The number of single-use surgical instruments 26 will generally depend on factors such as the diagnostic or surgical procedure and the space limitations in the operating room. If one or more of the surgical instruments 26 in use need to be replaced during the procedure, the assistant 20 may remove the surgical instrument 26 from the manipulation system 22 and replace it with another surgical instrument 26 from the tray 30 in the operating room.

[0123] Figure 2 is a perspective view of the user control system 16. The user control system 16 includes a display area 31 having a left-eye display 32 and a right-eye display 34 for presenting a coordinated stereoscopic view of the surgical site enabling depth perception to the surgeon 18.

[0124] The user control system 16 further includes one or more control inputs 36. One or more surgical instruments on the manipulation system 22 (as shown in Figure 1 move in response to manipulation of the one or more control inputs 36 by the surgeon 18. The control inputs 36 may provide the same mechanical degrees of freedom as the associated surgical instrument 26 (as shown in Figure 1 to provide the surgeon 18 with a sense of telepresence or that the control inputs 36 are integral with the instrument 26, such that the surgeon has a strong sense of directly controlling the instrument 26. To this end, position, force, and tactile feedback sensors (not shown) may be employed to transmit position, force, and touch from the surgical instrument 26 back to the surgeon's hand through the control inputs 36. The height of the control inputs 36 may be adjusted by a height adjustment joystick 38.

[0125] The user control system 16 is generally located in the same room as the patient, so that the surgeon can directly monitor the procedure, be present in person if necessary, and directly communicate with the assistant on the patient side, rather than through a phone or other communication medium. However, the surgeon may be located in a different room from the patient, a completely different building, or other remote location, thus allowing for a remote surgical procedure.

[0126] Figure 3 is a perspective view of the electronic device cart 24. The electronic device cart 24 may be coupled to the endoscope 28 and includes a computer processor for processing the captured images for subsequent display, such as to the user control system 16 or a surgeon on another suitable display located locally and / or remotely. For example, if a stereoscopic endoscope is used, the computer processor on the electronic device cart 24 may process the captured images to provide the surgeon with coordinated stereoscopic images of the surgical site. Such coordination may include alignment between the relative images and may also include adjusting the stereoscopic working distance of the stereoscopic endoscope.

[0127] As another example, image processing may include rotating the received images based on the received angle offset measurements to maintain a constant image horizontal line in the images displayed on the display 25 of the electronic device cart 24 or on the display area 31 of the user control system 16.

[0128] Optionally, the devices in the electronic device cart 24 may be integrated into the user control system 16 or the manipulation system 22, or they may be distributed at various other locations in the operating room. More generally, the electronic device cart 24 or the user control system 16 having integrated devices from the electronic device cart 24 may be referred to herein as a controller system for receiving angle offset measurements and rotating images from an image capture device to maintain a constant display image horizontal line.

[0129] Figure 4 Schematically illustrates a remote operating surgical system 50 (such as Figure 1 the minimally invasive remote operating surgical system 10). A surgeon may use a user control system 52 (such as Figure 1 the user control system 16 in Figure 1 to control a manipulation system 54 (such as Figure 1 the manipulation system 22 in

[0130] during a minimally invasive surgical procedure. The manipulation system 54 may use an image capture device (such as a stereoscopic endoscope) to capture images of the surgical site and output the captured images to a computer processor located on an electronic device cart 56 (such as

[0131] the electronic device cart 24 in Figure 1The endoscope 28). The endoscope 28 captures images of the surgical site and outputs the captured images to a computer processor located on the electronic device cart 56.

[0132] In one aspect, the computer processor can process the captured images in a variety of ways before any subsequent display. For example, the computer processor can use angular offset measurements to rotate the images from the image capture device to maintain a constant display image horizontal line. Additionally or alternatively, the captured images can be processed by a computer processor located outside the electronic device cart 56.

[0133] In one aspect, the remote operating surgical system 50 includes an optional computer processor 58 (shown in dashed lines), which is similar to the computer processor located on the electronic device cart 56, and the manipulation system 54 outputs the captured images to the computer processor 58 for image processing and then displays them on the user control system 52. In another aspect, the captured images are first processed by the computer processor on the electronic device cart 56, and then additional image processing is performed by the computer processor 58, and then displayed on the user control system 52 or the display 60.

[0134] The remote operating surgical system 50 can include an optional display 60, as indicated by the dashed lines. The display 60 is coupled to the computer processor located on the electronic device cart 56 and the computer processor 58, and the captured images processed by these computer processors can be displayed on the display 60 in addition to being displayed on the display of the user control system 52. In various embodiments, the display 60 can be located on the electronic device cart 56, such as the display 25 on the electronic device cart 24. In some embodiments, the display 60 can be separated from the user control system 52 and the electronic device cart 58.

[0135] Figure 5 is a cross-sectional view of a chip-in-tip (CIT) image capture device 500 according to various embodiments. The CIT image capture device 500 is positioned at the distal end 502 of the axis 504 of the CIT image capture device 500. In some embodiments, the CIT image capture device 500 is used in the above-mentioned endoscope 28. The CIT image capture device 500 includes an optical component 506, which is positioned to receive light incident on the distal face 508 of the CIT image capture device 500.

[0136] In the illustrated example, the distal face 508 is formed at an angle 514 (e.g., 10°, 20°, 30°, 45°, or any other desired angle from 0° to 90°) between a plane 511 orthogonal to the axis 504 and a plane 513 parallel to the distal face 508. In some embodiments, the distal face 508 is orthogonal to the axis 504 (i.e., the angle 514 is 0°). In some embodiments, the distal face 508 is parallel to the axis 504 (i.e., the angle 514 is 90°).

[0137] The optical assembly 506 includes one or more lenses for guiding incident light along an optical path to the image sensor 510. The image sensor 510 captures still images and / or video images of a scene (e.g., a surgical site) and transmits the captured images along a wired or wireless communication path 512.

[0138] In Figure 5 the illustrated example, the image sensor 510 is shown as a single image sensor. In some embodiments, more than one image sensor may be positioned within the optical path. In some embodiments, the CIT image capture device 500 includes multiple optical paths, each optical path guiding light to one or more image sensors.

[0139] For example, the CIT image capture device 500 may include a stereoscopic capture device having a left optical path and a right optical path and one or more image sensors positioned along each of the left and right optical paths. In some embodiments, a different number of image sensors may be used on each of the left and right optical paths.

[0140] In Figure 5 the illustrated example, the communication path 512 is a wired communication path within the axis 504 of the CIT image capture device 500. For example, the wired communication path 512 may have wires, wire harnesses, cables, shielded cables, flat flex cables, or any other wired communication path.

[0141] In some embodiments, the CIT image capture device 500 includes an illumination element 515, such as a light pipe, a single optical fiber, or an optical fiber bundle. The illumination element 515 guides light from an illumination source to illumination optics 516 in the distal end 502 of the axis 504 of the CIT image capture device 500. The illumination optics 516 guides light from the illumination source to illuminate the scene captured by the image sensor 510. Although Figure 5The example shows a single illumination element 515, but it is contemplated that multiple illumination elements may be present for providing light from multiple illumination sources. In some embodiments, light provided by a single optical fiber, a phosphor conversion layer at the tip of the camera, multiple single optical fibers that transmit individual or combined light wavelengths, or one or more illumination sources (such as light-emitting diodes positioned at the tip of the camera (e.g., white or color multiplexed)) may illuminate the scene being captured by the image sensor 510.

[0142] Each of the feature portions of the CIT image capture device 500 described above can be used alone or in combination with another or other feature portions described throughout this disclosure. Various modifications and additions to the CIT image capture device 500 are readily discernible by those of ordinary skill in the art and are also contemplated by this disclosure. For example, this disclosure contemplates alternative illumination, filtering, optical components, focal length manipulation, and image sensor feature portions known to those of ordinary skill in the art.

[0143] Figures 6A - 6B An endoscopic image capture device 600 is shown that has a rotatable endoscopic assembly 601 (e.g., a rotatable endoscopic shaft) positioned to capture images from different fields of view (FOV). In various embodiments, the endoscopic image capture device 600 is used as the endoscopic 28 described above.

[0144] As Figure 6A shown, the endoscopic image capture device 600 is shown to have a rotatable endoscopic assembly 601 positioned to capture images from a first FOV. Figure 6B The endoscopic image capture device 600 is shown to have a rotatable endoscopic assembly 601 positioned to capture images from a second FOV 605. In the shown example, the second FOV 605 is 180° different from the first FOV 604, however different FOVs can be at any angle.

[0145] The rotatable endoscopic assembly 601 includes a CIT image capture device 606, such as the CIT image capture device 500 described above. The rotatable endoscopic assembly 601 includes a shaft 602, such as the shaft 504 described above. The shaft 602 is a rigid shaft, a flexible shaft, a partially flexible shaft, a manipulable flexible shaft, a manipulable rigid shaft, or any combination thereof. In some embodiments, the CIT image capture device 606 is positioned at the manipulable tip of the rotatable endoscopic shaft 602.

[0146] The rotatable endoscopic assembly 601 is coupled to a handheld cable housing 608 and is rotatable relative thereto. The rotatable endoscopic assembly 601 includes a joystick 610 to facilitate rotation of the rotatable endoscopic assembly 601 relative to the handheld cable housing 608.

[0147] In the illustrated example, the joystick 610 is an alignment wheel with multiple ergonomic protrusions to facilitate rotation by the user via the viewfinder or thumb. In some embodiments, the joystick 610 can simply be a single arm extending from the rotatable endoscope assembly 601 for providing a lever force to rotate the rotatable endoscope assembly 601. The present disclosure contemplates other variations of the joystick 610.

[0148] The rotatable endoscope assembly 601 can rotate relative to a reference position (e.g., the original position). For example, the rotatable endoscope assembly 601 can rotate at an angle of + / - 180° relative to a midpoint position, rotate at an angle of +360° relative to a starting position, rotate at an angle of -360° relative to an ending position, or place the reference position at any other intermediate position between the starting position and the ending position and can rotate across any subset of the angle. In some embodiments, the rotatable endoscope assembly 601 does not include a stop such that the rotatable endoscope assembly can rotate indefinitely in either direction (e.g., can rotate continuously).

[0149] In some embodiments, the joystick 610 includes a physical feature indicating the reference position. In Figure 6A and Figure 6B the illustrated example, the protrusions of the joystick 610 form a pentagon, where the central protrusion of the pentagon indicates the reference position. The present disclosure contemplates other physical features for indicating the reference position, such as notches, lines, colored stripes, or the like on the joystick 610 or the rotatable endoscope shaft 602.

[0150] In some embodiments, the handheld cable housing 608 also includes a physical feature indicating the reference position. For example, the handheld cable housing 608 can include a notch, line, or colored stripe corresponding to the physical feature on the joystick 610.

[0151] In some embodiments, the rotatable endoscope assembly 601 includes a focus wheel 612 to facilitate manipulation of the focal length of the endoscope image capture device 600. For example, referring Figure 5 , the focus wheel 612 can manipulate a focus lens 524 within the optical path of the optical component 506 to change the focal length of the CIT image capture device 500 in response to manipulation of the focus wheel 612.

[0152] The handheld cable housing 608 includes a coupling portion 614 that couples the handheld cable housing 608 to the rotatable endoscope assembly 601. The coupling portion 614 facilitates the wired and / or wireless transmission of data and power between the rotatable endoscope assembly 601 and the handheld cable housing 608. For example, the coupling portion 614 can supply power from the handheld cable housing 608 to power the CIT image capture device 606 in the rotatable endoscope assembly 601. Similarly, the coupling portion 614 facilitates the wired and / or wireless transmission of video or still image data from the CIT image capture device 606 to the handheld cable housing 608.

[0153] In some embodiments, the coupling portion 614 facilitates the transmission of light from a light source in the handheld cable housing 608 to the rotatable endoscope assembly 601. For example, the rotatable endoscope assembly 601 includes an illumination element (not shown), such as illumination element 515 (e.g., a light pipe or fiber optic bundle), to direct light from the handheld cable housing 608 through the coupling portion 614 to the CIT image capture device 606. For example, the CIT image capture device 606 includes illumination optics (such as illumination optics 516 described above) to direct light from the handheld cable housing 608 to illuminate the scene being captured by the CIT image capture device 606.

[0154] In some embodiments, the coupling portion 614 is a releasable coupling such that the endoscope assembly 601 can be removably removed from the handheld cable housing 608. Accordingly, each of the rotatable endoscope assembly 601 and the handheld cable housing 608 can be cleaned separately. Additionally, different rotatable endoscope assemblies 601 can be attached to the handheld cable housing 608, such as rotatable endoscope assemblies 601 having different angles for angle 514 or having different tools, features, or functions. For example, the rotatable endoscope assembly 601 can be a 30-degree laparoscope, a zero-degree laparoscope, a 30-degree cystoscope, or any other such tool.

[0155] In some embodiments, the coupling portion 614 is a fixed coupling such that the endoscope assembly 601 is not removable from the handheld cable housing 608.

[0156] The handheld cable housing 608 includes a grip portion 616. In some embodiments, the grip portion 616 can include contours or other ergonomic features to facilitate the use and operation of the endoscope image capture device 600. In embodiments where the rotatable endoscope shaft 602 is removably attached to the handheld cable housing 608, the grip portion 616 can additionally include a release button (not shown) for releasing a locking mechanism (not shown) that holds the rotatable endoscope shaft 602 to the handheld cable housing 608.

[0157] The handheld cable housing 608 includes a control surface 618 on which one or more control buttons (not shown) are positioned. The control buttons are one or more of physical buttons, soft buttons on a touchscreen, switches, touch pads, rollers, directional keys, or any other user input device. The control surface 618 of the handheld cable housing 608 is positioned such that when the user holds the grip portion 616, the one or more control buttons are readily accessible even if the rotatable endoscope assembly 601 rotates relative to the handheld cable housing 608. That is, even if the rotatable endoscope assembly 601 rotates relative to the handheld cable housing 608, the control surface 618 with the one or more control buttons remains fixed.

[0158] In Figure 6A and Figure 6B the example shown, the control surface 618 is positioned on the top surface of the handheld cable housing 608 that extends away from the grip portion 616. However, the control surface 618 can be positioned at any location on the handheld cable housing 608, such as on a side surface, on a surface extending from the handheld cable housing 608 (e.g., extending parallel to the grip portion 616 to facilitate easy viewing of the control surface 618 when the user is positioned behind the grip portion 616), or any other surface of the handheld cable housing 608.

[0159] In some embodiments, the control surface 618 is a movable surface to allow the user to position the control surface 618 in a desired orientation to ergonomically activate the one or more control buttons thereon. For example, the control surface 618 can be tilted upward or otherwise oriented in different directions to ergonomically activate the one or more control buttons.

[0160] The handheld cable housing 608 includes a connector cable 620 that facilitates wired and / or wireless transmission of data and power between the handheld cable housing 608 and an external device (such as the electronic equipment cart 56 or the electronic equipment cart 24 discussed above). The connector cable 620 includes a socket (not shown) to facilitate coupling the handheld cable housing 608 to the external device. For example, the connector cable 620 supplies power received from the external device to the handheld cable housing 608. The connector cable 620 supplies data from the handheld cable housing 608 to the external device. For example, the handheld cable housing 608 supplies video or still image data from the CIT image capture device 606 to the external device. Similarly, after selecting a control button on the control surface 618, the handheld cable housing 608 supplies one or more control signals. For example, the control signal provides an instruction to turn on or off a light source, capture a still image from an image sensor, start / stop video recording, define an image horizontal line, turn on or off horizontal image alignment, or perform any other control function for operating the endoscope image capture device 600.

[0161] In some embodiments, the connector cable 620 facilitates the transmission of light from a lighting source in an external device to the handheld cable housing 608. For example, as described above, the handheld cable housing 608 includes lighting elements (not shown), such as light guides, individual optical fibers, or fiber optic bundles, to receive light from the connector cable 620 and direct the received light through the coupling portion 614 to the CIT image capture device 606. Similarly, the handheld cable housing 608 includes lighting elements (not shown), such as light guides, individual optical fibers, or fiber optic bundles, to receive light from an external device and direct the received light through a socket to the handheld cable housing 608. In some embodiments, the lighting source is located within the handheld cable housing 608.

[0162] In some embodiments, the lighting elements of the handheld cable housing 608 are an extension of the lighting elements of the connector cable 620. For example, after the handheld cable housing 608 is assembled to the connector cable 620, a portion of the lighting elements of the connector cable 620 extends into the handheld cable housing 608 to the coupling portion 614.

[0163] Figure 7 is a block diagram of elements of the endoscopic image capture device 800. In some embodiments, the endoscopic image capture device 800 is implemented as the endoscopic image capture device 600 described above, where like numerals represent like parts.

[0164] The endoscopic image capture device 800 has a rotatable endoscopic assembly 802 (e.g., a rotatable endoscopic shaft), such as the rotatable endoscopic assembly 601 described above, which is releasably or fixedly coupled to a fixed handheld cable housing 804, such as the handheld cable housing 608.

[0165] The rotatable endoscopic assembly 802 includes a rotatable endoscopic shaft 806 and a fixed distal housing 808. An image sensor 810, such as the CIT image capture device 606 or the CIT image capture device 500 described above, is positioned at the distal end of the rotatable endoscopic shaft 806. The fixed distal housing 808 is located at the proximal end of the rotatable endoscopic assembly 802. The fixed distal housing 808 is sized and configured to be releasably received within a socket 812 of the fixed handheld cable housing 804. The rotatable endoscopic shaft 806 is rotatable relative to the fixed distal housing 808 about a rotatable interface 814. The rotatable endoscopic assembly 802 and the fixed handheld cable housing 804 are releasably secured to each other by a locking mechanism 816. When the locking mechanism 816 is unlocked (e.g., after pressing a button (not shown) of the locking mechanism 816), the rotatable endoscopic assembly 802 is released from the fixed handheld cable housing 804.

[0166] AlthoughFigure 7 The example shown in Figure 7 provides that the rotatable endoscope assembly 802 is releasably coupled to the fixed hand-held cable housing 804. However, in some embodiments, the rotatable endoscope assembly 802 may be fixedly coupled to the hand-held cable housing 804. In this way, the fixed distal housing 808 and the locking mechanism 816 may be omitted, and the rotatable interface 814 of the rotatable endoscope assembly 802 is directly coupled to the hand-held cable housing 804 via the socket 812.

[0167] The endoscope image capture device 800 includes one or more angular position sensors that measure the angular offset relative to a defined image horizontal line when the rotatable endoscope assembly 802 rotates relative to the hand-held cable housing 804.

[0168] In some embodiments, the defined image horizontal line is defined relative to the rotatable endoscope assembly 802 or the hand-held cable housing 804. Thus, the first angular position sensor is located at the first angular position sensor location 818 at the coupling portion 614 and is configured to measure the angular offset of the rotatable endoscope assembly 802 relative to the hand-held cable housing 804. As shown, the first angular position sensor location 811 is located at the distal end of the rotatable endoscope axis 806, such as at the coupling portion 614. The angular position sensor may be a Hall effect sensor, an encoder (e.g., mechanical, optical, magnetic, electromagnetic induction), a rotary potentiometer, a resolver, and / or any other angle measurement sensor.

[0169] Alternatively or additionally, the angular position sensor may be located at a second angular position sensor location 820 within or on the rotatable interface 814, at a third angular position sensor location 822 within or on the fixed distal housing 808 (e.g., at the proximal end of the rotatable endoscope assembly 802 that extends within the hand-held cable housing 804 and remains in a fixed orientation relative to the hand-held cable housing 804), at a fourth angular position sensor location 824 within or on the hand-held cable housing 804, and / or at any other location suitable for measuring the angular offset between the rotatable endoscope assembly 802 and the hand-held cable housing 804.

[0170] Although the first angular position sensor location 818 is depicted as being located at the coupling portion 614, the first angular position sensor location 818 can be located at any position within or on the rotatable endoscope shaft 806. Although the third angular position sensor location 822 is depicted as being located at the coupling portion 614, the third angular position sensor location 822 can be located at any position within or on the fixed distal housing 808. Although the fourth angular position sensor location 824 is depicted as being located at the coupling portion 614, the fourth angular position sensor location 824 can be located at any position within or on the handheld cable housing 804.

[0171] In some embodiments, angular position sensors are located at the plurality of angular position sensor locations 818 - 824. For example, for Hall effect sensors, a ring magnet can be positioned at the first of the plurality of angular position sensor locations 818 - 824, and a Hall effect sensor can be positioned at the second of the plurality of angular position sensor locations 818 - 824. For example, the ring magnet can be positioned at any one of the first or second angular position sensor locations 818 - 820, while the Hall effect sensor can be positioned at any one of the third or fourth angular position sensor locations 822 - 824, and vice versa.

[0172] In some embodiments, instead of the defined image horizontal line being defined relative to the rotatable endoscope assembly 802 or the handheld cable housing 804, the defined image horizontal line is based on the sensed direction of gravity. In such embodiments, one or more of the angular position sensors include gravity sensors to measure the angular offset relative to gravity. For example, the defined image horizontal line can be the sensed direction of gravity, or some angle based on the sensed direction of gravity, such as a direction orthogonal to the sensed direction of gravity.

[0173] In some embodiments, the gravity sensor is a gyroscope, magnetometer, and / or linear acceleration sensor, or any other gravity sensor. For example, an accelerometer provides multi - axis measurements of the intrinsic acceleration. Based on the measurements, a calibration step (e.g., the intrinsic acceleration of the endoscope image capture device 800 in a calibration orientation) can be formed or the measurements of the intrinsic acceleration can be combined with one or more other sensors (e.g., a magnetometer) to determine the value of the intrinsic acceleration due to gravity on one or more of the axes. Based on the relative values of gravity on each of the axes, the orientation of the endoscope image capture device 800 relative to gravity can be calculated. Although examples of gravity measurement are provided above, the present disclosure also contemplates any other method or sensor for measuring gravity.

[0174] In some embodiments, the angular offset is the final orientation of the endoscopic image capture device 800 relative to gravity, the relative magnitudes of gravity along each axis of orientation, the offset from the sensed direction of gravity (e.g., orthogonal to the sensed direction of gravity), or any other measurement indicative of the orientation of the endoscopic image capture device 800 relative to gravity. In some embodiments, the direct measurements of the gravity sensor (e.g., one or more accelerometer, magnetometer, and / or gyroscope measurements) are provided as the angular offset and are used to calculate the orientation of the endoscopic image capture device 800 on an external device such as the electronic equipment cart 56 or the electronic equipment cart 24.

[0175] In one example, the gravity sensor is positioned at the fifth angular position sensor location 826, which is located at or along any position of the rotatable endoscopic axis 806 of the rotatable endoscopic assembly 802, such as at the CIT image capture device 810, at the distal end of the rotatable endoscopic axis 806. Thus, the gravity sensor measures the orientation of the CIT image capture device 810 relative to gravity as the angular offset. That is, the gravity sensor directly measures the orientation of the CIT image capture device 810 relative to gravity so that the captured images and videos can be rotated to maintain a gravity-based horizontal line.

[0176] In another example, in the handheld cable housing 804 or the fixed distal housing 808, one gravity sensor is positioned at the first, second, or fifth angular position sensor locations 818 - 120, 826, and a second gravity sensor is positioned at the third or fourth angular position sensor locations 822 - 824. Thus, the orientation of the CIT image capture device 810 or any other part of the rotatable endoscopic axis 806 relative to gravity and the orientation of the handheld cable housing 804 or the fixed distal housing 808 can be determined. The angular offset is the difference between the orientation of the CIT image capture device 810 or any other part of the rotatable endoscopic axis 806 relative to gravity and the orientation of the handheld cable housing 804 or the fixed distal housing 808 relative to gravity.

[0177] In some embodiments, the defined image horizontal line can be defined based on the orientation of the handheld cable housing 804 or the rotatable endoscopic assembly 802, but the angular offset between the two is determined based on their respective relative orientations relative to gravity, as described in the examples below in Figure 8A and Figure 8B as described.

[0178] In some embodiments, instead of positioning one or more gravity sensors in the handheld cable housing 804, the one or more gravity sensors are positioned at the proximal end of the rotatable endoscope assembly 802, which extends within the handheld cable housing 804 and remains fixed and centered relative to the handheld cable housing 804, such as at the third angle sensor location 822 in the fixed distal housing 808. For example, for a replaceable rotatable endoscope assembly 802, the one or more gravity sensors are positioned in or on the fixed distal housing 808 (e.g., the fixed proximal end of the rotatable endoscope assembly 802 that is connected to the handheld cable housing 804).

[0179] In a further example, one or more gravity sensors are positioned in or on the handheld cable housing 804 to measure the orientation of the handheld cable housing 804 relative to gravity. Additionally, an angular position sensor positioned at the coupling portion 614 measures the angular offset of the rotatable endoscope assembly 802 relative to the handheld cable housing 608. Thus, the orientation of the rotatable endoscope assembly 802 relative to gravity, and thus the orientation of the CIT image capture device 810, can be determined based on a combination of the angular offset measurements from the one or more gravity sensors and the angular offset measurements from the angular position sensor.

[0180] Similarly, one or more gravity sensors are positioned in the rotatable endoscope shaft 806 to measure its orientation relative to gravity. Additionally, an angular position sensor positioned at the coupling portion 614 measures the angular offset of the rotatable endoscope shaft 806 relative to the handheld cable housing 804. Thus, the orientation of the handheld cable housing 804 relative to gravity can be determined based on a combination of the angular offset measurements from the one or more gravity sensors and the angular offset measurements from the angular position sensor.

[0181] In some embodiments, the gravity sensors described above can include two or more gravity sensors mounted in a particular orientation relative to each other, such as two gravity sensors mounted orthogonally to each other, to prevent gimbal lock. In some embodiments, more than two gravity sensors can be used. In some embodiments, two or more gravity sensors are mounted in a non-90° orientation. For example, the gravity sensors in the handheld cable housing 608 include two gravity sensors mounted orthogonally to each other within the handheld cable housing 608.

[0182] As Figure 8AAs shown, a horizontal line 622 is defined relative to the handheld cable housing 608. Specifically, the horizontal line 622 is the horizontal midplane 624 of the handheld cable housing 608 at the coupling portion 614. In other words, the horizontal line 622 is orthogonal to the vertical midplane of the handheld cable housing 608 and parallel to the longitudinal axis of the handheld cable housing 608.

[0183] In this example, the rotatable endoscope assembly 601 includes a CIT image capture device 606 with an angle 514 greater than 0° (e.g., 30° tip). In a first configuration, the rotatable endoscope assembly 601 is positioned at a reference position relative to the handheld cable housing 608 and thus relative to the horizontal line 622. Accordingly, the CIT image capture device 606 is capable of capturing an image from the first DOV 627.

[0184] In a second configuration, the rotatable endoscope assembly 601 is rotated by an angle 630 from the reference position while the handheld cable housing 608 remains fixed. Accordingly, the CIT image capture device 606 is capable of capturing an image from the second DOV 629. Thus, the first angular position sensor measures the angle 630 as the angular offset. As described in more detail, in the case of enabling horizontal image alignment, the image data captured by the CIT image capture device 606 is rotated according to the measured angle 630 so that the displayed image is aligned with the image horizontal line defined by the handheld cable housing 608. Figure 9

[0185] Figure 8B In Another example shown, the rotatable endoscope assembly 601 includes a CIT image capture device 606 in which the angle 514 is 0° (e.g., zero-degree tip or zero-degree optics). Thus, when the rotatable endoscope assembly 601 is rotated relative to the handheld cable housing 608, the DOV of the CIT image capture device 606 remains unchanged but rotates.

[0186] In some embodiments, it may not be desirable to enable horizontal image alignment. For example, with zero-degree optics, horizontal image alignment can be manually turned off by selecting a control button on the control surface 618, or automatically turned off by detecting the rotatable endoscope assembly 601 with zero-degree optics. In either case, the user can manually manipulate the joystick 610 (e.g., alignment wheel) to maintain the image horizontal line. For example, the user rotates the joystick 610 so that the rotatable endoscope assembly 601 is aligned with the horizontal line.

[0187] In this case, instead of defining a horizontal line relative to the handheld cable housing 608, a horizontal line is defined relative to the rotatable endoscope assembly 601.

[0188] For example, as Figure 8BAs shown, a horizontal line 634 is defined relative to the rotatable endoscope assembly 601. Specifically, the horizontal line 634 is the horizontal midline plane of the rotatable endoscope assembly 601 at the coupling portion 614. In other words, the horizontal line 634 is orthogonal to the vertical midline plane of the rotatable endoscope assembly 601 and parallel to the longitudinal axis of the rotatable endoscope assembly 601.

[0189] In some embodiments, the vertical midline plane of the rotatable endoscope assembly 601 intersects the reference position. Thus, the horizontal line 634 is orthogonal to the midline plane that intersects the reference position of the rotatable endoscope assembly 601 and parallel to the longitudinal axis of the rotatable endoscope assembly 601. More generally, the horizontal line 634 is orthogonal to the midline plane and parallel to the longitudinal axis of the rotatable endoscope assembly 601, and this midline plane intersects the midpoint of the movement path of the rotatable endoscope assembly 601.

[0190] In the first configuration, the rotatable endoscope assembly 601 is positioned at the reference position relative to the handheld cable housing 608, where both the rotatable endoscope assembly 601 and the handheld cable housing 608 are oriented at an angle (e.g., tilted to the side).

[0191] In the second configuration, the rotatable endoscope assembly 601 is manually rotated by an angle 636 from the reference position while the handheld cable housing 608 remains fixed. Thus, the user manually maintains the horizontal image alignment. Since the handheld cable housing 608 is oriented at an angle, the control surface 618 may be more visible and / or more accessible to select one or more control buttons for certain procedures compared to when the handheld cable housing 608 is vertically oriented.

[0192] In Figure 8A and Figure 8B In the example shown, the reference position is the midpoint position of the movement path of the rotatable endoscope assembly 601. The rotatable endoscope assembly 601 can be rotated relative to the midpoint position by an angle of + / - 180° or any subset of its angles. At the reference position, the measured angular offset is 0°.

[0193] The present disclosure contemplates other reference positions, such as a starting position in a movement path, where the rotatable endoscope assembly 601 can rotate +360° relative to the starting position or any subset of its angles. In another example, the reference position is an ending position in the movement path, where the rotatable endoscope assembly 601 can rotate -360° relative to the ending position or any subset of its angles. More generally, the reference position is located at any position along the movement path, where the rotatable endoscope assembly 601 can rotate from the reference position across any subset of angles of + / −360°. In some embodiments, the rotatable endoscope assembly 601 does not include a stopper, such that the rotatable endoscope assembly can rotate without limit in either direction (e.g., can rotate continuously).

[0194] In the examples provided above, a horizontal line is defined based on the handheld cable housing 608, based on the rotatable endoscope assembly 601, relative to gravity sensed at the distal end of the rotatable endoscope assembly 601 (e.g., at the CIT image capture device 606), relative to gravity sensed at the fixed proximal end of the rotatable endoscope assembly 601 connected to the handheld cable housing 608, or relative to the handheld cable housing 608. The angular offset is measured by an angular position sensor (such as at the coupling portion 614) and / or one or more gravity sensors positioned in the rotatable endoscope assembly 601 and / or the handheld cable housing 608, as described with respect to Figure 7 that described above.

[0195] In another example, the horizontal line is user-defined. For example, after orienting the endoscope image capture device 600 to a desired orientation, a control button on the control surface 618 is activated (e.g., pressed) to define the horizontal line as the desired orientation. Thereafter, any movement from the desired orientation is measured as an angular offset. For example, as described above, the movement of the rotatable endoscope assembly 601 and / or the handheld cable housing 608 from the desired orientation is measured using an angular position sensor (such as at the coupling portion 614) and / or one or more gravity sensors positioned in the rotatable endoscope assembly 601 and / or the handheld cable housing 608.

[0196] In various embodiments, buttons on the control surface 618 can be selected to change the operating mode of how the horizontal line is defined for the endoscope image capture device 600 and how the angular offset is measured. For example, each press of a button can switch between different modes of how the horizontal line is defined and how the angular offset is measured. Each mode corresponds to one of the examples of how the horizontal line is defined and the angular offset is measured described above. In some embodiments, a subset of the examples can be used as the set of operating modes for the endoscope image capture device 600. In some embodiments, the endoscope image capture device 600 includes an operating mode in which the horizontal line is not defined and the angular offset is not measured.

[0197] Each of the feature portions of the endoscopic image capture device 600 described above can be used alone or in combination with another or other feature portions described throughout this disclosure. Various modifications and additions to the endoscopic image capture device 600 are readily discernible by those of ordinary skill in the art and are also contemplated by this disclosure. For example, this disclosure contemplates alternative illumination, filtering, optical components, focal length manipulation, and image sensor feature portions known to those of ordinary skill in the art.

[0198] Figure 9 is a timing diagram of an image processing operation for maintaining horizontal image alignment. The first view 702 shows an original scene oriented relative to a defined horizontal line 704. The defined horizontal line 704 can be defined based on any one or combination of the examples for defining an image horizontal line described above.

[0199] A picture 706 of the first view 702 is captured. In some embodiments, the picture 706 is captured using the optical component 506 and the image sensor 510 of the chip-in-tip (CIT) image capture device 500. In some embodiments, the picture 706 is captured by the endoscopic image capture device 600 described above together with the CIT image capture device 606. In any case, the picture 706 of the first view 702 is captured using a device oriented at an angular offset relative to the defined horizontal line 704. In the example shown, the image horizontal line 708 of the picture 706 has an angular offset of 180° relative to the defined horizontal line 704. For example, the picture 706 can be captured by the image capture device 600 oriented as shown in Figure 6B The measured value of the angular offset and the picture 706 are transmitted to an image processor to rotate the picture 706 based on the angular offset, thereby maintaining a constant image horizontal line in the displayed image 710. Thus, the displayed image 710 has a displayed image horizontal line 712 that matches the defined horizontal line 704.

[0200] For example, the endoscopic image capture device 600 transmits the picture 706 as a still image or video stream to the electronic device cart 56, the electronic device cart 24, or the processor 58 to rotate the picture 706 based on the angular offset. Specifically, the image captured by the CIT image capture device 606 and the measured value of the corresponding angular offset measured by one or more angular position sensors are transmitted from the endoscopic image capture device 600 to the electronic device cart 56 or the electronic device cart 24 via the connector cable 620 for processing (e.g., image rotation). After processing, the rotated image will be displayed on a display (such as the display 60, the display 25, or the display area 31) with a constant image horizontal line.

[0201]

[0202] ​Figure 10A is a cross-sectional view of an exemplary endoscopic image capture device 900, which shows the coupling between a rotatable endoscopic assembly 902 and a fixed hand-held cable housing 904. Figure 10B is Figure 10A an exploded view of the rotatable endoscopic assembly 902 of Figure 10C is Figure 10A a perspective view of the rotatable endoscopic assembly 902 of , which shows details of an angular position sensor for measuring rotation between the rotatable endoscopic assembly 902 and the fixed hand-held cable housing 904.

[0203] In some embodiments, the endoscopic image capture device 900 is implemented as the endoscopic image capture device 600 or the endoscopic image capture device 800 described above, where the same numerals represent the same parts. Thus, the rotatable endoscopic assembly 902 is implemented as the rotatable endoscopic assembly 601 or the rotatable endoscopic assembly 802. Similarly, the hand-held cable housing 904 is implemented as the hand-held cable housing 608 or the hand-held cable housing 804.

[0204] As Figure 10A shown in , the thick dashed line shows the interface 906 between the rotatable endoscopic assembly 902 (e.g., the rotatable endoscopic shaft) and the fixed hand-held cable housing 904. The rotatable endoscopic assembly 902 includes a rotatable endoscopic shaft 908 (such as the rotatable endoscopic shaft 806) and a fixed distal housing 910 (such as the fixed distal housing 808). The fixed distal housing 910 is located at the proximal end of the rotatable endoscopic assembly 902.

[0205] The fixed distal housing 910 is sized and configured to be releasably received in a socket 912 of the fixed hand-held cable housing 904. A lock 913 maintains the rotatable endoscopic shaft 908 within the socket 912 of the fixed hand-held cable housing 904. In various embodiments, the lock 913 is biased in a locked configuration. A release lever 915 is optional for configuring the lock 913 to an unlocked configuration to facilitate removal of the rotatable endoscopic shaft 908 from the socket 912 of the fixed hand-held cable housing 904.

[0206] The rotatable endoscopic shaft 908 is rotatable relative to the fixed distal housing 910 about a rotatable interface 914. For example, a lever 610 is coupled to the rotatable interface 914 to rotate the rotatable endoscopic shaft 908 relative to the fixed distal housing 910.

[0207] In some embodiments, a radial shaft seal ring 918 seals the interior volume of the fixed distal housing 910 to prevent ingress of the environment around the endoscopic image capture device 900.

[0208] The rotatable endoscope assembly 902 has an angular position sensor located in a fixed distal housing 910. In the illustrated example, the angular position sensor includes an annular magnet 920 that is coupled to the rotatable interface 914 and is configured to rotate with the rotatable endoscope shaft 908. A Hall effect sensor 922 is positioned in the fixed distal housing 910 to detect the changing magnetic field of the annular magnet 920 as it rotates with the rotatable endoscope shaft 908.

[0209] As Figure 10B Best shown in, the rotatable endoscope assembly 902 has an electronics assembly 924 that is coupled to the fixed distal housing 910 and is configured to communicate data and power with the handheld cable housing 904. The electronics assembly 924 has a power interface 926, such as a receiver induction coil. The electronics assembly 924 also has one or more data interfaces 928, such as a transmitter induction coil. The electronics assembly 924 also has a ferrule 934 that is configured to receive light from a light source via the fixed handheld cable housing 904.

[0210] Although the transmitter and receiver induction coils shown in the example are used for wireless power and data transfer, any wired and / or wireless power and / or data interface may be used. Because the illustrated example uses wireless data and power transfer, the rotatable endoscope assembly 902 is a sealed system such that the rotatable endoscope can be cleaned and disinfected, such as in an autoclave.

[0211] In some embodiments, the electronics assembly 924 also has a ball bearing coupled between the fixed distal housing 910 and the rotatable interface 914 to facilitate rotation of the rotatable endoscope shaft 908 relative to the fixed distal housing 910. A proximal bearing 930 is positioned at the proximal end of the rotatable interface 914. A distal bearing 932 is positioned at the distal end of the rotatable interface 914.

[0212] The rotatable endoscope shaft 908 has a cavity 936 that is configured to communicate power, data, and illumination with an image capture device (such as the CIT image capture device 606 or the CIT image capture device 500) at the distal end of the rotatable endoscope shaft 908. In the illustrated example, a flat flex 938 is provided for transmitting data and / or power through the cavity 936. In some embodiments, a lighting element (not shown), such as a light pipe or fiber optic bundle, transmits the light received from the light source at the ferrule 934 through the cavity 936.

[0213] The handheld cable housing 904 has a control surface 618 on which a plurality of control buttons 940 are positioned. The control buttons 940 control one or more operating functions (e.g., turning the illumination on or off, changing the light source, turning the horizontal image alignment on or off, changing the defined horizontal line, capturing a still image, etc.), as described above.

[0214] Each of the feature portions of the endoscopic image capture device 900 described above can be used alone or in combination with another or other feature portions described throughout this disclosure. Various modifications and additions to the endoscopic image capture device 900 are readily discernible to those of ordinary skill in the art and are also contemplated by this disclosure. For example, this disclosure contemplates alternative illumination, filtering, optical components, focal length manipulation, and image sensor feature portions known to those of ordinary skill in the art.

[0215] Figure 11 is an operation flowchart 1000 of an image capture device according to various embodiments described herein. In some embodiments, the image capture device is the endoscopic image capture device 900, the endoscopic image capture device 800, or the endoscopic image capture device 600 described above. Similar to the systems described above, the image capture device has an image capture assembly in which an image sensor is positioned, and the image capture assembly is rotatably attached to a control assembly having a control surface with one or more control buttons thereon. For example, the image capture assembly can be the rotatable endoscopic assembly 601, the rotatable endoscopic assembly 802, or the rotatable endoscopic shaft 908 described above. Similarly, the control assembly can be the handheld cable housing 608, the fixed handheld cable housing 804, or the fixed handheld cable housing 904 described above. Thus, as the image capture assembly rotates relative to the control assembly, the one or more control buttons remain readily operable.

[0216] At 1002, the image capture device captures a still image or a video image. For example, a still image or a video image is captured by the CIT image capture device 500, the CIT image capture device 606, or the CIT image capture device 810.

[0217] At 1004, the image capture device measures the angular offset from a defined horizontal line using one or more angular position sensors. For example, the one or more angular position sensors are Hall effect sensors, encoders (e.g., mechanical, optical, magnetic, electromagnetic induction), rotary potentiometers, resolvers, any other angle measurement sensors, gyroscopes, magnetometers, linear acceleration sensors, and / or any other gravity sensors. The angular position sensors can be positioned at one or more of the image capture assembly, the control assembly, and / or the coupling portion between the image capture assembly and the control assembly.

[0218] At 1006, the control assembly receives the selection of a control button on the control surface. For example, the control button generates a control signal for performing one or more operation functions (e.g., turning on or off illumination, changing the illumination source, turning on or off horizontal image alignment, changing the defined horizontal line, capturing a still image, etc.).

[0219] At 1008, the control component transmits the image data captured by the image capture device and the angular offset data measured by the one or more angular position sensors to an external device. For example, as discussed above, the connector cable of the control component (such as the connector cable 620 described above) facilitates the wired and / or wireless transmission of data and power between the control component and an external device (such as the electronic equipment cart 56 or the electronic equipment cart 24).

[0220] Similarly, at 1010, the control component transmits a control signal to an external device in response to the selection of the control button at 1006. For example, the connector cable (such as the connector cable 620) facilitates the wired and / or wireless transmission of the control signal to an external device (such as the electronic equipment cart 56 or the electronic equipment cart 24).

[0221] Figure 12 is an operation flowchart of an image processor according to various embodiments described herein. In some embodiments, the image processor is the electronic equipment cart 56, the electronic equipment cart 24, or the processor 58.

[0222] At 1102, the image processor receives image data and angular offset data from an image capture device (such as Figure 11 the image capture device (e.g., the endoscopic image capture device 900, the endoscopic image capture device 800, or the endoscopic image capture device 600) described above).

[0223] At 1104, the image processor rotates the image data based on the angular offset data. The rotated image data is displayed at 1106, thereby maintaining a constant image horizontal line on the displayed image data. For example, the rotated image is displayed on a display with a constant image horizontal line, such as on the display 60, the display 25, or the display area 31.

[0224] At 1108, the image processor receives and processes a control signal. For example, it receives a control signal from a control component (such as Figure 11 the control component). The image processor performs one or more operations based on the processing of the control signal. For example, the image processor operates to turn on or off the light source provided by the image processor, capture a still image from the received image data, start / stop video recording from the received image data, define an image horizontal line (e.g., switch between different modes of how to define the horizontal line and how to measure the angular offset as described above), turn on or off horizontal image alignment, or perform any other control function for operating the connected image capture device.

[0225] It should be understood that the logical operations described herein for the respective drawings can be implemented as (1) in a computing device (e.g., Figure 13A series of computer-implemented actions or program modules (i.e., software) running on a computing device (as described in ), (2) machine logic circuits or circuit modules (i.e., hardware) interconnected within the computing device, and / or (3) a combination of software and hardware of the computing device. Thus, the logical operations discussed herein are not limited to any particular combination of hardware and software. The implementation is a matter of choice depending on the performance and other requirements of the computing device. Thus, the logical operations described herein are variously referred to as operations, structural devices, actions, or modules. These operations, structural devices, actions, and modules can be implemented in software, firmware, dedicated digital logic, and any combination thereof. It should also be understood that more or fewer operations than those shown in the figures and described herein can be performed. These operations can also be performed in a different order than those described herein.

[0226] Reference Figure 13 , an exemplary computing device 1200 is illustrated on which embodiments of the present invention can be implemented. For example, each of the computer processors located in the electronic device cart 56 or the electronic device cart 24 and the computer processor 58 described herein can be implemented as a computing device, such as the computing device 1200. It should be understood that the exemplary computing device 1200 is merely one example of a suitable computing environment on which embodiments of the present invention can be implemented. Optionally, the computing device 1200 can be a well-known computing system, including but not limited to a personal computer, a server, a handheld or notebook device, a multiprocessor system, a microprocessor-based system, a network personal computer (PC), a minicomputer, a mainframe computer, an embedded system, and / or a distributed computing environment including any of the above systems or devices. A distributed computing environment enables remote computing devices connected to a communication network or other data transmission medium to perform various tasks. In a distributed computing environment, program modules, applications, and other data can be stored on local and / or remote computer storage media.

[0227] In one embodiment, computing device 1200 may include two or more computers that communicate with and cooperate with each other to perform tasks. For example (but not by way of limitation), an application may be partitioned in some manner to allow for concurrent and / or parallel processing of the application's instructions. Alternatively, the data processed by the application may be partitioned in some manner to allow two or more computers to concurrently and / or parallel process different portions of the data set. In one embodiment, computing device 1200 may employ virtualization software to provide the functionality of many servers, which is not directly tied to the number of computers in computing device 1200. For example, the virtualization software may provide twenty virtual servers on four physical computers. In one embodiment, the functionality of the present disclosure above may be provided by executing the application and / or multiple applications in a cloud computing environment. Cloud computing may include providing computing services using dynamically scalable computing resources via a network connection. Cloud computing may be at least partially supported by virtualization software. The cloud computing environment may be established by an enterprise and / or leased from a third-party provider as needed. Some cloud computing environments may include cloud computing resources owned and operated by an enterprise as well as cloud computing resources leased and / or rented from a third-party provider.

[0228] In its most basic configuration, computing device 1200 generally includes at least one processing unit 1220 and system memory 1230. Depending on the specific configuration and type of the computing device, system memory 1230 may be volatile (such as random access memory (RAM)), non-volatile (such as read-only memory (ROM), flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in dashed lines 1210 in Figure 13 The processing unit 1220 may be a standard programmable processor that performs the arithmetic and logical operations required for the operation of computing device 1200. Although only one processing unit 1220 is shown, multiple processors may exist. Thus, although instructions may be discussed as being executed by a processor, the instructions may be executed simultaneously, serially, or otherwise by one or more processors. Computing device 1200 may also include a bus or other communication mechanism for transferring information between the various components of computing device 1200.

[0229] The computing device 1200 may have additional features / functions. For example, the computing device 1200 may include additional memory, such as removable memory 1240 and non-removable memory 1250, which includes but is not limited to magnetic disks, optical disks, or magnetic tapes. The computing device 1200 may also include one or more network connectors 1280, which allow the device to communicate with other devices, such as via the communication paths described herein. The one or more network connectors 1280 may take the form of a modem, a modem bank, an Ethernet card, a universal serial bus (USB) interface card, a serial interface, a token ring card, a fiber distributed data interface (FDDI) card, a wireless local area network (WLAN) card, a radio transceiver card (such as code division multiple access (CDMA)), a global system for mobile communications (GSM), long term evolution (LTE), worldwide interoperability for microwave access (WiMAX), and / or other air interface protocol radio transceiver cards, and other well-known network devices. The computing device 1200 may also have one or more input devices 1270, such as a keyboard, keypad, switch, dial, mouse, trackball, touch screen, voice recognizer, card reader, paper tape reader, or other well-known input devices. One or more output devices 1260, such as a printer, video monitor, liquid crystal display (LCD), touch screen display, display, speaker, etc., may also be included. Additional devices may be connected to the bus to facilitate data communication between the components of the computing device 1200. All of these devices are well known in the art and need not be discussed in detail herein.

[0230] The processing unit 1220 may be configured to execute program code encoded in a tangible, computer-readable medium. A tangible computer-readable medium is any medium that can provide data that causes the computing device 1200 (i.e., the machine) to operate in a particular manner. Various computer-readable media may be utilized to provide instructions to the processing unit 1220 for execution. Exemplary tangible computer-readable media may include, but are not limited to, volatile media, non-volatile media, removable media, and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. System memory 1230, removable memory 1240, and non-removable memory 1250 are all examples of tangible computer storage media. Exemplary tangible, computer-readable recording media include, but are not limited to, integrated circuits (e.g., field programmable gate arrays or application specific ICs), hard disks, optical disks, magneto-optical disks, floppy disks, magnetic tapes, holographic storage media, solid state devices, RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, CD-ROM, digital versatile disks (DVD), or other optical memory, cassette tapes, magnetic tapes, disk memories, or other magnetic storage devices.

[0231] Fundamental to the fields of electrical engineering and software engineering is that functions implemented by loading executable software into a computer can be transformed into a hardware implementation by well-known design rules. The decision of whether to implement a concept in software or hardware generally depends on considerations of design stability and the number of units to be produced, rather than any issues involved in the transition from the software domain to the hardware domain. Generally, designs that are still subject to frequent changes may be more suitable for implementation in software because it is more expensive to quickly switch back to a hardware implementation than to quickly switch back to a software design. Generally, stable designs that are to be mass-produced may be more suitable for implementation in hardware such as application-specific integrated circuits (ASICs) because, for large-scale production, a hardware implementation may be less expensive than a software implementation. Frequently, a design can be developed and tested in software form and then transformed into an equivalent hardware implementation in an ASIC, for example, by well-known design rules, where the ASIC hardwires the instructions of the software. Just as a machine controlled by a new ASIC is a specific machine or device, a computer programmed and / or loaded with executable instructions can also be regarded as a specific machine or device.

[0232] In an exemplary embodiment, the processing unit 1220 can execute program code stored in the system memory 1230. For example, a bus can transfer data to the system memory 1230, and the processing unit 1220 receives and executes instructions from the system memory 1230. Optionally, data received by the system memory 1230 can be stored on the removable memory 1240 or the non-removable memory 1250 before or after execution by the processing unit 1220.

[0233] It should be understood that the various techniques described herein can be implemented in hardware or software, or in a combination thereof as appropriate. Accordingly, the methods and apparatuses, or certain aspects or portions thereof, of the presently disclosed subject matter may be embodied in the form of program code (i.e., instructions) in a tangible medium such as a floppy disk, a CD-ROM, a hard disk, or any other machine-readable storage medium, where when the program code is loaded into and executed by a machine such as a computing device, the machine becomes an apparatus for practicing the presently disclosed subject matter. In the case of executing program code on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. One or more programs may implement or utilize the processes described in connection with the presently disclosed subject matter, e.g., by using an application programming interface (API), reusable controls, or the like. Such programs may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the (one or more) programs may be implemented in assembly or machine language. In any case, the language may be a compiled or interpreted language, and may be combined with hardware implementation.

[0234] Embodiments of the methods and systems may be described herein with reference to block diagrams and flowchart illustrations of methods, systems, apparatuses, and computer program products. It will be understood that each block of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, can be implemented by computer program instructions. These computer program instructions may be loaded onto a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed on the computer or other programmable data processing apparatus create means for implementing the functions specified in one or more of the flowchart blocks.

[0235] These computer program instructions may also be stored in a computer-readable memory, which may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including computer-readable instructions for implementing the functions specified in one or more of the flowchart blocks. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the flowchart blocks.

[0236] Accordingly, the blocks in the block diagrams and flowchart illustrations support combinations of means for performing the specified functions, combinations of steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be understood that each block in the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, can be implemented by a special purpose hardware-based computer system for performing the specified functions or steps, or by a combination of special purpose hardware and computer instructions.

[0237] Although several embodiments are provided in the present disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. This example should be considered exemplary and not restrictive, and is not intended to be limited to the details given herein. For example, various elements or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0238] Furthermore, without departing from the scope of the present disclosure, the techniques, systems, subsystems, and methods described and illustrated as discrete or separate in various embodiments may be combined or integrated with other systems, modules, techniques, or methods. Other items shown or discussed as being directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component, whether electrical, mechanical, or otherwise. Those skilled in the art can identify other instances of alterations, substitutions, and changes, and can make alterations, substitutions, and changes without departing from the spirit and scope of the disclosure herein.

Claims

1. A surgical system, comprising: A handheld cable housing with a user input device positioned thereon; An endoscope shaft having a distal end and a proximal end, wherein the proximal end of the endoscope shaft is coupled to the handheld cable housing, wherein the distal end of the endoscope shaft includes an image sensor, and wherein the endoscope shaft is rotatable relative to the handheld cable housing; and One or more angular position sensors configured to measure an angular offset relative to a defined image horizontal line.

2. The surgical system according to claim 1, wherein the one or more angular position sensors are positioned at a coupling portion between the endoscope shaft and the handheld cable housing.

3. The surgical system according to any one of claims 1 or 2, wherein the defined image horizontal line is relative to the handheld cable housing.

4. The surgical system according to claim 3, wherein the angular offset is a measurement of the angular rotation of the endoscope shaft relative to the handheld cable housing.

5. The surgical system according to any one of claims 1 to 4, wherein the angular offset is a measurement of the angular rotation between the defined image horizontal line and the viewing direction of the endoscope shaft.

6. The surgical system according to any one of claims 3 or 4, wherein the defined image horizontal line is the horizontal midline plane of the handheld cable housing at the coupling portion.

7. The surgical system according to any one of claims 3 or 4, wherein the defined image horizontal line is orthogonal to the vertical midline plane of the handheld cable housing and parallel to the longitudinal axis of the handheld cable housing.

8. The surgical system according to any one of claims 1 or 2, wherein the defined image horizontal line is relative to the endoscope shaft.

9. The surgical system according to claim 8, wherein the angular offset is a measurement of the angular rotation of the handheld cable housing relative to the endoscope shaft.

10. The surgical system according to any one of claims 8 or 9, wherein the defined image horizontal line is the horizontal midline plane of the endoscope shaft at the coupling portion.

11. The surgical system according to any one of claims 8 or 9, wherein the defined image horizontal line is orthogonal to the vertical midline plane of the endoscope shaft and parallel to the longitudinal axis of the endoscope shaft.

12. The surgical system according to claim 1 or 2, wherein the defined image horizontal line is based on the sensed direction of gravity.

13. The surgical system according to claim 1 or 2 or 12, wherein the one or more angular position sensors are positioned at the distal end of the endoscope shaft.

14. The surgical system according to claim 1 or 2 or 12, wherein the one or more angular position sensors are positioned at the handheld cable housing.

15. The surgical system according to any one of claims 1 to 14, wherein the one or more angular position sensors comprise one or more sensors selected from the group consisting of Hall effect sensors, mechanical encoders, optical encoders, magnetic encoders, electromagnetic induction encoders, encoders, rotary potentiometers, resolvers, gravity sensors, gyroscopes, magnetometers, and linear acceleration sensors.

16. The surgical system according to claim 1, wherein the user input device is positioned on the control surface of the handheld cable housing.

17. The surgical system according to claim 16, wherein the control surface is the top surface of the handheld cable housing.

18. The surgical system according to any one of claims 1 to 17, wherein the user input device is selected from the group of user input devices consisting of physical buttons, capacitive sensing buttons, soft buttons on a touch screen, switches, touch pads, rollers, and directional keys.

19. The surgical system according to any one of claims 1 to 18, wherein the handheld cable housing includes a socket sized and configured to receive the proximal end of the endoscopic shaft.

20. The surgical system according to claim 19, wherein the handheld cable housing includes a lock configured to maintain the proximal end of the endoscopic shaft within the socket of the handheld cable housing.

21. The surgical system according to claim 20, wherein the lock is biased in a locked configuration.

22. The surgical system according to any one of claims 20 or 21, wherein the handheld cable housing includes a release lever that selectively configures the lock to an unlocked configuration for releasing the proximal end of the endoscopic shaft within the socket of the handheld cable housing.

23. The surgical system according to any one of claims 1 to 22, wherein the handheld cable housing includes a data interface configured to receive image data from the image sensor and the angular offset from the one or more angular position sensors.

24. The surgical system according to claim 23, wherein the proximal end of the endoscopic shaft includes a corresponding data interface configured to supply image data from the image sensor and the angular offset from the one or more angular position sensors to the handheld cable housing.

25. The surgical system according to claim 23 or 24, wherein the handheld cable housing includes a connector cable configured to transmit the image data and the angle offset data to an external device.

26. The surgical system according to any one of claims 1 to 25, wherein the proximal end of the endoscope shaft includes a housing configured to remain fixed relative to the handheld cable housing, and wherein the proximal end of the endoscope shaft further includes a rotatable interface configured to facilitate rotation of the endoscope shaft relative to the handheld cable housing.

27. The surgical system according to any one of claims 1 to 26, wherein the distal end of the endoscope shaft includes an optical assembly positioned to receive light incident on the distal face of the endoscope shaft.

28. The surgical system according to claim 27, wherein the distal face is angled with respect to the endoscope shaft.

29. The surgical system according to claim 28, wherein the angle is any angle between 0° and 90°.

30. The surgical system according to any one of claims 27 to 29, wherein the optical assembly includes one or more lenses for guiding light incident on the distal face along an optical path to the image sensor.

31. The surgical system according to any one of claims 1 to 30, wherein the field of view direction of the endoscope shaft is configured to change in response to rotation of the endoscope shaft relative to the handheld cable housing.

32. A method, comprising: Defining an image horizontal line for an orientation relative to a surgical system, wherein the surgical system includes a handheld cable housing and a shaft, a user input device is positioned on the handheld cable housing, an image sensor is positioned in the distal end of the shaft, and wherein the shaft is coupled to the handheld cable housing such that the shaft is rotatable relative to the handheld cable housing; Determining an angular offset between the image horizontal line and a field of view direction of the shaft; and Transmitting image data captured by the image sensor and transmitting angular offset data indicative of the angular offset, the angular offset data being used for rotation of the image data.

33. The method according to claim 32, wherein the angle offset is measured by one or more angular position sensors of the surgical system.

34. The method according to claim 33, wherein the one or more angular position sensors are positioned at a coupling portion between the shaft and the handheld cable housing.

35. The method according to claim 34, wherein the image horizontal line is defined as the horizontal midplane of the handheld cable housing at the coupling portion.

36. The method according to any one of claims 32 to 34, wherein the image horizontal line is defined relative to the handheld cable housing.

37. The method according to claim 36, wherein the angular offset is a measure of the angular rotation of the axis relative to the handheld cable housing.

38. The method according to any one of claims 32 to 37, wherein the angular offset is a measure of the angular rotation between the image horizontal line and the viewing direction of the image sensor.

39. The method according to any one of claims 36 or 37, wherein the image horizontal line is defined as being orthogonal to the vertical median plane of the handheld cable housing and parallel to the longitudinal axis of the handheld cable housing.

40. The method according to any one of claims 34 to 35, wherein the image horizontal line is defined relative to the axis.

41. The method according to claim 40, wherein the angular offset is a measure of the angular rotation of the handheld cable housing relative to the axis.

42. The method according to any one of claims 40 or 41, wherein the image horizontal line is defined as the horizontal median plane of the axis at the coupling portion.

43. The method according to any one of claims 40 or 41, wherein the image horizontal line is defined as being orthogonal to the vertical median plane of the axis and parallel to the longitudinal axis of the axis.

44. The method according to claim 33, wherein the image horizontal line is defined based on the sensed direction of gravity.

45. The method according to any one of claims 33 or 44, wherein the one or more angular position sensors are positioned at the axis.

46. The method according to claim 33 or 44, wherein the one or more angular position sensors are positioned at the handheld cable housing.

47. The method according to any one of claims 33 or 44 to 46, wherein the one or more angular position sensors comprise one or more sensors selected from the group consisting of Hall effect sensors, mechanical encoders, optical encoders, magnetic encoders, electromagnetic induction encoders, encoders, rotary potentiometers, resolvers, gravity sensors, gyroscopes, magnetometers, and linear acceleration sensors.

48. The method according to claim 32, wherein the user input device is positioned on the control surface of the handheld cable housing.

49. The method according to claim 48, wherein the control surface is the top surface of the handheld cable housing.

50. The method according to any one of claims 32 to 49, wherein the user input device is selected from the group consisting of physical buttons, capacitive sensing buttons, soft buttons on a touch screen, switches, touch pads, rollers, and directional keys.

51. The method according to any one of claims 32 to 50, wherein the handheld cable housing includes a socket sized and configured to receive the proximal end of the shaft.

52. The method according to claim 51, wherein the handheld cable housing includes a lock configured to maintain the proximal end of the shaft within the socket of the handheld cable housing.

53. The method according to claim 52, wherein the lock is biased in a locked configuration.

54. The method according to any one of claims 52 or 53, wherein the handheld cable housing includes a release lever selectively configurable to unlock the lock for releasing the proximal end of the shaft within the socket of the handheld cable housing.

55. The method according to any one of claims 32 to 54, wherein the handheld cable housing includes a data interface configured to receive the image data and the angular offset data.

56. The method according to claim 55, wherein the proximal end of the shaft includes a corresponding data interface, and wherein transmitting the image data and the angular offset data includes transmitting the image data and the angular offset data from the shaft to the handheld cable housing.

57. The method according to any one of claims 55 or 56, wherein the handheld cable housing includes a connector cable, and wherein transmitting the image data and the angular offset data includes transmitting the image data and the angular offset data from the handheld cable housing to an external device.

58. A method comprising: Receiving image data captured by an image sensor positioned in the distal end of a shaft, wherein the shaft is coupled to a handheld cable housing, a user input device is positioned on the handheld cable housing, and wherein the shaft is rotatable relative to the handheld cable housing; Receiving angular offset data indicative of an angular offset relative to a defined image horizontal line; Generating rotated image data based on the angular offset data; and Causing the rotated image data to be displayed.

59. The method according to claim 58, further comprising: Receiving a control signal in response to a selection of the user input device, wherein the control signal sets the defined image horizontal line to one of a plurality of image horizontal lines.

60. The method according to any one of claims 58 to 59, further comprising: Supplying light from a light source to the handheld cable housing.

61. The method according to claim 60, further comprising: Receiving a second control signal in response to a selection of the user input device, wherein the second control signal causes the light from the light source to change.

62. The method according to claim 61, wherein the second control signal causes the light from the illumination source to be turned off or the frequency to be changed.

63. The method according to any one of claims 58 to 62, wherein causing the rotated image data to be displayed includes transmitting the rotated image data to an external display.

64. The method according to any one of claims 58 to 63, wherein the defined image horizontal line is with respect to the handheld cable housing.

65. The method according to any one of claims 58 to 64, wherein the angular offset is a measurement of the angular rotation of the axis relative to the handheld cable housing.

66. The method according to any one of claims 58 to 65, wherein the angular offset is a measurement of the angular rotation between the defined image horizontal line and the field of view direction of the image sensor.

67. The method according to any one of claims 58 to 66, wherein the defined image horizontal line is the horizontal midline plane of the handheld cable housing at the coupling portion between the axis and the handheld cable housing.

68. The method according to any one of claims 58 to 67, wherein the defined image horizontal line is orthogonal to the vertical midline plane of the handheld cable housing and parallel to the longitudinal axis of the handheld cable housing.

69. The method according to any one of claims 58 to 63, wherein the defined image horizontal line is with respect to the axis.

70. The method according to any one of claims 58 to 63 or 69, wherein the angular offset is a measurement of the angular rotation of the handheld cable housing relative to the axis.

71. The method according to any one of claims 58 to 63 or 69 to 70, wherein the defined image horizontal line is the horizontal midline plane of the axis at the coupling portion between the axis and the handheld cable housing.

72. The method according to any one of claims 58 to 63 or 69 to 71, wherein the defined image horizontal line is orthogonal to the vertical midline plane of the axis and parallel to the longitudinal axis of the axis.

73. The method according to any one of claims 58 to 63, wherein the defined image horizontal line is based on the sensed direction of gravity.

74. The method according to any one of claims 58 to 63 or 73, wherein one or more angular position sensors for measuring the angular offset are positioned at the axis.

75. The method according to any one of claims 58 to 63 or 73 to 74, wherein one or more angular position sensors for measuring the angular offset are positioned at the handheld cable housing.

76. The method according to any one of claims 58 to 75, wherein one or more angular position sensors configured to measure the angular offset are selected from the group consisting of Hall effect sensors, mechanical encoders, optical encoders, magnetic encoders, electromagnetic induction encoders, encoders, rotary potentiometers, resolvers, gravity sensors, gyroscopes, magnetometers, and linear acceleration sensors.

77. The method according to any one of claims 58 to 76, wherein the user input device is positioned on a control surface of the handheld cable housing.

78. The method according to claim 77, wherein the control surface is a top surface of the handheld cable housing.

79. The method according to any one of claims 58 to 78, wherein the user input device is selected from the group of user input devices consisting of physical buttons, capacitive sensing buttons, soft buttons on a touch screen, switches, touch pads, rollers, and directional keys.

Images