A combined ultrasound and endoscope system

By integrating vision and ultrasound devices, combined with multi-angle rotation and tilting ultrasound probes and cameras, the problem of lack of real-time visual assistance in uterine acoustic angiography is solved, and precise examination of the internal structure of the uterine cavity and bladder is achieved and efficient surgery is achieved. support.

CN113143188BActive Publication Date: 2024-11-08SUZHOU ACUVU MEDICAL TECH CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202011240759.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-06-11
Filing Date
2020-11-09
Publication Date
2024-11-08
Estimated Expiration
2040-11-09

AI Technical Summary

Technical Problem

During hysteric acoustic symptomography, the ultrasound probe insertion often lacks real-time visual assistance, resulting in "blind" and inaccurate enough.

Method used

An integrated vision and ultrasound device is designed, including an ergonomic handle, casing, camera, ultrasound probe, probe steering mechanism and rotation mechanism, allowing the ultrasound probe to rotate and tilt at multiple angles, combining the camera to provide real-time vision. and ultrasound images, adapt to the shape of the organ through the flexible design of the cannula, and generate three-dimensional ultrasound images through the processing system.

Benefits of technology

It realizes the integration of real-time visual and ultrasound images in uterine acoustic angiography and cystographies, improves the accuracy and safety of operations, and enhances the efficiency of examination and surgery of internal structures of the uterus, bladder and other organs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113143188B_ABST
    Figure CN113143188B_ABST
Patent Text Reader

Abstract

The present invention relates to a combined ultrasound and endoscope system. The system includes a cannula having a distal end, the cannula being configured to be inserted into an internal organ or other internal body structure. The distal end is configured to include an ultrasound probe and a camera module. Ultrasound and direct visual endoscopic images can be displayed to a user on a display monitor at the same time. The ultrasound probe can be rotated and turned to scan any position in the cavity of a human organ. The ultrasound probe can be reusable or disposable. The endoscope system can be configured to have a handheld portion, which includes a reusable handle portion and a disposable portion configured to be discarded after a single use. The system can also be configured to use a conventional reusable endoscope with a working channel and an endoscope processing tower system.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 62 / 933,216 filed on November 8, 2019 and PCT Application No. PCT / IB2020 / 000470 filed on June 11, 2020, which are incorporated herein by reference. Technical Field

[0003] This patent specification generally relates to a medical device for tissue and organ examination. More specifically, some embodiments relate to a combined ultrasound and endoscope system for examining internal organs and other internal body structures. Background Art

[0004] Ultrasound is a sound wave with a frequency higher than that audible to humans (>20,000Hz). Ultrasound images, also called sonograms, are made by sending ultrasound pulses into tissues using a probe. The ultrasound pulses echo from tissues with different reflection characteristics and are recorded and displayed as images. Medical ultrasound is often used to create images of internal organs and other body structures such as tendons, muscles, joints, and blood vessels. Although medical ultrasound often uses transducers designed to be used externally, such as through the lower abdominal wall in gynecological ultrasound examinations, sometimes ultrasound transducers are configured to be inserted into internal organs or other structures. One such example is sonohysterogram for uterine imaging. The procedure involves inserting fluid and an ultrasound probe into the uterus and can provide acoustic images of uterine structures. Although endoscopy can be performed before sonohysterogram to obtain direct visual images of the uterine wall, sonohysterogram is usually performed "blindly" or without any real-time visual assistance during the insertion of the ultrasound probe. Summary of the invention

[0005] According to some embodiments of the present invention, an integrated vision and ultrasound device includes: an ergonomic handle for hand grasping and having a proximal portion and a distal portion; a sleeve extending distally from the distal portion of the handle and having a distal portion extending along a longitudinal axis; a distally facing camera fixed to the distal portion of the sleeve and having a camera field of view (FOV) including a selected solid angle and a camera field of view direction (DOV) angled relative to the axis; an ultrasound probe positioned at the distal portion of the sleeve for rotation around the axis relative to the distal portion of the sleeve and tilting relative to the axis; a probe steering mechanism mounted at the proximal end of the handle and operably connected to the ultrasound probe so that the ultrasound probe can be selectively tilted relative to the axis within a selected angle range; and a probe rotation mechanism mounted at the proximal end of the handle and operably connected to the ultrasound probe so that the ultrasound probe can be selectively rotated relative to the sleeve around the axis.

[0006] According to some embodiments, the visual and ultrasound integrated device may further include one or more of the following features: the sleeve may include at least one inner cavity, and may further include a shaft connecting the probe steering mechanism and the ultrasound probe, wherein the shaft is removably accommodated in the inner cavity; a belt located in the shaft, which can be coupled to and driven by the probe rotation mechanism, and a gear that can be fixed to the ultrasound probe and driven by the belt to selectively rotate the ultrasound probe around the axis; the probe rotation mechanism can be configured to rotate the ultrasound probe at least 180 degrees; the ultrasound probe can be fixed to a rotating plate that rotates around a pivot axis transverse to the longitudinal axis, and a rod located in the shaft that can The probe steering mechanism is connected to the rotating plate and responds to the rotation of the steering mechanism to pivot the rotating plate and thereby rotate the ultrasound probe relative to the longitudinal axis; the steering mechanism is configured to tilt the ultrasound probe at an angle in two opposite directions relative to the longitudinal axis, and the angle is up to 180 degrees in at least one of the directions; the steering mechanism can be configured to tilt the ultrasound probe at different angles relative to the longitudinal axis in the two opposite directions; the handle can include (i) a multiple-use portion and image processing electronics coupled to the camera and the ultrasound probe, and (ii) a disposable portion, which is detachably fixed to the multiple-use portion and accommodates the The invention also provides a rotation mechanism and a steering mechanism; the sleeve can be flexibly bent when inserted into the patient's bladder or ureter; the ultrasound image processor can be operably connected to the ultrasound probe, and the ultrasound image display can be configured to display the ultrasound image provided by the ultrasound probe and processed by the ultrasound processor, and the camera image processor and the camera image display can be configured to display the image provided by the camera and processed by the camera image processor; the ultrasound image display and the camera image display can be configured to simultaneously display the ultrasound and camera images on a single screen; the ultrasound and visual aspects can be integrated by inserting the ultrasound probe through the working channel formed in the sleeve; the sleeve is configured to have at least a portion of The handle may include a cannula having a hardness property selected from a group consisting of rigid, semi-rigid and flexible; the DOV may be in the range of 0 to 30 degrees; the cannula rotation mechanism may be positioned at the proximal portion of the handle and operably coupled to the cannula so as to selectively rotate the cannula and thereby rotate the camera relative to the handle about the axis; the probe rotation mechanism may be a probe rotation wheel, and the rotation sensor may be operably coupled to the probe rotation mechanism and configured to provide an electrical signal indicating rotation of the ultrasound probe about the axis; the processing system may be configured to process ultrasound images from the ultrasound probe and automatically generate a three-dimensional ultrasound image therefrom based in part on the electrical signal from the rotation sensor; the probe steering mechanism may be a probe steering wheel.

[0007] According to some embodiments, a medical device includes: an elongated shaft having a distal portion and a proximal portion extending along a longitudinal axis, wherein the shaft is shaped and sized to be inserted into a working channel or sheath of an endoscope cannula configured to be inserted into a patient's body; an ultrasound probe located at the distal portion of the shaft and configured to protrude from the distal end of the sheath or cannula and provide an ultrasound image; a housing fixed to the proximal portion of the shaft; a probe rotation mechanism mounted on or in the housing and operably connected to the shaft to rotate the shaft and thereby rotate the ultrasound probe around the axis within a selected rotation angle range; and a probe steering mechanism mounted on or in the housing and operably connected to the ultrasound probe to tilt the ultrasound probe relative to the axis within a selected tilt angle range.

[0008] The medical device may further include one or more of the following features: a rotation sensor operably connected to the probe rotation mechanism and configured to provide an electrical signal indicating rotation of the ultrasound probe about the axis; and the shape and size of the shaft can be designed to be inserted into a working channel of an endoscope having a camera at the distal end, wherein the ultrasound probe is configured to protrude distally from the camera when the shaft is inserted into the working channel.

[0009] According to some embodiments, a method includes providing an integrated camera and ultrasound imaging device including an elongated cannula having a distal portion extending along a longitudinal axis; inserting the cannula into a subject; operating a camera mounted at the distal portion of the cannula to provide a camera image of the interior of the subject taken with a field of view of the camera angled relative to the axis; operating an ultrasound probe also mounted at the distal portion of the cannula and projecting distally from the camera to provide an ultrasound image of the interior of the subject; and moving the cannula and the camera around a proximal portion of the imaging device under manual control applied to the proximal portion of the cannula. The axis is selectively rotated through selected rotation angles to observe the interior of the object from different directions; under manual control applied to the proximal portion of the imaging device, the ultrasound probe is selectively rotated about the axis and the ultrasound probe is selectively tilted relative to the axis to obtain ultrasound images of selected portions of the object taken from different directions within a solid angle greater than 180 degrees; wherein operating the camera includes including at least a portion of the image of the ultrasound probe in at least some of the images provided by the camera; processing the camera and the ultrasound images; and displaying the resulting processed camera and ultrasound images.

[0010] According to some embodiments, the method may further include one or more of the following: sensing a rotation of the ultrasound probe and controlling the display of an ultrasound image based on the sensed rotation of the ultrasound probe; providing a handle configured to be grasped by a hand, wherein the sleeve is fixed to the handle; providing the handle includes: providing a handle including a disposable handle portion and a multiple-use handle portion, the device being fixed to the disposable handle portion, the multiple-use handle portion being detachably fixed to the disposable portion and including an electronic device for processing the ultrasound image from the ultrasound probe; selectively withdrawing the ultrasound probe from the sleeve while keeping the sleeve inserted into the object, and inserting a surgical instrument into the sleeve lumen vacated by the withdrawn ultrasound probe; selectively bending the sleeve; rotating the sleeve under manual control by rotating a camera and an ultrasound integrated imaging device; selectively rotating the sleeve under manual control by rotating a sleeve rotation mechanism operably connected to the sleeve, thereby rotating the sleeve and thereby the camera; and performing surgery on the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The novel features of the present invention are particularly set forth in the appended claims. The features and advantages of the present invention will be better understood by reference to the following detailed description, which sets forth exemplary embodiments utilizing the principles of the present invention and the accompanying drawings, wherein:

[0012] Figures 1A to 1C is a schematic diagram illustrating an example of a combined ultrasound and endoscopy system (CUES) according to some embodiments;

[0013] Figure 2A , Figure 2B , Figure 2C and Figure 2D are right side, left side, top and front views of a handheld portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments;

[0014] Figure 3A and Figure 3B are perspective and top views of a distal end of a handheld portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments;

[0015] Figure 4A and Figure 4B is a partial perspective view of a cannula forming part of a combined ultrasound and endoscopy system (CUES) according to some embodiments;

[0016] Figure 5A , Figure 5B and Figure 5C is a perspective view showing further details of the tilt, swivel and rotation mechanism for an ultrasound probe forming part of a combined ultrasound and endoscopy system (CUES) according to some embodiments;

[0017] Fig. 6A is a perspective view of further details of a distal tip of a combined ultrasound and endoscopy system (CUES) according to some embodiments;

[0018] Figure 6B is a diagram showing that the distal end portion of the cannula is flexible and tiltably steerable according to some embodiments;

[0019] Figure 6C is a diagram showing that the distal end of the cannula has a fixed steering angle;

[0020] Figure 7 is a perspective view of a flexible handheld portion forming part of a combined ultrasound and endoscopy system (CUES) according to some embodiments;

[0021] Fig. 8A and 8B is a diagram showing further details of a detachable handheld portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments;

[0022] 9A to 9C is a diagram showing further details of the distal tip and cannula of a combined ultrasound and endoscope probe according to some embodiments;

[0023] FIG. 10A to FIG. 10C is a diagram showing further details of the distal tip and cannula of a combined ultrasound and endoscope probe according to some embodiments; and

[0024] Fig.11A and 11B is a schematic diagram illustrating a further example of a combined ultrasound and endoscopy system (CUES) according to some embodiments. DETAILED DESCRIPTION

[0025] A detailed description of an example of a preferred embodiment is provided below. Although several embodiments are described, it should be understood that the new subject matter described in this patent specification is not limited to any one embodiment or combination of embodiments described herein, but includes many alternatives, modifications, and equivalents. In addition, although many specific details are set forth in the following description in order to provide a thorough understanding, some embodiments can be practiced without some or all of these details. In addition, for the sake of clarity, certain technical materials known in the prior art are not described in detail to avoid unnecessarily obscuring the new subject matter described herein. It should be clear that the various features of one or several specific embodiments described herein can be used in combination with the features or other features of other embodiments described. In addition, the same figure numbers and names in the various drawings represent the same elements.

[0026] As used herein, a processor includes one or more processors, such as a single processor or multiple processors of a distributed processing system. A controller or processor as described herein generally includes a tangible medium for storing instructions to implement the steps of a process, and the processor may include, for example, one or more central processing units, programmable array logic, gate array logic, or field programmable gate arrays.

[0027] As used herein, the terms "distal" and "proximal" refer to positions referenced from the device and may be in contrast to anatomical references. For example, the distal position of the probe may correspond to the proximal position of the patient's elongated member, and the proximal position of the probe may correspond to the distal position of the patient's elongated member.

[0028] Although some exemplary embodiments are directed to cystoscopy and / or hysteroscopy, those skilled in the art will appreciate that this is not intended to be limiting and that the devices described herein may be used for other therapeutic or diagnostic procedures and other anatomical regions of a patient's body.

[0029] According to various embodiments, a device includes a probing portion for direct insertion into a body cavity. The probing portion is proximate to a tissue and / or region to be examined. As used herein, a probe encompasses an object inserted into a subject such as a patient.

[0030] According to some embodiments, an endoscopic ultrasound hysterography and cystography system (EUHCS) is described. According to some embodiments, the system may be more generally described as a combined ultrasound and endoscopy system (CUES). EUHCS and CUES are medical devices that allow doctors to obtain video images and organ information from the inside of the uterus, bladder, or other organs. According to some embodiments, the endoscopic image and the ultrasound image are displayed simultaneously in real time on a monitor or two separate monitors, allowing the doctor to see the surface and internal tissue of the organ, and can be electronically transmitted to other devices, such as a workstation and / or PACS (Picture Archiving and Communication Systems) that may be located at a remote location.

[0031] In the case of gynecological clinical applications, according to some embodiments, the applications of EUHCS include: (1) early diagnosis of endometrial cancer; (2) providing endometrial cancer staging information based on the depth, area and range of cancer infiltration into the myometrium; (3) monitoring hysteroscopic surgery to improve surgical safety, accuracy and success rate; (4) detecting, diagnosing and determining the stage of ovarian cancer and / or fallopian tube cancer, and detecting and diagnosing fallopian tube obstruction.

[0032] In providing a combined ultrasound and endoscopic image of the uterus, the apparatus and method of the patent specification can illustrate the shape of the uterus. The cervical canal and part of the uterine cavity are cylindrical. The lower surface of the upper uterine fundus is horizontally oriented relative to the cervical canal and the lower uterine cavity. According to some embodiments, the hysteroscope and the ultrasound probe are configured to image in the vertical and horizontal directions, so that an ultrasound image of the plane of the uterine fundus can be obtained.

[0033] Similarly, when a combined ultrasound and endoscopic image of the bladder is provided, the shape of the bladder can be illustrated. The bladder is spherical. According to some embodiments, a flexible cystoscope and an ultrasound catheter probe can be used to image the entire bladder or at least a desired portion thereof.

[0034] Figures 1A to 1C is a schematic diagram showing an example of a combined ultrasound and endoscopy system (CUES) according to some embodiments. Figure 1AIn the embodiment, the system 100 includes a handheld portion 110 and a tower system 112 interconnected by cables 134, 136 and processing units 180, 182 and 184. According to some embodiments, the handheld portion 110 is configured as a disposable unit and can be discarded after a single use. According to some other embodiments, the handheld portion 110 is configured to be divided into an upper disposable portion 120 and a lower multiple-use handle portion 130. In this case, the disposable portion 120 is detachable from the handle portion 130, such as along the dotted line 132, so that the handle portion 130 is configured to be reusable. According to some embodiments, the same multiple-use portion matching of the disposable portion of different types of forms can be made. In the example shown in Figure 1, three forms of disposable portions in sterile packaging or pouches 121, 122 and 123 are shown. In the pouch 121, a complete handheld portion 110 is provided. In pouch 123, a detachable upper disposable portion is provided to mate with the same or a similar multiple-use portion transported in pouch 121, and in pouch 124, a flexible sleeve form is provided. As will be described in further detail below, all devices may include a camera module located at the distal end, an LED lighting and ultrasound transducer module, and one or more lumens for conveying fluids. The tower system 112 includes a support column 140 mounted on a wheeled base 142. The tower system 112 includes two displays 150 and 152, a keyboard 160, a mouse 162, and a processing system 170. The processing system 170 may functionally include an ultrasound image processing unit 182, an endoscopic optical image processing unit 180 (labeled as a hysteroscope unit in the drawings), and a graphics unit 184 that performs image display and management processing. According to some embodiments, the display 150 is configured to display an ultrasound image 154 from an ultrasound transducer module located at the distal end of the unit 110, and the display 152 is configured to display a real-time direct visual image from a camera module located at the distal end of the unit 110. According to some embodiments, the display monitors 150 and 152 can be touch-sensitive to receive user input and high resolution. According to some embodiments, each display 150 and 152 is configured to display high-definition graphics at a pixel resolution of 1280x720, 1920x1080, 2048x1080, 2560x1440, 3840x2160 or higher. According to some embodiments, the camera image display 152 displays a real-time video image 156 in an anatomical orientation of the interior of a human body cavity and an anatomical relative position of the camera end. The ultrasound image 154 displayed on the display 152 is generated by the ultrasound transducer at a known position relative to the camera. According to some embodiments, ultrasound images and camera images may be accurately correlated to locations and orientations within a body cavity (eg, uterus 102 shown in FIG. 1 ).The side-by-side display of the anatomical ultrasound 154 and the camera image 156 allows the physician to simultaneously see the ultrasound image guided by the camera image in real time. According to some embodiments, the camera module on the distal end of the unit 110 is configured and mounted so that at least a portion of the ultrasound probe 252 is visible to the operator in the camera image 156 as the ultrasound probe portion 158. Providing the ultrasound probe portion 158 on the real-time camera image 156 can provide valuable information about the current direction and position of the ultrasound probe 252 and feedback to the operator.

[0035] According to some embodiments, the processing system 170 includes an ultrasound image processing unit 180, an endoscopic image processing / hysteroscopy unit 182, and an image display and management system / graphics unit 184. The handheld unit 110 is connected to the ultrasound processing unit 182 and the hysteroscopy unit 180 by cables 134 and 136, respectively. The processing system 170 may also include a suitable personal computer or workstation, which includes: one or more processing units 174; input / output devices such as CD and / or DVD drives; internal storage devices such as RAM, PROM, EPROM and magnetic type storage media, such as one or more hard disks 172 for storing medical images and related databases and other information; and a graphics processor suitable for powering the graphics displayed on the displays 150 and 152. According to some embodiments, the tower system 112 is powered by a medical grade power supply (not shown). According to some embodiments, the fluid control system 186 is attached to the handheld portion 110 via the fluid line 132. In some cases, there are two fluid lines so that the inflow and outflow of fluids can be controlled.

[0036] According to some embodiments, Figure 1B Shows something like Figure 1A systems with different display configurations. Figure 1B As shown, a single display 150 can be used instead of two displays. The endoscopic image 156 and the ultrasound image 154 are combined by the graphics unit 184. As shown, the endoscopic image 156 and the ultrasound image 154 are displayed side by side simultaneously on the single monitor 150 at high resolution. In some cases, this display configuration can provide better visual effects for doctors during diagnosis and surgery.

[0037] Figure 1C Another example of a combined ultrasound and endoscopy system (CUES) according to some embodiments is shown. In this example, the CUES system 100 includes a handheld portion 110 that is attached to two separate tower systems 116 and 118. Each system 116 and 118 can be connected to Figure 1A and Figure 1B The tower system 112 shown is the same or similar. Figure 1C In the illustrated case, an ultrasound image 154 is displayed on a monitor 150 of a tower system 116, and an endoscopic image 156 is displayed on a monitor 152 of a tower system 118. The two tower systems 116 and 118 are positioned so that the monitors 150 and 152 are placed side by side so that the physician can view both images simultaneously. It should be noted in this case that a hysteroscopy unit 180 may be included in the tower system 118, and an ultrasound processing unit 182 may be included in the tower system 116.

[0038] Figure 2A , Figure 2B , Figure 2C and Figure 2D 1 is a right side view, a left side view, a top view, and a front view of a handheld portion of a combined ultrasound and endoscope system (CUES) according to some embodiments. The handheld portion 110 includes a cannula 240 and a distal end 250. According to some embodiments, the cannula 240 can be rigid, flexible (e.g., Figure 7 The distal end includes an ultrasound probe 252 that can be rotated and "steered". In particular, the ultrasound probe 252 is configured to rotate around a central longitudinal axis 254 as shown by a dashed arrow 256, and to steer as shown by a dashed arrow 258 to angle the ultrasound probe 252 relative to the axis 254. In some embodiments, both the rotation and steering of the probe 252 are controlled at the proximal end 260 of the cannula 240. According to some embodiments, at the proximal end 260, the steering of the probe 252 is controlled by turning a wheel 262, and the rotation of the probe 252 is controlled by turning a wheel 264. According to some embodiments, a mechanism other than a wheel may be used to control the rotation of the probe 252. Such mechanisms include, but are not limited to, levers, sliders, rods, and trackballs.

[0039] According to some embodiments, the ultrasound probe 252 includes an ultrasound transducer made of a conventional piezoelectric material such as PZT or of a semiconductor material. The dimensions of the transducer may be 2-4 mm wide and 10-20 mm long. The transducer in the probe 252 may include up to 128 (or more) elements. See, e.g. Fig. 6A610 in the ultrasound transducer array. According to some embodiments, the ultrasound transducer array can be configured as a linear or phased array, or a single element or linear transducer with frequencies from 5 MHz to 10 MHz for general imaging covering the entire uterus and bladder; and 10 MHz to 30 MHz for endometrial and superficial bladder images. According to some embodiments, when fewer transducer elements are used in the array in the probe 252, such as 64 or fewer elements, each transducer element can have its own cable. The cables can be bundled together and connected to an ultrasound processing unit (e.g., Figure 1A According to some embodiments, the ultrasonic processing unit 182 may be integrated into the processing system 170 (shown in FIG. 1 ) and form a part of the processing system 170, and in some other embodiments, part or all of the ultrasonic processing unit 182 may be separated from the processing system 170. According to some embodiments, in the case where the handle 130 is configured as part of a disposable portion, such as Figure 2A As shown, the handle 130 may include a compact ultrasonic processing unit 272. Compared to the case where the handle is configured for non-disposable use, for example Fig. 8A In the handle 830 shown in FIG. 1 , the compact ultrasonic processing unit 272 in the handle 130 can be manufactured at a lower cost and perform fewer functions. According to some embodiments, in order to reduce the number of cables extending from the probe 252, an ASIC can be included in the probe 252. The ASIC can include high-voltage switches and control circuits to drive the individual transducer elements and route the echo signals to the processing unit 252. In this case, a reduced number of coaxial cables can then be used to transmit and receive ultrasonic signals, control signals, and electrical power between the probe 252 and the ultrasonic processing unit 182. Fig. 6A In the present invention, the ultrasound probe 252 can also be designed as or similar to a conventional ultrasound catheter, such as an intracardiac echocardiogram catheter (ICE) and an intravascular ultrasound catheter (IVUS). The ultrasound transducer can be a phased array, a linear array, and a single element, with a transmitter frequency from 5 MHz to 30 MHz. The transducer is mounted on the end portion of a rigid or flexible drive shaft. The drive shaft can rotate 360 ​​degrees, and the electronic device and mechanical unit can be moved forward or backward manually or by a motor. The rotation and forward movement of the probe acquires a 3D volume image, and thus, a real-time 3D intracavitary image can be displayed. When the probe drive shaft is positioned parallel to the cavity surface, the cavity surface ultrasound signal can be acquired and displayed in a B-mode format and a panoramic format.

[0040] like Figure 6BAs shown, the cannula 620 has two parts. The cannula end portion 630 is a flexible tube with a length of less than 5 cm. The other part of the cannula is a rigid tube 640. The flexible tube can be made of medical grade polyurethane elastomer material. The flexible end portion 630 can be turned + / - 90 degrees. The cross-section of the flexible tube is similar to Fig. 6A In another design, the sleeve 650 may have a fixed angle. The fixed angle may be 5 to 30 degrees. The fixed angle end portion is Figure 6C The steerable cannula tip portion allows a flexible ultrasonic head and cable assembly, a flexible drive shaft, and a flexible surgical tool to pass through the cannula working channel. The flexible cannula tip portion can be implemented as Fig. 9B The single working channel cannula shown has a cross section at Fig. 9C Using a flexible steering or fixed angle cannula tip portion, the transducer surface and the cavity surface can be aligned in a parallel position, thereby acquiring an ultrasound image of the cavity surface.

[0041] Refer again FIG. 2A to FIG. 2D , the cannula 240 can be long, thin and rigid or semi-rigid. According to some embodiments, the cross-section of the cannula 240 perpendicular to its main longitudinal axis can be substantially circular. It should be noted that the cross-section can have any suitable shape, such as an elliptical shape. The diameter of the cannula can vary depending on the type of endoscope, for example, from 3 mm to a maximum of 15 mm. The cannula can include a working channel (at Figure 2D280 is shown in the figure). The working channel can be accessed from a port (not shown) on the rear end of the proximal end 260. The cannula 240 may include one or more fluid channels in fluid communication with various fluid ports. The cannula 240 may include a channel shared by the inflow and outflow. Optionally, the cannula may include two or more channels with separate inflow and outflow. According to some embodiments, the cannula 240 also includes one or more fluid chambers isolated from the working channel fluid. The fluid chamber may lead to fluid ports 246 and 244 respectively disposed on the left and right sides of the distal end of the cannula 240. The fluid chamber may be in fluid communication with fluid ports 230 and 232 at the bottom of the handle 130. According to some embodiments, the right side fluid port 230 is connected to the right side fluid port 244, and the left side fluid port 232 is connected to the left side fluid port 246. The sleeve 240 is also configured to house a plurality of electrical conductors for providing power, control signals, and receiving video / image data from the camera, lighting module 270, and ultrasound probe 252 at the distal end 250. In some cases, the conductors may be insulated and placed in separate lumens within the sleeve 240, in other cases, some or all of the conductors may be placed in lumens used for other purposes (e.g., fluid and / or device / tool ​​channels), and in other cases, the conductors may be placed on the outer wall of the sleeve 240, and the conductors may be wrapped with heat shrink tubing and fixed to the outer wall of the sleeve 240. (Not shown in the figure). According to some embodiments, one or more optical fibers may pass through the sleeve 240 for data transmission and / or illumination light supply to the distal end 250. The sleeve 240 may also include one or more separate lumens for one or more rods or bars, which are used to control the rotation and / or steering of the ultrasound probe 252. 9A to 9C and 10A to Fig. 10C Further details of possible sleeve designs are shown. The handle portion 130 includes a body whose size and shape allow the operator's hand to grip securely and ergonomically. The handle portion 130 also includes one or more buttons 212, which can be configured to allow routine tasks to be performed during use. For example, the button 212 can be programmed to control the LED illumination (of the LED at the distal end 250), capture still images and / or start and stop recording video images, and capture and / or start and stop recording ultrasound data and / or ultrasound images. According to some embodiments, an upper housing 242 can also be provided as shown.

[0042] According to some embodiments, the camera and lighting module 270 at the distal end of the cannula 240 includes a camera module having a field of view of approximately 80 to 120 degrees. According to some embodiments, the camera module can be mounted so that its direction of view (DOV) is at an oblique angle relative to the main longitudinal axis of the cannula 240. According to some embodiments, the oblique angle is between 0 and 30 degrees. Optical components such as prisms can also be used to provide the oblique angle. Further details of the technology for changing the field of view of the camera module can be found in co-pending U.S. Patent Application Serial No. 16 / 268,819, which is incorporated herein by reference.

[0043] According to some embodiments, cannula 240 is connected to a rotation wheel 290 located near the proximal end of upper housing 242. By turning cannula rotation wheel 290 as indicated by dashed arrow 292, cannula 240 and camera and illumination module 270 may also be rotated as indicated by dashed arrow 294. According to some embodiments, cannula 240 may be configured to rotate 180 degrees (or more) so that camera and illumination module 270 may provide at least a 180 degree field of view angle in the uterus, bladder, and other organ cavities.

[0044] According to some embodiments, the handle portion 130 includes a set of electronic devices 274 that filter and transmit raw image data to the hysteroscopic image processing unit 180, which can be part of the processing system 170 (shown in FIG. 1 ). According to some embodiments, a rotation sensor 276 is included on the wheel 264 and is configured to measure or detect the rotational position of the ultrasound probe 252. The rotation sensor 276 data is used by the ultrasound processing unit 182 (shown in FIG. 1 ) to construct a three-dimensional ultrasound image.

[0045] Figure 3A and 3B is a perspective view and a top view of the distal end of a handheld portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments. Figure 3A , the camera and lighting module 270 is shown to include a camera module 320 and two LEDs 330 and 332. Figure 3B, dashed outlines 312 and 314 illustrate the steering range of the ultrasound probe 252. The solid outline shows the probe 252 in a vertical or "middle" position 310. In the example shown, the probe 252 pivots around the hub 312 and can be positioned approximately 90 degrees in either direction, as shown. According to some embodiments, fluid flushing can be provided by allowing fluid to flow into and / or out of the working channel distal port 280 and / or the side fluid ports 246 and 244. In one case, the working channel distal port 280 is used to "flow in" or flow fluid into the organ / tissue of interest, while the side fluid ports 246 and 244 are used to "flow out" or receive fluid flowing out of the organ / tissue of interest. In this case, one of the fluid ports 230 and 232 at the bottom of the handle 130 can be used for inflow, while the other can be used for outflow.

[0046] Figure 4A and 4B is a partial perspective view of an ultrasound probe forming part of a combined ultrasound and endoscopy system (CUES) according to some embodiments. An ultrasound probe assembly 410 is shown that is configured for deployment through a channel of an endoscope cannula. Figure 4A Further details of the steering control of the ultrasound probe 252 are shown. The rotation of the wheel 262 at the proximal end 260 as shown by the dashed arrow 462 controls the left and right steering of the probe 252 as shown by the dashed arrow 452, the dashed outlines 412 and 414. According to some embodiments, mechanisms other than wheels can be used to control the steering of the probe 252. Such mechanisms include, but are not limited to, levers, sliders, rods, and trackballs. Also shown is an axis 440 that extends the length of the assembly 410 from the probe 252 to the proximal end 260. Figure 4B 4 shows further details of the rotation control of the ultrasound probe head 252 about the axis 254 as indicated by arrow 454. The rotation of the probe 252 is controlled by rotating the wheel 264 as indicated by arrow 464. According to some embodiments, Figure 4A and Figure 4B The entire ultrasound assembly 410 shown in the figure can be configured to be used in the working channel of a conventional endoscope system to provide ultrasound functionality to the endoscope system, thereby forming a combined ultrasound and endoscope system (CUES). In this case, a cable connector (not shown) is provided at the proximal end 260 of the assembly 410. The cable connector is used to provide power to the ultrasound probe, control, and transmit and receive ultrasound signals and / or data to and from the ultrasound probe 252.

[0047] Figure 5A , Figure 5B and Figure 5C is a perspective view of further details of a mechanism for steering and rotating an ultrasound probe forming part of a combined ultrasound and endoscopy system (CUES) according to some embodiments. Figure 5A Further details of steering of the ultrasound probe 252 are shown. Steering motion is shown by dashed arrow 552. According to some embodiments, steering is controlled by axial translation of the push rod 520 as shown by arrow 518. Moving the rod 520 forward or backward pushes or pulls the rotating plate 550, which pivots about the axis 512 on a hub (not shown). According to some embodiments, the rotating plate 550 is configured to provide a range of motion of at least 120 degrees. In some embodiments, the steering system can be configured to provide an equal amount of steering range in either direction, but in other embodiments, the steering ranges are not equal, providing greater steering / probe deflection in one direction. Note that by providing rotation of the probe 252 and / or rotation of the entire cannula shaft 240, the ultrasound transducer array 610 (e.g., Fig. 6A ) can be positioned relative to tissue of interest over a relatively wide range, thereby enhancing the ability to image clearly and usefully ultrasound images.

[0048] Figure 5B Further details of the rotation of ultrasound probe 252 are shown. The rotational movement about axis 254 is controlled by moving belt 570 as shown by arrow 574. Belt 570 rotates gear member 540 about axis 542. Gear member 540 has a bevel gear that meshes with a bevel gear on the proximal end of probe 252. Figure 5B Also visible is a steering control push rod 520. According to some embodiments, the probe 252 can be configured to have a rotational range of motion about the axis 254 of approximately 360 degrees.

[0049] Figure 5C Further details of the operation of the distal end steering and rotation control wheels 262 and 264 are shown. The steering control wheel 262 rotates as shown by arrow 536. Wheel 262 is connected via a rod to a bevel gear that meshes with a bevel gear on wheel 566 that rotates as shown by arrow 536. Wheel 566 has another gear face that meshes with a worm gear formed on the steering control push rod 520. The axial movement of the applied rod 520 is represented by arrow 526. Figure 5CThe rotation control wheel 264 is also shown to rotate as shown by arrow 564. The wheel 264 rotates a bevel gear that meshes with another bevel gear, which in turn moves the belt 570 as shown by arrow 572. A rotation sensor 276 is also shown. As shown, the rotation sensor 276 can be connected to the hub of the wheel 264, or can be positioned inside the rotating wheel 264. The rotation sensor 276 can be an optical encoder or another instrument such as a potentiometer. The rotation sensor 276 provides the position of the ultrasound transducer that acquires the ultrasound data frame. The position information is attached to the ultrasound data frame that will be used to construct a 3D ultrasound image. The rotation position information can also be sent to the ultrasound processing unit 182. According to some embodiments, the position information can be used to trigger the ultrasound timing control unit to start transmitting ultrasound and receiving ultrasound data. Using the rotation position sensed from the sensor 276, the distance between two frames of ultrasound images can be accurately controlled to a spatial resolution of about 0.1 mm.

[0050] Fig. 6A 2 is a perspective view of further details of the distal end of a combined ultrasound and endoscope system (CUES) according to some embodiments. The distal end 250 includes a camera and lighting module 270, a working channel distal port 280, and an ultrasound probe 252. According to some embodiments, the working channel has an inner diameter of about 0.5 mm to 3.5 mm, so that standard surgical devices can be arranged therein to perform various surgical procedures. Examples of such devices include: biopsy needles, injection needles, forceps, tubes, knives, snares, probes, coagulator devices, brushes, laser devices, microwave devices (e.g., for ablation), and photodynamic tools. The camera and lighting module 270 includes a camera module 320 and two LEDs 330 and 332. According to some embodiments, the camera module 320 may include, for example, a CMOS image sensor of model OVM6946 from OmniVision Technologies, Inc. According to some embodiments, camera module 320 is approximately 1.05 mm x 1.05 mm, has a resolution of approximately 400 x 400 pixels, has a field of view of approximately 120 degrees, and can capture video at 30 frames per second.

[0051] Figure 7 1 is a perspective view of a flexible handheld portion forming part of a combined ultrasound and endoscopy system (CUES) according to some embodiments. The flexible handheld portion 710 is configured to be connected to a tower system, such as tower system 112 via cables 134 and 136, as shown in FIG. 1. Referring again to Figure 7 , the sleeve 740 has a flexible cable that provides the sleeve 740 with a plurality of directions to turn or bend. According to some embodiments, for example, Figure 3B and Figure 5A-Figure 5CAs shown, mechanical controls for rotation and steering may be provided. The handheld portion 710 includes a proximal handle portion 734 that is configured for an ergonomic grip for the hand. According to some embodiments, the flexible handheld portion 710 includes a distal end 750 that includes a camera and lighting module 770, an ultrasound probe 752, and a working channel distal port 780. These components may be similar or identical to the camera and lighting module 270, ultrasound probe 252, and working channel distal port 280 described and shown elsewhere herein. In the case where the handheld portion 710 is integrated with mechanical rotation and steering controls, the ultrasound probe 752 need not include self-rotation capabilities. According to some embodiments, the ultrasound probe 752 is configured to reduce the steering and / or rotation range of motion when compared to the probe 252 shown and described elsewhere herein. According to some embodiments, the two proximal fluid ports 730 and 732 are configured to provide fluid inflow and fluid outflow, and are similar to Figure 2A-2D Fluid ports 230 and 232 are shown.

[0052] According to some embodiments, such as Figure 7 The flexible CUES shown in the figure may be more suitable for male urological patients than the rigid CUES. For example, when using the flexible CUES, it may be easier for male patients to access locations in the bladder and reduce pain. According to some embodiments, the flexible CUES may be used to better access the ureters and perform ultrasound scanning of the kidneys. The flexible CUES may also be more suitable for reaching and acquiring ultrasound images of certain locations within the uterus that cannot be easily accessed by a rigid hysteroscope.

[0053] Fig. 8A and 8B8 is a diagram showing further details of a detachable handheld portion of a combined ultrasound and endoscope system (CUES) according to some embodiments. In this case, the handheld portion 810 has many components similar and / or identical to the handheld portion 110 shown and described elsewhere herein. In this case, the handheld portion 810 is configured to be divisible into an upper disposable portion 840 and a lower multiple-use portion 830. The two parts 830 and 840 can be matched and unmatched with each other by manual operation without the need for tools. Specifically, the handle portion 830 includes a socket 860, the size of which is arranged to be connected to a convex mating portion 861 protruding from the disposable portion 840. The action of installation and removal is shown by a dotted arrow 866. Electrical connectors 862 and 864 protrude from the mating portion 861. According to some embodiments, the handle portion 830 can accommodate or include components configured to process image data, generate control signals, provide power, or establish communication with other external devices. In some cases, communication can be wireless or wired communication. For example, wireless communication can include Wi-Fi, radio communication, Bluetooth, IR communication or other types of direct communication. In some embodiments, the handle portion can accommodate a sensor assembly to measure the relative position between the sleeve and the handle portion. In other embodiments, the sensor assembly can measure the relative position or direction of the handle relative to its environment. The example of such a sensor assembly is further described below. Figure 8B The disposable second-use portion 840 is shown provided in a sterile package or pouch 123. Since the lower portion of the handle 830 is configured for multiple uses (rather than single use, e.g. Figures 2A-2D 1 ), according to some embodiments, greater processing capabilities may be included therein. For example, an electronics unit 872 may be included in the lower handle portion 830 to provide some or all of the functionality of the ultrasound processing unit 182 (shown in FIG. 1 ).

[0054] Further details of the operation of the combined ultrasound and endoscopy system (CUES) will now be provided. In the case of a rigid endoscope and ultrasound probe, such as the handheld portion 110 and 810 shown and described herein, the following sequence may be used: (1) a camera module 320 (e.g., at the end of the endoscope) is provided. Figure 3A and Fig. 6A (2) To image the uterine and cervical canal, the ultrasound probe is moved along its vertical position (as shown in the figure). Figure 3B The rotation control wheel 264 is manipulated to rotate the ultrasound probe 252 about the axis 254 (eg, Figure 5BAs shown in FIG. 1 ). Thus, an ultrasound image of a vertical cross-section of the cervical wall and / or uterine wall is obtained. (3) The distal end 250 is moved into the uterine or bladder cavity. The steering control wheel 262 is manipulated to adjust the probe 252 to a desired angle, and then the rotation control wheel 264 is manipulated to rotate the ultrasound probe 252 around the axis 254 (as shown in FIG. 1 ). Figure 5B ), thereby obtaining an ultrasound image of the upper uterus or upper bladder wall.

[0055] In the case of flexible endoscopes and ultrasound probes, such as Figure 7 The handheld portion 710 shown can be used in the following sequence. (1) The distal end 750 of the flexible sleeve 740 is inserted into the uterine or bladder cavity. During insertion, the ultrasound probe 752 can be in a retracted position, in which the distal end of the probe 752 is recessed proximally from the distal end of the sleeve 740. (2) The real-time video endoscope image from the camera module is used to find the area where the ultrasound image should be taken. (3) The ultrasound probe is pushed out distally so that the probe 752 protrudes distally from the distal end 750 (as shown in FIG. 1 ). Figure 7 (4) Use the ultrasound steering control wheel 762 to adjust the probe to a desired angle, and then manipulate the ultrasound rotation control wheel 764 to rotate the ultrasound probe 752 to a desired angle, thereby acquiring a suitable ultrasound image.

[0056] 9A to 9C is a diagram showing further details of the distal tip and cannula of a combined ultrasound and endoscope probe according to some embodiments. Fig.9A is a schematic cross-sectional view showing cannula 240 including an upper lumen or channel 910 and a lower lumen or channel 920 . Fig. 9B is a perspective view of the distal tip 250 showing the ultrasound probe assembly 410 deployed in the lower channel 920. 9A to 9C In the example shown, the ultrasound probe assembly 410 is deployed in a cannula 240 configured with a single working channel (channel 920). In some cases, the cannula 240 may include multiple working channels, such as FIG. 10A to FIG. 10C shown. Fig. 9C A cross section of a cannula 240 having a single working channel is shown. In this case, the cannula 240 has an outer wall 900 that accommodates four separate channels. An upper channel 910 is used for cables to and from the camera module 320, LEDs 330 and 332, and possibly other cables and / or fiber optic cables. Two side channels 930 and 932 are used for outflow of fluid (i.e., outflow of fluid from an organ or cavity of the patient into the device). The side channels 930 and 932 can be fluidically connected to the side fluid ports 244 (e.g., Figure 2A and 3B) and 246. The lower channel 920 is shared by the inflow fluid (i.e., the fluid flowing into the organ or cavity), the ultrasound probe 410, and possibly other surgical tools. According to some embodiments, the inner diameter of the lower channel or working channel 920 ranges from 1 mm to 4 mm. According to some embodiments, as Figures 9A-9C As shown, in the case of a single working channel, the overall outer diameter of the endoscope sleeve 240 is less than about 8 mm. During surgery, the uterine or bladder cavity is filled with fluid through the channel 920. Surgical tools can be inserted into the cavity to perform the surgery. During surgery, the doctor can withdraw the surgical tools and insert the ultrasound probe 410 through the working channel 920 to measure the thickness of the cavity wall, the shape of the surgical object, and obtain other information so that the doctor can make a decision to complete the surgery.

[0057] Figures 10A-10C is a diagram showing further details of the distal tip and cannula of the ultrasound and endoscope combination probe according to some embodiments. In the case shown, the cannula 240 includes two working channels. Fig. 10A is a perspective view of the distal tip 250 in the case where the cannula 240 includes two working channels 920 and 1020. In the illustrated case, the ultrasound probe assembly 410 is deployed through the working channel 920 when the working channel 1020 is empty. Fig. 10B is a cross section of the cannula 240. As with the single working channel, there is an upper channel 910 for cables for the camera module 320, LEDs 330 and 332, and possibly other power and / or fiber optic cables. Similarly, two side channels 930 and 932 are used to connect the side fluid ports 244 (such as Figure 2A and Figure 3B The working channel 920 is used for the ultrasound probe assembly 410, while a separate working channel 1020 can be used for liquid inflow and surgical tools. According to some embodiments, the diameter of each working channel 920 and 1020 ranges from 1 mm to 4 mm, and the overall outer diameter of the endoscope cannula 240 is less than about 10 mm. During surgery, the uterine or bladder cavity is filled with liquid through the channel 1020, and the ultrasound probe 410 is inserted into the uterine or bladder cavity through the channel 920. After the uterine or bladder cavity is properly filled with liquid (e.g., saline), the surgical tool is inserted into the uterine or bladder cavity through the channel 1020. During surgery, the use of surgical tools can be temporarily stopped while the ultrasound probe is moved to the surgical area to generate an ultrasound image. The ultrasound image can provide guidance on the thickness of the uterine or bladder cavity wall, the shape of the surgical object, and other information for making surgical decisions. If a biopsy operation is being performed, the ultrasound image can guide the biopsy tool to collect a tissue sample.

[0058] Fig.11A and11B is a schematic diagram illustrating further examples of combined ultrasound and endoscopy systems (CUES) according to some embodiments. In these examples, an ultrasound probe assembly 410 is inserted into a working channel of a conventional reusable rigid, semi-rigid, or flexible hysteroscope, cystoscope, or other conventional endoscopy system to form a combined ultrasound and endoscopy system (CUES). The ultrasound probe assembly 410 may be similar or identical to the ultrasound probe assemblies described elsewhere herein, including the controllable steering and rotation capabilities of the probe 252. Fig.11A In the embodiment, the CUES system 1100 is formed by inserting the ultrasound probe assembly 410 through the working channel of the rigid hysteroscope 1110. Fig.11A and Fig. 11B As shown, the shaft 440 of the ultrasound probe assembly 410 is inserted into the working channel of the cannula 1120 so that the ultrasound probe 252 protrudes from the distal end 1150 of the hysteroscope 1110. The hysteroscope 1110 includes a proximal handle portion 1130 configured for ergonomic hand gripping. In FIG. 11A, the CUES system 1100 includes two tower systems 116 and 118, which are connected to the tower systems 116 and 118. Figure 1C The tower system shown in FIG. Fig.11A In the case of hysteroscope 1110, hysteroscope unit 180 of endoscope tower system 118 is attached via cable 1136. Endoscope tower system 118 includes monitor 152, which is configured to display hysteroscopic image 156, as shown. Fluid lines leading to the fluid control system are not shown in the figure. Conventional hysteroscope 1110 and tower system 118 can be conventional, independent endoscope systems. Ultrasound assembly 410 is connected to ultrasound processing unit 182 via cable 134, in which case, ultrasound processing unit 182 is located in ultrasound tower system 116. Ultrasound tower system 116 includes monitor 150, which is configured to display hysteroscopic image 154, as shown. In this way, ultrasound probe assembly 410, cable 134 and tower system 116 form an independent ultrasound unit. By positioning tower systems 116 and 118 close to each other so that the operator can see monitors 150 and 152 simultaneously, the CUES system 1110 can provide effects as described elsewhere herein, including: allowing the operator to see the surface and internal tissue of an organ; and providing ultrasound images guided in real time by camera images, thereby enhancing the efficiency of various diagnostic and surgical procedures.

[0059] Examples of surgeries that can be performed using the combined ultrasound and endoscope system (CUES) described herein include, but are not limited to: detection, screening, and / or diagnosis of endometrial cancer; ultrasound-based surgical planning; ultrasound-based treatment planning; surgical monitoring; monitoring of tubal patency surgery, and uterine surface roughness assessment. According to some embodiments, the combined ultrasound and endoscope system (CUES) described herein can be used to monitor gynecological and urological surgeries, such as: uterine wall resection; endometrial ablation; endometrial resection; submucosal myomectomy; intramural myomectomy; transmural myomectomy; cervical and / or endocervical resection; prostate resection and uterine myomectomy. The combined ultrasound and endoscope system (CUES) described herein can also be used to make measurements, such as: uterine wall thickness; endometrial thickness; polyp size; prostate thickness; intrauterine measurement; urethral thickness. The combined ultrasound and endoscope system (CUES) described herein can also be used to generate three-dimensional images of various organs and body parts, such as: ovaries; fallopian tubes; uterus; prostate; and various tumors and / or polyps.

[0060] Although the treatment plans and definitions of treatment regimens and amounts described herein are presented in the context of urological or gynecological diagnosis or surgery, the methods and devices described herein can be used to treat any tissue and any organ and vessel of the body, for example: brain; heart; lungs; intestines; eyes; skin; kidneys; liver; pancreas; stomach; uterus; ovaries; testicles; bladder; ears; nose; mouth; soft tissues such as bone marrow, adipose tissue, muscle, glandular and mucosal tissue, spinal cord and nerve tissue, cartilage; hard biological tissues such as teeth, bones, etc.; and body cavities and passages, such as sinuses, ureters, colon, esophagus, lung passages, blood vessels and throat.

[0061] The embodiments disclosed herein can combine one or more of a variety of approaches to provide improved diagnosis and treatment to patients. The disclosed embodiments can be combined with existing methods and devices to provide improved treatment, such as combined with known methods of urology or gynecology diagnosis, surgery, and surgery of other tissues and organs. It should be understood that any one or more structures and steps as described herein can be combined with any one or more additional structures and steps of the methods and devices as described herein, and the accompanying drawings and supporting text provide a description consistent with the embodiments.

[0062] Although preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that these embodiments are provided only by way of example. Without departing from the present invention, those skilled in the art will now appreciate many variations, changes and replacements. It should be understood that various alternatives to the embodiments of the present invention described herein may be used to implement the present invention. The appended claims are intended to define the scope of the present invention, and thus encompass methods and structures within these claim ranges and their equivalents.

Claims

1. An integrated visual and ultrasonic device comprising: an ergonomic handle for hand grasping and having a proximal portion and a distal portion; a cannula extending distally from the distal portion of the handle and having a distal portion extending along a longitudinal axis; a distally facing camera affixed to the distal portion of the cannula and having a camera field of view (FOV) encompassing a selected solid angle and a camera direction of view (DOV) angled relative to the longitudinal axis; an ultrasound probe positioned at the distal portion of the cannula for rotation about the longitudinal axis relative to the distal portion of the cannula and for tilting relative to the distal portion of the cannula; a probe tilting and steering mechanism mounted at the proximal end of the handle and operably coupled to the ultrasound probe to selectively tilt the ultrasound probe relative to the distal end portion of the cannula within a selected angle range; and a probe rotation mechanism mounted on the proximal end of the handle and operably coupled to the ultrasound probe to selectively rotate the ultrasound probe relative to the cannula about the longitudinal axis, wherein the cannula further comprises at least one lumen, and further comprises a shaft connecting the probe tilt steering mechanism and the ultrasound probe, the shaft being configured to tilt the ultrasound probe relative to the distal end portion of the cannula, wherein the shaft is removably received in the lumen, and The device further comprises: a rotation plate to which the ultrasound probe is secured and which rotates about a pivot axis transverse to the longitudinal axis; and a rod within the shaft that couples the probe tilt steering mechanism to the rotation plate and that pivots the rotation plate, and thereby pivots the ultrasound probe relative to the distal end portion of the cannula, in response to manual actuation of the probe tilt steering mechanism; and A gear member is located at a fixed position within the shaft, the gear member is connected to the ultrasound probe via a bevel gear, and the gear member is connected to the probe rotation mechanism via a belt, so that in response to manual actuation of the probe rotation mechanism, the gear member is rotated about an axis transverse to the longitudinal axis to selectively rotate the ultrasound probe about the longitudinal axis.

2. The device of claim 1, wherein the probe rotation mechanism is configured to rotate the ultrasound probe at least 180 degrees.

3. The device according to claim 1, wherein the steering mechanism is configured to tilt the ultrasound probe at an angle relative to the longitudinal axis in two opposite directions, the angle being up to 180 degrees in at least one of the directions. 4 . The device according to claim 3 , wherein the steering mechanism is configured to tilt the ultrasound probe in different angular ranges along the two opposite directions relative to the longitudinal axis.

5. The device of claim 1, wherein the handle comprises: (i) a multiple-use portion and image processing electronics therein coupled to the camera and the ultrasound probe, and (ii) a disposable portion removably secured to the multiple-use portion and housing the rotation mechanism and the steering mechanism.

6. The device of claim 1, wherein the cannula is flexible and bends when inserted into the patient's bladder or ureter.

7. The apparatus according to claim 1, further comprising: an ultrasound image processor operably connected to the ultrasound probe; and an ultrasound image display configured to display an ultrasound image provided by the ultrasound probe and processed by the ultrasound image processor; and a camera image processor and a camera image display configured to display an image provided by the camera and processed by the camera image processor. 8 . The apparatus according to claim 7 , wherein the ultrasound image display and the camera image display are configured to simultaneously display the ultrasound image and the camera image on a single screen.

9. The device of claim 1, wherein ultrasound and visual aspects are integrated by inserting the ultrasound probe through a working channel formed within the cannula.

10. The apparatus of claim 1, wherein the sleeve is configured to have at least a portion having a hardness property selected from the group consisting of rigid, semi-rigid, and flexible.

11. The device of claim 1, wherein the DOV is in the range of 0 degrees to 30 degrees.

12. The device of claim 1, further comprising a cannula rotation mechanism located at a proximal portion of the handle and operably coupled to the cannula to selectively rotate the cannula and thereby rotate the camera relative to the handle about the longitudinal axis.

13. The device according to claim 1, wherein the probe rotation mechanism is a probe rotation wheel.

14. The device according to claim 1 further includes a rotation sensor, which is operably connected to the probe rotation mechanism and is configured to provide an electrical signal indicating that the ultrasound probe is rotated about the longitudinal axis. The device also includes a processing system, which processes the ultrasound image from the ultrasound probe and automatically generates a three-dimensional ultrasound image therefrom based in part on the electrical signal from the rotation sensor.

15. The device according to claim 1, wherein the probe tilt steering mechanism is a probe steering wheel.

16. A medical device comprising: an elongated shaft having a distal portion and a proximal portion extending along a longitudinal axis, wherein the shaft is shaped and dimensioned to be inserted into a working channel or sheath of an endoscope cannula configured to be inserted into a patient; an ultrasound probe located at the distal portion of the shaft and configured to protrude from the distal end of the sheath or cannula and provide ultrasound images; a housing secured to the proximal portion of the shaft; a probe rotation mechanism mounted on or in the housing and operably coupled to the shaft to rotate the shaft and thereby rotate the ultrasound probe about the longitudinal axis within a selected range of rotation angles; and a probe tilting and steering mechanism mounted on or in the housing and operably connected to the ultrasound probe so as to tilt the ultrasound probe relative to the distal end portion of the cannula within a selected tilting angle range, and a rotation plate to which the ultrasound probe is secured and which rotates about a pivot axis transverse to the longitudinal axis; and a rod within the shaft that couples the probe tilt steering mechanism to the rotation plate and that pivots the rotation plate, and thereby pivots the ultrasound probe relative to the distal portion of the cannula, in response to manual actuation of the probe tilt steering mechanism; as well as A gear member is located at a fixed position within the shaft, the gear member is connected to the ultrasound probe via a bevel gear, and the gear member is connected to the probe rotation mechanism via a belt, so that in response to manual actuation of the probe rotation mechanism, the gear member is rotated about an axis transverse to the longitudinal axis to selectively rotate the ultrasound probe about the longitudinal axis.

17. The medical device of claim 16, further comprising a rotation sensor operably coupled to the probe rotation mechanism and providing an electrical signal indicative of rotation of the ultrasound probe about the longitudinal axis.

18. The medical device of claim 16, wherein the shaft is shaped and sized to be inserted into a working channel of an endoscope having a camera at a distal end, wherein when the shaft is inserted into the working channel, the ultrasound probe protrudes distally from the camera.

Citation Information

Patent Citations

  • Flexible master - slave robotic endoscopy system

    CN104883991A

  • Endoscope system with view field deviating from center direction and endoscope

    CN210472105U

  • Ultrasonic endoscope probe

    JP2015047321A

  • Transesophageal ultrasound probe having a rotating endoscope shaft

    US20020133077A1

  • Endoscopy system with off-center direction of view

    US20190246884A1