Hybrid Robot for TEE Probe - Image Plane Control

Abstract

The following generally relates to systems and methods for transesophageal echocardiography (TEE) automation. Some aspects relate to a TEE probe having an ultrasound transducer on the distal end of the TEE probe. In some implementations, if a target is within the field of view (FOV) of the ultrasound transducer, the electronic beam steering of the probe is adjusted; if the target is at the edge of the FOV, both the electronic beam steering and the mechanical joints of the probe are adjusted; and if the target is not within the FOV, only the mechanical joints of the probe are adjusted.

Description

Technical Field

[0001] The following generally relates to systems and methods for transesophageal echocardiography (TEE).

Background Art

[0002] Transesophageal echocardiography (TEE) is an approach for cardiac ultrasound imaging that includes a flexible tubular cable having an ultrasonic transducer with an ultrasonic probe disposed at its distal tip. The TEE probe is inserted into the esophagus for placement near the heart. Existing TEE probes typically provide substantial flexibility in positioning the ultrasonic transducer and imaging plane to obtain a desired view of the heart, along with controlled insertion distance and angulation of the TEE probe, and electronic beam steering of the ultrasonic imaging plane. However, concerns include the risk of perforating the esophagus and the difficulty of manipulating the degree of control with non-intuitive visual feedback to achieve the desired clinical view.

[0003] TEE is often used as a visualization tool for performing catheter-based cardiac interventions. In such tasks, typically, standard views are obtained such that the TEE images have a common pattern well known to the operator, and thus the interventional physician controlling the catheter-based device. As the cardiac intervention progresses, the operator often desires to move between different standard views that provide different perspectives regarding the heart and the catheter. Each movement of the TEE probe to a different view takes a significant amount of time and may cause damage to the esophagus. Further, the closer the actual TEE probe position is to a standard view, the closer the US image is to the common pattern of the view that the operator expects to see.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The following discloses certain improvements.

Means for Solving the Problem

[0005] In one aspect disclosed, an ultrasonic (US) device includes a probe including a flexible tubular cable, an ultrasonic transducer on the distal end of the tube, and a mechanical joint, at least one electronic processor, and at least one memory storing computer program code. The at least one memory and the computer program code may be configured, together with the at least one electronic processor, to steer the probe to visualize a target by an iterative process, each iteration of the iterative process including obtaining an ultrasonic image using the ultrasonic transducer, adjusting only the electronic beam steering of the ultrasonic transducer if the target is within the field of view (FOV) of the ultrasonic image, adjusting both the electronic beam steering of the ultrasonic transducer and the mechanical joint of the probe if the target is at the edge of the FOV of the ultrasonic image, and adjusting at least the mechanical joint of the probe if the target is not within the FOV of the ultrasonic image.

[0006] In some embodiments, at least one electronic processor is configured to execute computer-readable instructions to cause a US device to present a target in a target view of an anatomical object based on a preset position at a distal end, the preset position being one of an upper esophageal position, an intermediate esophageal position, a transgastric position, and a deep transgastric position. In some embodiments, at least one electronic processor is configured to execute computer-readable instructions to cause the US device to store at least one reference ultrasound image created by an ultrasound transducer and to compare the ultrasound image with the at least one reference ultrasound image in an iteration to determine whether the target is within the FOV. In some embodiments, the US device further includes a database of reference images, and at least one electronic processor may be configured to execute computer-readable instructions to cause the US device to use a reference image to determine whether the target is within the FOV.

[0007] In some implementations, at least one electronic processor may be configured to execute computer-readable instructions to cause a US device to determine motion constraints based on an ultrasound image and to modify an adjustment of a mechanical joint of a probe based on the determined motion constraints in an iteration. In some embodiments, the US device further includes a force sensor, and at least one electronic processor may be configured to execute computer-readable instructions to cause the US device to use the force sensor to determine motion constraints and to modify an adjustment of a mechanical joint of the probe based on the determined motion constraints in an iteration.

[0008] In some embodiments, at least one electronic processor may be configured to execute computer-readable instructions to cause a US device to determine that a plurality of targets are present within a field of view (FOV), the plurality of targets including targets, and to adjust an electronic beam steering to balance the plurality of targets to be closer to the center of the FOV. In some embodiments, the target is a first target, and at least one electronic processor causes the US device to determine that the first target and a second target are within the FOV, determine which of the first target and the second target is assigned a higher weight, and execute computer-readable instructions to adjust the beam steering to bring the higher-weight target closer to the center of the FOV. In some embodiments, the US device further includes a user interface, and at least one electronic processor is configured to execute computer-readable instructions to cause the US device to display instructions on the user interface on how to control the movement of the distal end.

[0009] In another disclosed aspect, there is a method having the steps of determining that a target is within a field of view (FOV) of an ultrasonic transducer, adjusting an electronic beam steering of the ultrasonic transducer in response to the determination that the target is within the FOV, determining that the target is not within the FOV, and adjusting a mechanical joint of a probe in response to the determination that the target is not within the FOV.

[0010] In some embodiments, the method further includes determining that a target is at an edge of the FOV and, in response to the determination that the target is at the edge of the FOV, adjusting both the electronic beam steering and the mechanical joint of the probe.

[0011] In yet another disclosed aspect, there is an ultrasonic (US) device having an ultrasonic transducer on a distal end of a probe, at least one electronic processor, and at least one memory storing computer program code. The at least one memory and the computer program code are configured to cause the at least one electronic processor to perform operations using the at least one electronic processor to: (i) set a target view; (ii) determine whether the target view has been reached according to a field of view (FOV) of the ultrasonic transducer; and (iii) if the target view has not been reached, perform either (a) an operation of adjusting beam steering of the ultrasonic transducer, or (b) an operation of adjusting a mechanical joint of the probe.

[0012] In some embodiments, in a US device as described in the previous paragraph, the at least one electronic processor may be configured to execute computer-readable instructions to cause the US device to iterate through operations (ii) to (iii) until the target view is reached according to the FOV.

[0013] One advantage lies in the safer use of probes such as a transesophageal echocardiography (TEE) probe.

[0014] Another advantage lies in an easy-to-operate TEE probe system.

[0015] Another advantage lies in the more rapid achievement of a desired TEE probe position.

[0016] A given embodiment may or may not provide any, one, two, more, or all of the aforementioned advantages, and / or may provide other advantages, as will be apparent to those skilled in the art upon reading and understanding the present disclosure.

[0017] The present invention can take the form of various elements and element configurations, as well as various steps and step configurations. The drawings are for the purpose of illustrating preferred embodiments only and should not be construed as limiting the present invention.

Brief Description of the Drawings

[0018]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 3E

Figure 4

Modes for Carrying Out the Invention

[0019] Over the past approximately 15 to 20 years, many interventional procedures related to the heart, including aortic valve repair, mitral valve repair or replacement, patent foramen ovale closure, and atrial septal defect closure, have moved from a surgical approach to a transcatheter approach. Transfemoral access is a common technique where a small incision is made near the patient's groin and serves as an instrument portal to the femoral vein for the path to the heart. In transcatheter interventions, the clinician introduces a long flexible tool into the heart through the vasculature.

[0020] Transcatheter approaches are becoming increasingly popular because they cause less trauma to the patient and require less postoperative recovery time compared to surgery. At the same time, these are technically difficult procedures to perform due to the lack of dexterity, visualization, and tactile feedback. Some of these capabilities are restored through technologies such as transesophageal echocardiography (TEE). In particular, TEE imaging restores the visualization lost by the minimally invasive approach and, to a lesser extent, replaces tactile feedback with visual feedback of the interaction between the tool and the tissue.

[0021] Insertion illustration A of FIG. 1 shows a probe 605 (in some embodiments, a TEE probe 605) that includes a tube 606 having a distal end 610 that includes an ultrasonic transducer 612 (see FIGS. 3A, 3B, 3C, typically a phased array of side-emitting ultrasonic transducers). In a manual design, the operator may control the movement of the distal end 610 by turning the knob 170 and manually extending (i.e., pushing) the tube 606 deeper into the esophagus, pulling the tube 606 out of the esophagus (i.e., pulling), or rotating the tube while it is in the esophagus. Alternatively, a servo motor 120 (e.g., controlled by an electrical controller 130) may be used to perform these operations under computer control. Combinations of manual and servo motor control are also conceivable (e.g., servo motor control joints and manual extension / retraction). Other types of control are also conceivable. For example, magnets, shape memory alloys, hydraulics, air, etc. may be used. The operator may use the ultrasonic image generated by the ultrasonic transducer 612 at the distal end 610 as a guide when controlling the movement of the TEE probe.

[0022] However, TEE is an inconvenient substitute for natural vision in important hand-eye coordination tasks for at least the following reasons. First, ultrasound images do not provide the rich set of visual cues, including color, illumination, shadows, perspective, and texture, found in natural vision and optical camera images, and are an abstract representation of the surgical scenario. Second, ultrasound images can have noise, such as acoustic reflections, and unnatural artifacts that require cognitive effort to overcome. These first two factors combine to make the interpretation of ultrasound images cumbersome. As a result, echocardiography technicians require extensive training and experience to become proficient. Even experienced echocardiographers sometimes encounter difficult cases because each new patient presentation is a new challenge.

[0023] Third, the perspective or viewing position of TEE images arising from the esophagus behind the heart is a different perspective from that of the clinician, forcing a separate hand-eye adjustment that makes it difficult to obtain a particular desired view. Fourth, the clinician must mentally reconstruct spatial information from within the heart from multiple cross-sectional views. In other words, one ultrasound image does not contain sufficient actionable information. Although TEE can provide 3D volume rendering, its visualization (e.g., resolution) often does not enable the clinician to depict the fine details actually required.

[0024] As a result, clinicians prefer higher-resolution orthogonal 2D image slices and manipulate the plane angle in two dimensions (e.g., up / down, left / right). The task of adjusting both the TEE probe position and the image plane angle is cumbersome, cognitively demanding, and ultimately vulnerable to inefficiencies and errors, against the backdrop of difficult image interpretation and disjointed hand-eye coordination.

[0025] The techniques described herein address these and other problems. Disclosed herein is a control system (e.g., the electronic controller 130 of FIG. 1) that uses a servomotor 120 to robotically manipulate a TEE probe and automatically electronically manipulate the image plane angle, in a service for finding anatomical views and device views necessary to provide visualization in performing structural heart interventions such as steering to standard views such as a standard upper esophagus (UE) view, a standard mid-esophagus (ME) view, a standard transgastric or deep transgastric (TG) view, etc. The control system can process context information, interpret image content, and calculate viewpoint transformations in a more efficient way to make view acquisition a more deterministic and reliable process. In a variant embodiment where the TEE probe is controlled by a manual knob 170 and manual tube extension / retraction rather than a servomotor 120, the control system may provide human-perceivable instructions (e.g., "retract the probe", "advance the probe", "rotate the probe to the right", etc.) displayed on a control computer display (such as the display 140 of FIG. 1 and / or articulated using a speech synthesizer). The electronic controller 130 suitably has at least one electronic processor that reads and executes instructions (i.e., computer program code) stored in at least one memory (i.e., at least one non-transitory storage medium) to perform the disclosed TEE probe control method. The electronic processor may have, for example, a computer having one or more digital or analog ports operably communicating with the servomotor 120. For example, the communication can be digital communication via a USB port or other standard digital port, or via digital and / or analog control signals generated by a dedicated control I / O card installed in the computer.In some embodiments, the electronic controller 130 may be integrated with an ultrasonic imaging controller that controls ultrasonic imaging performed by the TEE probe 605, in which case the display 140 also presents ultrasonic images acquired by the TEE probe 605.

[0026] As a non-limiting example, FIG. 2 shows a TEE view for providing visualization during a transseptal puncture procedure. The long axis 510 and the short axis 520 delineate the contours of orthogonal image planes that include the needle and the target puncture site within the septum. These views can be difficult to find by operating the TEE probe, and furthermore, a clinician may need to move the probe to different views and then attempt to restore these views.

[0027] Managing a large number of degrees of freedom is known to be cognitively difficult, especially given the challenges of image / space interpretation. Referring to FIG. 3A, the ultrasonic transducer 612 is typically a phased array of ultrasonic transducers that generates an electronically steerable image plane 601, where the left drawing in FIG. 3A shows an example of electronic beam steering. The right drawing in FIG. 3A shows additional degrees of freedom achievable by controlling the joints of the TEE probe 605 using the servo motor 120 or (in a manual embodiment of the variant) the knob 170. The architecture of the joint control mechanism can be varied, for example, as a non-limiting example using control wires that are connected to the joints and pass through the lumen of the tube 606 to connect to the servo motor 120 or the knob 170.

[0028] FIG. 3B further shows the degrees of freedom of the distal end 610 of the probe 605 (including the ultrasonic transducer). For example, the distal end 610 can be moved along the axial direction of the probe as in directions 1(a) and 1(b), in a circular direction as in directions 2(a) and 2(b), or in a pivoting direction as in directions 3(a), 3(b), 4(a) and 4(b). In some embodiments, the movement in directions 1(a), 1(b), 2(a), 2(b), 3(a), 3(b), 4(a) and 4(b) is achieved by a mechanical joint. In some implementations, the mechanical joint includes servo motor-driven extension / retraction of the tube 606. Note that FIGS. 3A and 3B show exemplary examples, and it should be noted that more or fewer and / or different degrees of freedom can be provided in a particular TEE probe design.

[0029] FIGS. 3C, 3D, and 3E show embodiments in which the distal end 610 further includes a camera sensor 650 having a camera FOV 655. In this variant embodiment, the lateral emission ultrasonic transducer 612 is augmented by the forward viewing optical camera 650. (In another variant embodiment, the camera sensor 650 may be a second forward emission ultrasonic transducer array). Throughout the present disclosure, it should be understood that unless otherwise specified, the term "FOV" can refer to either an ultrasonic FOV or a camera FOV. FIG. 3D shows a front view of an exemplary distal end. FIG. 3E shows a side view of an exemplary distal end.

[0030] FIG. 1 shows an example of a probe 605 inserted into the esophagus 615 and shows an example of an ultrasonic FOV 620. For 2D ultrasonic imaging, the FOV 620 is typically in a two-dimensional wedge shape. FIG. 1 further shows exemplary preset distal end positions that may correspond to standard TEE views such as the upper esophageal position 625, the middle esophageal position 630, the transgastric position 635, and the deep transgastric position 640.

[0031] Generally, in some embodiments, there is a control system that robotically maneuvers a probe and / or distal end to find anatomical views and device views necessary to provide visualization for performing a structural heart intervention and automatically maneuvers the image plane angle. The control system can incorporate context information, interpret the image content, and calculate the viewpoint transformation in an efficient manner, making the viewpoint acquisition a more deterministic and reliable process compared to unassisted manual operation of a TEE probe.

[0032] In other words, some embodiments include improvements where the manual control of the TEE probe 605 is enhanced or replaced by robotic control. Some implementations use a set of rules for iterative robotic control that, for each iteration, adjusts only the electronic beam steering when the target is within the field of view (FOV), adjusts both the beam steering and the mechanical joint when the target is at the edge of the FOV, biases towards the electronic beam steering, and adjusts only the mechanical joint when the target is not within the FOV. To recognize the target, a database of reference ultrasound images in standard views may be used, or a model of a standard ultrasound view may be used (e.g., a view where all four chambers of the heart are visible can be modeled using the expected four-compartment image format). In another approach, when a clinician is in a particular view, this image may be stored as a reference image, and if the clinician later desires to return to that view, the stored reference image is retrieved.

[0033] In addition, some embodiments use an intracardiac echo (ICE) probe (not the TEE probe described above), which is a thin probe inserted into the heart. The approaches described herein can properly position these probes with respect to, for example, the right atrium and the left atrium. In other embodiments, an intravascular ultrasound (IVUS) probe is used, which is a thin probe used within a blood vessel. Further, some implementations include endobronchial US, and some implementations include transrectal US and transurethral US for urology. It should be understood that the techniques described herein generally include any in vivo ultrasound imaging.

[0034] In one embodiment, the hybrid TEE probe-image plane controller follows the strategy depicted in the example of FIG. 4. The controller is triggered by setting a desired target view at operation 710, which may be directly specified as a set of robot joint positions and plane angles, or visually specified as a desired target image and then internally converted to the corresponding joints and angles. For example, if a particular view is stored as a reference image with a set of robot joint positions and plane angles, and the clinician later desires to return to that view, the stored reference image is converted to the corresponding joints and angles by retrieving the stored corresponding joints and angles. In some embodiments, the operator may set the target position to one of the preset target positions shown in FIG. 1 (upper esophageal position 625, mid-esophageal position 630, transgastric position 635, and deep transgastric position 640). For example, the expected joint and angle positions may be tabulated for standard views within a data structure (e.g., a table) supplied with the TEE probe. Preferably, the expected joints and angles are tabulated for different patient-specific parameters such as patient size. When the target is reached, at operation 720, the procedure ends at operation 730. If the target has not been reached, the hybrid TEE controller performs a function at operation 740. Specifically, the robot joints and image plane angles must then be moved from their initial configuration to converge towards the target view. The hybrid TEE controller may move both the probe (operation 750) and the image plane (operation 760) in the required directions and may repeatedly check whether the target view has been reached (operation 720). During each iterative check 720, the controller determines how to set the next movement increment based on various factors. This process repeats until the target view is reached. If the view is not achievable, the controller stops, indicates an error, and, if possible, provides a proposal to reset the probe position to a known state.

[0035] When performing operation 740, a comparison is preferably made between the currently acquired ultrasonic image and a reference ultrasonic image in a standard view (or alternatively, a model of a standard ultrasonic view). This is because, as discussed earlier, the target is specified in operation 710 with respect to joints and angles, but in practice, it is expected that those joint and imaging plane angular positions may not accurately provide the desired ultrasonic imaging view. For example, when using a table of expected joint and angular positions supplied with a TEE probe, the anatomical structure of a particular patient is likely to be sufficiently different such that simply setting the table's joint and angular positions does not provide an accurate view. When using the joint position and angular position stored with a previously stored image as a target, hysteresis and / or TEE probe drift, etc., may again not achieve exactly the same view when returning to the stored joint position and angular position.

[0036] To address this, the controller employs image-based adjustments as the target is approached. Thus, after each iteration, an ultrasound image is acquired and it is determined whether the target is within the FOV of the ultrasound image (based on a comparison of the currently acquired ultrasound image with a reference ultrasound image or a model of a standard ultrasound view). If not (as is at least the case in the initial iterations), the hybrid TEE controller moves only the probe in the direction of movement towards the specified joint and angular position relative to the target. Thus, after each iteration, the ultrasound image is compared with a reference ultrasound image in a standard view (or, alternatively, a model of a standard ultrasound view). Unless the target is within the ultrasound FOV, the iteration performs only operation 750, i.e., moves the probe without adjusting the beam angle. When the target hardly enters the FOV, the hybrid TEE controller may perform both (i) moving the probe and / or the distal end of the probe for each operation 750, and (ii) manipulating the image plane for each operation 760. After one or more iterations of this hybrid position / image plane angle adjustment, the target is likely to approach the center of the FOV. At this point, only the beam angle is adjusted (operation 760).

[0037] In summary, regarding the hybrid TEE controller in the example of FIG. 4, in operation 740, the hybrid TEE controller may consider whether the target is within the field of view (FOV). If so, the hybrid TEE controller may first manipulate the image plane (e.g., by beam steering the ultrasonic transducer at the distal end of the probe). If not, the hybrid TEE controller may move the probe or the distal end of the probe. If the target is barely within the FOV, the hybrid TEE controller may perform both (i) moving the probe and / or the distal end of the probe and (ii) manipulating the image plane. Further, in operation 740, the hybrid TEE controller may consider whether the planar angle is achievable and whether there are motion constraints (e.g., determined by a force sensor, as described in more detail below).

[0038] Not all embodiments are completely automatic. For example, in some embodiments, the angles of the image plane and / or the distal end position may be calculated and presented to the user as a recommendation, and then implemented by the user via knob 170 and / or by extending or retracting tube 606.

[0039] The following describes exemplary criteria for calculating the next movement increment. A clinician preferably sets the view such that an imaging target of interest, such as an anatomical feature, an intervention device, or some combination thereof, is approximately centered within the 3D ultrasound image, possibly from a particular viewpoint. This configuration makes the view robust to slight displacements of the TEE probe due to inadvertent probe movement, patient movement, or physiological movement. Instead, if the imaging target is at the edge of the volume, the imaging target may easily leave the view, thereby causing the clinician to lose the desired visualization. Thus, one criterion for determining the next movement increment is whether the imaging target is close to the center of the field of view (e.g., for a distinct target, by determining whether the center or edge of the target is a predetermined length from the center of the FOV, for the target viewpoint, the movement criterion may be the similarity between the current viewpoint and the desired viewpoint, which may be calculated by aligning the two views and determining the distance between them, e.g., by matching a zero distance). If so, the probe head is maintained while the imaging plane is adjusted (e.g., by beam steering the ultrasound transducer at the distal end of the probe). Otherwise, the distal end is moved so that the imaging target can be as close as possible to the center of the FOV. This strategy embeds a preference for minimizing probe movement since the available range of movement within the esophagus is limited. If both the probe and the image plane must be moved in an iteration, they can be moved sequentially or simultaneously.

[0040] Furthermore, note that "centering" is an example of a desired view of the target. For example, the target may be suitable at a certain offset from the center of the image, but the concepts and procedures are the same as those described above.

[0041] There may be situations where the imaging target cannot be precisely centered in the ultrasound volume due to the physical limitations of the probe. In such cases, the target is placed as centrally as possible, and planar angular steering is preferred. The physical limitations may include inherent constraints such as joint limitations, external constraints imposed by anatomical structures, or a combination of the two. These anatomical limitations can be (i) sensed by separate force sensors, (ii) inferred from tissue compression observed in the ultrasound image, and / or (iii) measured through a distorted image. In other words, optionally, the TEE probe can incorporate one or more force sensors to detect when the probe is colliding with the inner surface of the esophagus. Other disclosed approaches detect such collision events from tissue compression observed in the ultrasound image or measured through a distorted image. Advantageously, these constraints significantly enhance the safe use of the TEE probe.

[0042] If multiple imaging targets are present within the desired view, the system may balance the placement of the targets to be centered within the volume (e.g., by beam steering). Alternatively, the targets may be weighted such that those with a higher weight are directed more towards the center of the imaging volume than those with a lower weight.

[0043] Some embodiments assume that the planar angle corresponds to the X plane that is orthogonal. Other embodiments are equally applicable to planes at any angle.

[0044] Some embodiments use a similarity metric between a desired view and an achievable view. Specifically, in practice, views may be difficult to accurately reproduce due to changing anatomical states and imperfect robot kinematics. To handle these conditions, image-based matching between the current view and the desired view can be used to calculate the similarity metric, and the controller can stop once the similarity reaches a threshold. Alternatively, the similarity metric can use the probe configuration and / or planar angle as inputs, or in yet other embodiments, the probe position can be tracked externally relative to the anatomical structure, and the similarity between the achievable view and the desired target view can be measured accordingly.

[0045] Some embodiments use data-driven control. Specifically, as an alternative to analytical and explicit control of the probe and plane, these parameters can be determined empirically based on a database of stored views.

[0046] Some embodiments also include a user interface. Specifically, the applications of the present invention can be used to explicitly instruct the user to perform movements, or the system can automatically perform the required actions. Semi-automation in finding views is similarly applicable.

[0047] Some embodiments use intraoperative updates of views. Specifically, to address the problem of changing anatomical or intervention states that prevent an exact replication of the view, the desired view can be updated using the closest achievable view.

[0048] Furthermore, it should be understood that the techniques disclosed herein may be implemented by a non-transitory storage medium (i.e., at least one memory) that stores instructions readable and executable by an electronic data processing apparatus (e.g., the controller 130 of FIG. 1) for performing the disclosed techniques. Such non-transitory storage media may include hard drives or other magnetic storage media, optical disks or other optical storage media, cloud-based storage media such as RAID disk arrays, flash memory or other non-volatile electronic storage media.

[0049] The present invention has been described with reference to preferred embodiments. Modifications and variations may occur to others upon reading and understanding the foregoing detailed description. It is intended that all such modifications and variations be included insofar as they fall within the scope of the appended claims or the equivalents thereof.

Claims

1. a probe including a tube, an ultrasonic transducer at a distal end of the tube, and a mechanical joint; at least one electronic processor; at least one memory storing computer-readable instructions; in an ultrasonic (US) device having the same, the at least one memory and the computer-readable instructions are configured to cause the at least one electronic processor to maneuver the probe to a target by an iterative process, each iteration including: acquiring an ultrasonic image using the ultrasonic transducer; at least one of (i) adjusting an electronic beam steering of the ultrasonic transducer according to the ultrasonic image and (ii) adjusting the mechanical joint of the probe according to the ultrasonic image; including; the at least one electronic processor executes the computer-readable instructions to cause the US device, in the iteration, when the target is within a field of view (FOV) of the ultrasonic image, only adjust the electronic beam steering of the ultrasonic transducer; when the target is at an edge of the FOV of the ultrasonic image, adjust both the electronic beam steering of the ultrasonic transducer and the mechanical joint of the probe; when the target is not within the FOV of the ultrasonic image, adjust at least the mechanical joint of the probe; configured as such; US device.

2. the at least one electronic processor executes the computer-readable instructions to cause the US device to create a target view of an anatomical object based on a preset position of the distal end, and the preset position is an upper esophageal position; a middle esophageal position; a transgastric position; a deep transgastric position; one of them, the US device according to Claim 1.

3. the at least one electronic processor executes the computer-readable instructions to cause the US device to store at least one reference ultrasonic image created by the ultrasonic transducer; in the iteration, compare the ultrasonic image with the at least one reference ultrasonic image to determine whether the target is within the FOV; configured as such, the US device according to any one of Claims 1 to 2.

4. further having a database of reference images The at least one electronic processor is configured to execute the computer-readable instructions to cause the US device to use the reference image to determine whether the target is within the FOV, according to any one of claims 1 to 3.

5. The at least one electronic processor is configured to execute the computer-readable instructions to cause the US device to determine motion constraints based on the ultrasonic image, and during the iteration, modify the adjustment of the mechanical joint of the probe based on the determined motion constraints. The US device according to any one of claims 1 to 4, configured as such.

6. Further comprising a force sensor, The at least one electronic processor is configured to execute the computer-readable instructions to cause the US device to use the force sensor to determine motion constraints, and during the iteration, modify the adjustment of the mechanical joint of the probe based on the determined motion constraints. The US device according to any one of claims 1 to 5, configured as such.

7. The at least one electronic processor is configured to execute the computer-readable instructions to cause the US device to determine that there are a plurality of targets including the target within the FOV, and adjust the electronic beam steering to balance the plurality of targets to be closer to the center of the FOV. The US device according to any one of claims 1 to 6, configured as such.

8. The target is a first target, The at least one electronic processor is configured to execute the computer-readable instructions to cause the US device to determine that the first target and a second target are within the FOV, determine which of the first and second targets is assigned a higher weight, and adjust the beam steering to bring the target with the higher weight closer to the center of the FOV. The US device according to any one of claims 1 to 7, configured as such.

9. Further comprising a user interface, and the at least one electronic processor is configured to execute the computer-readable instructions to cause the US device to display instructions regarding a method of controlling the movement of the distal end on the user interface. The US device according to any one of claims 1 to 8, configured as such.

10. The at least one memory and the computer-readable instructions are configured to cause the at least one electronic processor to, in the iteration, cause the probe to when the target is not within the FOV of the ultrasonic image, adjust only the mechanical joint of the probe so as to be maneuvered to the target, the US device according to any one of claims 1 to 9. **Claim 11** In a method of operating an ultrasonic device to cause a probe to be maneuvered to a target by an iterative process, in each iteration, a step of an at least one electronic processor of the ultrasonic device obtaining an ultrasonic image using an ultrasonic transducer of the ultrasonic device; (i) a step of the at least one electronic processor adjusting electronic beam steering of the ultrasonic transducer according to the ultrasonic image, and (ii) at least one of the at least one electronic processor adjusting a mechanical joint of the probe according to the ultrasonic image including in the iteration, when the target is within the field of view (FOV) of the ultrasonic image, a step of the at least one electronic processor adjusting only the electronic beam steering of the ultrasonic transducer; when the target is at the edge of the FOV of the ultrasonic image, a step of the at least one electronic processor adjusting both the electronic beam steering of the ultrasonic transducer and the mechanical joint of the probe; when the target is not within the FOV of the ultrasonic image, a step of the at least one electronic processor adjusting at least the mechanical joint of the probe including, the method.

Images