Hybrid Robotic Image Plane Control for TEE Probes

Abstract

The following generally relates to systems and methods for automating transesophageal echocardiography (TEE). Some aspects relate to a TEE probe having an ultrasound transducer at its distal end. In some embodiments, if the target is within the field of view (FOV) of the ultrasound transducer, the electron beam directional adjustment of the probe is adjusted; if the target is at the edge of the FOV, both the electron beam directional adjustment and the mechanical joints of the probe are adjusted; and if the target is not within the FOV, only the mechanical joints of the probe are adjusted.

Description

Technical Field

[0001] The following text generally relates to the systems and methods of transesophageal echocardiography (TEE). Background Technology

[0002] WO 2019 / 175129 describes an automated medical examination method, which includes: receiving a first image representing the body of a subject from an imaging device, while the imaging device is positioned at a first imaging location relative to the body of the subject.

[0003] US 2018 / 0132724 A1 describes an imaging steering device that includes a sensor and an imaging processor, the device being configured to: acquire images of an object of interest via multiple sensors in the sensor and from the current location and current orientation.

[0004] Transesophageal echocardiography (TEE) is a method for cardiac ultrasound imaging in which an ultrasound (US) probe comprises a flexible tubular cable, with an ultrasound transducer positioned at the distal tip of the flexible tubular cable. The TEE probe is inserted into the esophagus to position it close to the heart. Existing TEE probes typically include mechanical joints, which, along with controlled insertion distance and angulation of the TEE probe and electron beam steering of the ultrasound imaging plane, provide considerable flexibility in positioning the ultrasound transducer and imaging plane to acquire the desired cardiac view. However, concerns include the risk of esophageal perforation and the difficulty of achieving the desired clinical view with a high degree of manipulative control without intuitive visual feedback.

[0005] TEE (Transcatheter Excision) is often used as a visualization tool for performing catheter-based cardiac interventions. In such tasks, a standard view is typically obtained, giving the TEE images a general pattern familiar to the operator, and thus to the interventional cardiologist controlling the catheter-based device. As the cardiac intervention progresses, the operator often wants to move between different standard views providing different perspectives on the heart and catheter. Each movement of the TEE probe towards a different view takes considerable time and can potentially cause esophageal injury. Furthermore, the closer the actual TEE probe position is to the standard view, the closer the US image will be to the general pattern of the view the operator expects to see.

[0006] Some improvements are disclosed below. Summary of the Invention

[0007] In one aspect of the disclosure, an ultrasound (US) device includes: a probe comprising a flexible tubular cable, an ultrasound transducer at the distal end of the cable, and a mechanical joint; at least one electronic processor; and at least one memory storing computer program code. The at least one memory and the computer program code can be configured to use the at least one electronic processor to orient the probe through an iterative process for visualizing a target, wherein each iteration includes: acquiring an ultrasound image using the ultrasound transducer; if the target is within the field of view (FOV) of the ultrasound image, adjusting only the electron beam orientation of the ultrasound transducer; if the target is at the edge of the FOV of the ultrasound image, adjusting both the electron beam orientation of the ultrasound transducer and the mechanical joint of the probe; and if the target is not within the FOV of the ultrasound image, adjusting at least the mechanical joint of the probe.

[0008] In some embodiments, the at least one electronic processor is configured to execute the computer-readable instructions to cause the US device to create a target view of the target as an anatomical object based on a preset location at the distal end, and the preset location is one of the following: an upper esophageal location; a middle esophageal location; a transgastric location; and a deep gastric location. In some embodiments, the at least one electronic processor is configured to execute the computer-readable instructions to cause the US device to perform the following operations: store at least one reference ultrasound image created by the ultrasound transducer; and compare the ultrasound image with the at least one reference ultrasound image in the iteration to determine whether the target is in the FOV. In some embodiments, the US device further includes a database of reference images; and the at least one electronic processor may be configured to execute the computer-readable instructions to cause the US device to use the reference images to determine whether the target is in the FOV.

[0009] In some embodiments, the at least one electronic processor may be configured to execute the computer-readable instructions to cause the US device to perform the following operations: determine motion constraints based on the ultrasound images; and modify the adjustment of the mechanical joint of the probe based on the determined motion constraints in the iterations. In some embodiments, the US device further includes a force sensor, and the at least one electronic processor may be configured to execute the computer-readable instructions to cause the US device to perform the following operations: determine motion constraints using the force sensor; and modify the adjustment of the mechanical joint of the probe based on the determined motion constraints in the iterations.

[0010] In some embodiments, the at least one electronic processor may be configured to execute the computer-readable instructions to cause the US device to perform the following operations: determine that a plurality of targets are present in the FOV, wherein the plurality of targets includes the target; and adjust the electron beam steering to balance the plurality of targets closer to the center of the FOV. In some embodiments, the target is a first target; and the at least one electronic processor may be configured to execute the computer-readable instructions to cause the US device to perform the following operations: determine that the first target and a second target are in the FOV; determine which of the first target and the second target has been assigned a higher weight; and adjust the beam steering to bring the target with the higher weight closer to the center of the FOV. In some embodiments, the US device further includes a user interface, and the at least one electronic processor may be configured to execute the computer-readable instructions to cause the US device to perform the following operations: display instructions on the user interface regarding how to control the movement of the distal end.

[0011] In another disclosed aspect, there is a method comprising: determining that a target is in the field of view (FOV) of an ultrasonic transducer; adjusting the electron beam steering of the ultrasonic transducer in response to the determination that the target is in the FOV; determining that the target is not in the FOV; and adjusting a mechanical joint of a probe in response to the determination that the target is not in the FOV.

[0012] In some embodiments, the method further includes: determining that the target is at the edge of the FOV; and adjusting both the electron beam deflection and the mechanical joint of the probe in response to the determination that the target is at the edge of the FOV.

[0013] In another disclosed aspect, an ultrasound (US) device includes: an ultrasound transducer located at the distal end of a probe; at least one electronic processor; and at least one memory storing computer program code. The at least one memory and the computer program code are configured to use the at least one electronic processor to cause the US device to perform the following operations: (i) setting a target view; (ii) determining, based on the field of view (FOV) of the ultrasound transducer, whether the target view has been reached; and (iii) if the target view has not been reached, performing either: (a) adjusting the beam steering of the ultrasound transducer, or (b) adjusting a mechanical joint of the probe.

[0014] In some embodiments, in a US device as described in the preceding paragraphs, the at least one electronic processor may be configured to run the computer-readable instructions to cause the US device to iterate through operations (ii)-(iii) until the target view has been reached according to the FOV.

[0015] One advantage is the safer use of probes, such as transesophageal echocardiography (TEE) probes.

[0016] Another advantage is that the TEE probe system is easier to operate.

[0017] Another advantage is that it allows for faster access to the desired TEE probe position.

[0018] The given embodiments may provide zero, one, two, more or all of the aforementioned advantages, and / or may provide other advantages, which will be readily understood by those skilled in the art after reading and understanding this disclosure. Attached Figure Description

[0019] This invention can take various forms, including various components and component arrangements, as well as various steps and step arrangements. The accompanying drawings are for illustrative purposes only and should not be construed as limiting the invention.

[0020] Figure 1 An example of a probe inserted into the esophagus is illustrated, along with an example of a preset distal position.

[0021] Figure 2 The diagram illustrates a TEE view for the atrial septal puncture procedure.

[0022] Figure 3A A depiction of a basic hybrid control system is shown, including supervision of the 2D ultrasound imaging plane and robotic control of the TEE probe itself.

[0023] Figure 3B The diagram illustrates the degrees of freedom of the TEE probe.

[0024] Figure 3C The illustration shows an example of both ultrasonic FOV and camera FOV at the far end.

[0025] Figure 3D The illustration shows a front view of the far end of the example.

[0026] Figure 3E The illustration shows a side view of the far end of the example.

[0027] Figure 4 This illustration illustrates an example embodiment of a hybrid control strategy for coordinating the motion of a robot's TEE probe and the angle of the image plane. Detailed Implementation

[0028] Over the past 15-20 years, many interventional procedures in the heart, including aortic valve repair, mitral valve repair or replacement, patent foramen ovale closure, and atrial septal defect closure, have migrated from surgical methods to transcatheter methods. Transfemoral access is a common technique in which a small incision is made near the patient's groin to act as a gateway for instruments into the femoral vein and through to the heart. In transcatheter interventions, clinicians introduce long, flexible instruments into the heart through the vascular system.

[0029] Transcatheter methods have become increasingly popular due to their less invasive nature and shorter recovery time compared to surgery. However, they are technically challenging procedures due to the lack of dexterity, visualization, and tactile feedback. Some of these functionalities have been restored through techniques such as transesophageal echocardiography (TEE). In particular, TEE imaging restores the visualization lost through minimally invasive methods and, to a lesser extent, replaces tactile feedback with visual feedback from tool-to-tissue interaction.

[0030] Figure 1 Illustration A shows a probe 605 (in some embodiments, a TEE probe 605), which includes a tube 606 having a distal end 610, the distal end 610 including an ultrasonic transducer 612 (see Figure 1). Figure 3A , Figure 3B , Figure 3C (Typically, a phased array of side-emitting ultrasonic transducers). In a manual design, the operator can control the movement of the distal end 610 by turning knob 170 and manually extending (i.e., pushing) the tube 606 deeper into the esophagus, retracting (i.e., pulling) the tube 606 out of the esophagus, or rotating the tube while it is in the esophagus. Alternatively, these operations can be performed under computer control using a servo motor 120 (e.g., controlled by an electric controller 130). Combinations of manual control with servo motor control are also contemplated (e.g., servo motor-controlled joints and manual extension / retraction). Other types of control are also contemplated. For example, magnets, shape memory alloys, hydraulics, air, etc., may also be used. The operator can use the ultrasonic images generated by the ultrasonic transducer 612 at the distal end 610 as a guide for controlling the movement of the TEE probe.

[0031] However, TEE is a clumsy substitute for natural vision in critical hand-eye coordination tasks for at least the following reasons. First, ultrasound images do not provide a rich set of visual cues, including colors, lighting, shadows, perspective, and textures found in natural vision and optical camera images; these are abstract representations of surgical scenes. Second, ultrasound images are capable of containing noise and unnatural artifacts (e.g., acoustic reflections) that require cognitive effort to overcome. The first two factors combined make the interpretation of ultrasound images cumbersome. As a result, echocardiography requires extensive training and experience to become proficient. Even expert echocardiologists occasionally encounter difficult cases, as each new patient presents a new challenge.

[0032] Third, the vantage point or viewpoint of TEE images originating from the esophagus behind the heart differs from the clinician's viewpoint, thus forcing a disconnect in hand-eye coordination and making it difficult to acquire the specific desired view. Fourth, clinicians must reconstruct spatial information from within the heart based on multiple cross-sectional views. In other words, a single ultrasound image does not contain sufficient information for action. While TEE can provide 3D volumetric mapping, its visualization (e.g., resolution) often does not allow clinicians to depict the fine details required for practice.

[0033] Therefore, clinicians prefer higher resolution, orthogonal 2D image slices, and planar angles in two dimensions (e.g., top / bottom, left / right). Adjusting both the TEE probe position and the image planar angle is a demanding, cognitively required task, and ultimately susceptible to inefficiency and error, within the context of challenging image interpretation and disjointed hand-eye coordination.

[0034] The techniques described herein solve these and other problems. This document discloses a control system (e.g., Figure 1 The electronic controller 130 can robotically steer the TEE probe using servo electrodes 120 and automatically and electronically steer the image plane to provide the anatomical and device views required to support the performance of structural cardiac interventions, such as steer to standard views (e.g., standard upper esophagus (UE) view, standard middle esophagus (ME) view, standard transgastric or deep gastric (TG) view, etc.). The control system is capable of processing background information more effectively, interpreting image content, and calculating viewpoint shifts, making view acquisition a more deterministic and reliable process. In a variant embodiment (where the TEE probe is controlled by a manual knob 170 and manual tube extension / retraction instead of servo motor 120), the control system can provide control of a computer display (e.g., ...) Figure 1The electronic controller 130 displays and / or clearly expresses human-perceptible directions (e.g., “retract probe,” “advance probe,” “turn probe to the right,” etc.) on the display 140 to assist in manual control of the TEE probe. The electronic controller 130 suitably includes at least one electronic processor that reads and executes instructions (i.e., computer program code) stored in at least one memory (i.e., at least one non-transient storage medium) to perform the disclosed TEE probe control method. The electronic processor may, for example, include a computer with one or more digital or analog ports for operative communication with one or more servo motors 120. For example, communication can be digital communication via a USB port or other standard digital port, or via digital and / or analog control signals generated by a dedicated control I / O card installed in the computer. In some embodiments, the electronic controller 130 may be integrated with an ultrasound imaging controller that controls ultrasound imaging performed by the TEE probe 605, in which case the display 140 also presents ultrasound images acquired by the TEE probe 605.

[0035] By using unrestricted examples Figure 2 The illustration shows TEE views used to provide visualization during atrial septal puncture procedures. The major axis 510 and minor axis 520 outline the contours of orthogonal image planes containing the needle and the target puncture site within the septum. These views can be difficult to locate by manipulating the TEE probe; furthermore, clinicians may need to move the probe to different views and then attempt to reconstruct them.

[0036] Many degrees of freedom are cognitively difficult to manage effectively, especially given the challenges of image / spatial interpretation. (Reference) Figure 3A The ultrasonic transducer 612 is typically a phased array of ultrasonic transducers that generate an electronically deflectable image plane 601, wherein... Figure 3A The left-hand side drawing illustrates an example of electron beam deflection. Figure 3A The right-hand side drawing illustrates the additional degrees of freedom obtainable by controlling the joint of the TEE probe 605 using the servo motor 120 or (in a variant manual embodiment) the knob 170. The architecture of the joint control mechanism can vary, for example, as a non-limiting example employing control lines connected to the joint and passing through the cavity of the tube 606 to connect to the servo motor 120 or the knob 170.

[0037] Figure 3BThe degrees of freedom of the distal end 610 of the probe 605 (which includes an ultrasonic transducer) are also illustrated. For example, the distal end 610 can move: along the axial direction of the probe, such as in directions 1(a) and 1(b); in a circular direction, such as in directions 2(a) and 2(b); or in a pivoting direction, such as in directions 3(a), 3(b), 4(a) and 4(b). In some embodiments, the movement in directions 1(a), 1(b), 2(a), 2(b), 3(a), 3(b), 4(a) and 4(b) is accomplished by a mechanical joint. In some embodiments, the mechanical joint includes the extension / retraction of a servo motor-driven tube 606. It should be noted that... Figure 3A and Figure 3B Illustrative examples are shown, and more or fewer and / or different degrees of freedom can be provided in a particular TEE probe design.

[0038] Figure 3C , Figure 3D and Figure 3E An embodiment is illustrated in which the distal end 610 further includes a camera sensor 650 having a camera FOV 655. In this variant embodiment, the forward-looking optical camera 650 enhances the side-emitting ultrasonic transducer 612. (In another variant embodiment, the camera sensor 650 may be a second forward-emitting ultrasonic transducer array). It should be understood that throughout this disclosure, unless otherwise specified, the term "FOV" can refer to either ultrasonic FOV or camera FOV. Figure 3D The illustration shows a front view of the far end of the example. Figure 3E The illustration shows a side view of the far end of the example.

[0039] Figure 1 An example of a probe 605 inserted into the esophagus 615 is illustrated, as is an example of an ultrasound FOV 620. For 2D ultrasound imaging, the FOV 620 is typically two-dimensional wedge-shaped. Figure 1 The illustration also shows example preset distal positions, which can correspond to standard TEE views, such as: upper esophagus 625; middle esophagus 630; transgastric position 635; and deep gastric position 640.

[0040] Typically, in some embodiments, a control system exists to robotically steer the probe and / or distal end, and to automatically steer the image plane angle to serve the purpose of finding the anatomical and device views required to provide visualization for performing structural cardiac interventions. The control system is capable of efficiently processing background information, interpreting image content, and calculating viewpoint shifts, thereby making view acquisition a more deterministic and reliable process compared to unassisted manual manipulation of the TEE probe.

[0041] In other words, some embodiments involve improvements where manual control of the TEE probe 605 is enhanced or replaced by robotic control. Some implementations use a set of rules for iterative robotic control, which, for each iteration, include: adjusting only the electron beam steering if the target is in the field of view (FOV); adjusting both the electron beam steering and the mechanical joints, biased towards the electron beam steering, if the target is at the edge of the FOV; and adjusting only the mechanical joints if the target is not in the FOV. To identify the target, a database of reference ultrasound images under standard views can be used, or a model of a standard ultrasound view can be used (e.g., a view capable of modeling a view where all four chambers of the heart are visible using a desired four-compartment image format). In another approach, if a clinician is in a particular view, that image can be stored as a reference image, and the stored reference image can be retrieved if the clinician later wishes to return to that view.

[0042] Additionally, some embodiments use intracardiac echo (ICE) probes (instead of the TEE probes described above), which are thin probes inserted into the heart. The methods described herein allow for proper positioning of these probes with respect to the right atrium, left atrium, etc. In other embodiments, intravascular ultrasound (IVUS) probes are used, which are thin probes used in blood vessels. Furthermore, some embodiments include endobronchial ultrasound (IVUS), and some embodiments include transrectal and transurethral ultrasound for urology. It should be understood that the techniques described herein generally encompass any form of in vivo ultrasound imaging.

[0043] In one embodiment, the hybrid TEE probe-image plane controller follows Figure 4 The strategy is illustrated in the example. The controller is triggered by setting a desired target view at operation 710. This desired target view can be directly specified as a set of robot joint positions and planar angles, or it can be visually specified as a desired target image, which is then internally converted to the corresponding joints and angles. For example, if a particular view is stored as a reference image along with a set of robot joint positions and planar angles, and if the clinician later wishes to return to that view, the stored reference image is converted to the corresponding joints and angles by retrieving the stored corresponding joints and angles. In some embodiments, the operator can set the target position to... Figure 1One of the preset target positions shown is (upper esophageal position 625; middle esophageal position 630; transgastric position 635; and deep gastric position 640). For example, the expected joint and angle positions can be listed for a standard view in a data structure (e.g., a table) supplied using the TEE probe. Preferably, the expected joints and angles are listed for different patient-specific parameters (e.g., patient body size). If the target has been reached at operation 720, the process ends at operation 730. If the target has not been reached, the hybrid TEE controller performs a function at operation 740. Specifically, the robot joints and image plane angles must be moved from their initial configuration to converge to the target view. The hybrid TEE controller can move both the probe (operation 750) and the image plane (operation 760) in the necessary directions and repeatedly check whether the target view has been reached (operation 720). During each iteration check 720, the controller decides how to set the next motion increment based on various factors. This process is repeated until the target view is reached. If the view cannot be reached, the controller aborts and indicates an error, and, if possible, provides suggestions for resetting the probe position to a known state.

[0044] During operation 740, it is preferable to perform a comparison between the currently acquired ultrasound image and a reference ultrasound image at the standard view (or, alternatively, a model of the standard ultrasound view). This is because, although targets were specified at operation 710 regarding the joints and angles discussed earlier, the actual expected joint and imaging plane angular positions may not accurately provide the desired ultrasound imaging view. For example, if a table of expected joint and angular positions supplied using a TEE probe is used, the anatomy of a particular patient may be different enough that simply setting the table of joint and angular positions will not accurately provide the correct view. When using joint and angular positions stored along with previously stored images as targets, hysteresis and / or TEE probe drift, etc., may again result in the inability to accurately achieve the same view when returning to the stored joint and angular positions.

[0045] To address this issue, the controller employs image-based adjustments as it approaches the target. Therefore, after each iteration, an ultrasound image is acquired and (based on a comparison of the currently acquired ultrasound image with a reference ultrasound image or a model of a standard ultrasound view) it is determined whether the target is within the FOV of the ultrasound image. If not (which is likely at least in the initial iteration), the hybrid TEE controller moves the probe only in the direction specified for the joint and angular position relative to the target. Thus, after each iteration, the ultrasound image is compared with a reference ultrasound image in a standard view (or, alternatively, with a model of a standard ultrasound view). As long as the target is not within the ultrasound FOV, the iteration only performs operation 750, i.e., moving the probe without adjusting the beam angle. When the target barely enters the FOV, the hybrid TEE controller can perform two operations: (i) moving the probe and / or the distal end of the probe at each operation 750, and (ii) turning the image plane at each operation 760. After one or more iterations of this hybrid position / plane angle adjustment, the target may be closer to the center of the FOV. At this point, only the beam angle is adjusted (operation 760).

[0046] In conclusion, regarding in Figure 4 In the example of the hybrid TEE controller in operation 740, the hybrid TEE controller may consider whether the target is in the field of view (FOV). If so, the hybrid TEE controller may first (e.g., by beam steering of the ultrasonic transducer at the distal end of the probe) steer the image plane. If not, the hybrid TEE controller may move the probe or the distal end of the probe. If the target is barely in the FOV, the hybrid TEE controller may perform two operations: (i) move the probe and / or the distal end of the probe, and (ii) steer the image plane. Additionally, in operation 740, the hybrid TEE controller may consider whether a plane angle is achievable and whether there are motion constraints (e.g., motion constraints determined by force sensors, as will be explained in more detail below).

[0047] Not all embodiments are fully automatic. For example, in some embodiments, the angle and / or distal position of the image plane may be calculated and presented to the user as one or more suggestions, which may then be implemented via knob 170 and / or by extending or retracting tube 606.

[0048] The following describes an example criterion for calculating the next motion increment. Clinicians tend to position the view so that the imaging target of interest (e.g., anatomical features, interventional instruments, or some combination thereof) is roughly concentrated within the 3D ultrasound image and, in some cases, seen from a specific viewpoint. This arrangement makes the view robust to minor displacements of the TEE probe due to unintentional probe movement, patient movement, or physiological movement. If the imaging target instead resides at the edge of the volume, it may easily move out of the view, causing the clinician to lose the desired visualization. Therefore, one criterion for determining the next motion increment is whether the imaging target is near the center of the field of view (e.g., for discrete targets, by determining whether the center or edge of the target is a predetermined distance from the center of the FOV; for the target viewpoint, the motion criterion could be the similarity between the current viewpoint and the desired viewpoint, which can be calculated by registering two views to determine the distance between them and such that, for example, 0 distance is a match). If so, the probe head is kept stationary while adjusting the imaging plane (e.g., by beam steering of the ultrasound transducer at the distal end of the probe). Otherwise, the distal end is moved so that the imaging target is as close as possible to the center of the FOV. This strategy incorporates a preference for minimizing probe motion, as the available range of motion within the esophagus is limited. When the probe and image plane must be moved during iterations, they can be moved one after another or simultaneously.

[0049] It should also be noted that "centering" is an example of the desired view of the target. For example, sometimes the target is slightly off-center from the center of the image, but the concept and process are the same as described above.

[0050] In some cases, due to the physical limitations of the probe, the imaging target cannot be precisely focused within the ultrasound volume. In such situations, the target will be as focused as possible, with a preference for planar angular orientation. Physical limitations can include inherent constraints (e.g., joint constraints), external constraints (e.g., constraints imposed by anatomical structures), or a combination of both. These anatomical constraints can be: (i) sensed by a separate force sensor, (ii) inferred from tissue compression observed in ultrasound images, and / or (iii) measured via strain imaging. In other words, optionally, the TEE probe can incorporate one or more force sensors to detect when the probe impinges on the inner surface of the esophagus. Other disclosed methods detect such impingement events from tissue compression observed in ultrasound images or measure such impingement events via strain imaging. Advantageously, these constraints make the TEE probe safer.

[0051] If multiple imaging targets are present in the desired view, the system can (e.g., by beam steering) balance the constellation of targets to be focused on within the volume. Alternatively, targets can be weighted such that targets with higher weights are shown as being closer to the center of the imaging volume than targets with lower weights.

[0052] Some embodiments assume that the plane angle corresponds to an orthogonal X-plane. Other embodiments are also applicable to planes at arbitrary angles.

[0053] Some embodiments use a similarity metric between the desired view and the reachable view. Specifically, in practice, it can be difficult to accurately replicate a view due to changes in anatomical conditions and imperfect robot kinematics. To address these situations, an image-based matching between the current view and the desired view can be used to calculate a similarity metric, and the controller can abort once the similarity reaches a threshold. Alternatively, the similarity metric can use probe configuration and / or planar angles as input; or, in other embodiments, the probe position can be tracked externally with respect to the anatomical structure, and the similarity between the reachable view and the desired target view can be measured accordingly.

[0054] Some embodiments use data-driven controls. Specifically, as an alternative to explicit analytical control of the probe and plane, these parameters can be empirically determined based on a database of stored views.

[0055] Some embodiments also relate to a user interface. Specifically, the invention can be used to guide a user to explicitly perform a movement, or the system can automatically perform the required actions. Semi-automated view discovery is also applicable.

[0056] Some embodiments use in-process updates of views. Specifically, to address the problem of changes in anatomical structures or interventional conditions (which prevents accurate replication of views), the desired view can be updated using the closest available view.

[0057] It will be further appreciated that the techniques disclosed herein can be implemented by a non-transient storage medium (i.e., at least one memory) storing instructions that can be processed by an electronic data processing device (e.g., Figure 1 The controller 130 reads and runs the disclosed techniques. Such non-transient storage media may include hard disk drives or other magnetic storage media, optical disks or other optical storage media, cloud-based storage media (e.g., RAID disk arrays), flash memory or other non-volatile electronic storage media, etc.

[0058] The invention has been described with reference to preferred embodiments. Modifications and substitutions may arise in the reader’s reading and understanding of the foregoing detailed description. It is intended that the exemplary embodiments be interpreted to include all such modifications and substitutions, provided they fall within the scope of the claims and their equivalents.

Claims

1. An ultrasonic ultrasound (US) device, comprising: The probe includes a tube (606), an ultrasonic transducer (612) at the distal end (610) of the tube (606), and a mechanical joint; At least one electronic processor (130); as well as At least one memory that stores computer-readable instructions; The at least one memory and the computer-readable instructions are configured to use the at least one electronic processor (130) to orient the probe toward a target through an iterative process, wherein each iteration includes: Ultrasonic images are acquired using the ultrasonic transducer (612); and Perform at least one of the following operations: (i) adjust the electron beam direction of the ultrasonic transducer (612) according to the ultrasonic image, and (ii) adjust the mechanical joint of the probe (605) according to the ultrasonic image. The at least one electronic processor (130) is configured to execute the computer-readable instructions to cause the US device to perform the following operations during the iteration: If the target is in the field of view (FOV) (620) of the ultrasound image, then only the electron beam directional of the ultrasound transducer (612) is adjusted; If the target is at the edge of the FOV (620) of the ultrasound image, then both the electron beam deflection of the ultrasound transducer (612) and the mechanical joint of the probe (605) are adjusted; and If the target is not in the FOV (620) of the ultrasound image, then at least the mechanical joint of the probe (605) is adjusted.

2. The US device according to claim 1, wherein, The at least one electronic processor (130) is configured to execute the computer-readable instructions to cause the US device to create a target view of the target as an anatomical object based on a preset position of the remote end (610), wherein the preset position is one of the following: Upper esophagus position (625); Location of the middle esophagus (630); Location of the stomach (635); and Deep stomach location (640).

3. The US device according to claim 1 or 2, wherein, The at least one electronic processor (130) is configured to execute the computer-readable instructions to cause the US device to perform the following operations: Store at least one reference ultrasound image created by the ultrasound transducer (612); and In the iteration, the ultrasound image is compared with the at least one reference ultrasound image to determine whether the target is in the FOV (620).

4. The US device according to claim 1 or 2, further comprising a database of reference images; in, The at least one electronic processor (130) is configured to run the computer-readable instructions to enable the US device to use the reference image to determine whether the target is in the FOV (620).

5. The US device according to claim 1 or 2, wherein, The at least one electronic processor (130) is configured to execute the computer-readable instructions to cause the US device to perform the following operations: Motion constraints are determined based on the ultrasound images; and In the iteration, the adjustment of the mechanical joint of the probe (605) is modified based on the determined motion constraints.

6. The US device according to claim 1 or 2 further includes a force sensor, and in, The at least one electronic processor (130) is configured to execute the computer-readable instructions to cause the US device to perform the following operations: The force sensor is used to determine motion constraints; and In the iteration, the adjustment of the mechanical joint of the probe (605) is modified based on the determined motion constraints.

7. The US device according to claim 1 or 2, wherein, The at least one electronic processor (130) is configured to execute the computer-readable instructions to cause the US device to perform the following operations: It is determined that there are multiple targets within the FOV (620), wherein the multiple targets include the target; and The electron beam is adjusted to balance the plurality of targets closer to the center of the FOV (620).

8. The US device according to claim 1 or 2, wherein: The objective is the primary objective; and The at least one electronic processor (130) is configured to execute the computer-readable instructions to cause the US device to perform the following operations: The first and second objectives are determined to be within the FOV (620); Determine which of the first and second objectives has been assigned a higher weight; and The electron beam is adjusted to bring the higher-weighted target closer to the center of the FOV (620).

9. The US device according to claim 1 or 2, further comprising a user interface (140), wherein, The at least one electronic processor (130) is configured to execute the computer-readable instructions to cause the US device to perform the following operations: Instructions on how to control the movement of the remote device (610) are displayed on the user interface (140).

10. The US device according to claim 1 or 2, wherein, The at least one memory and the computer-readable instructions are configured to use the at least one electronic processor (130) to orient the probe toward the target during the iteration by: If the target is not in the FOV (620) of the ultrasound image, then at least the mechanical joint of the probe (605) is adjusted.

11. The US device according to claim 1 or 2, in, The at least one memory and the computer-readable instructions are configured to cause the US device to perform the following operations using the at least one electronic processor (130): (i) Set the target view (710); (ii) Determine whether the target view (720) has been reached based on the field of view (FOV) (620) of the ultrasonic transducer (612); and (iii) If the target view (740) has not yet been reached, then do either of the following: (a) adjust the beam steering (760) of the ultrasonic transducer (612), or (b) adjust the mechanical joint (750) of the probe (605). The at least one electronic processor (130) is configured to run the computer-readable instructions to cause the US device to iterate through operations (ii)-(iii) until the target view has been reached according to the FOV (620).

Images