Method and system for real-time image acquisition

Abstract

Various methods and systems for displaying real-time images are provided. In one example, a method for motion compensation in the real-time images includes acquiring a first set of imaging data from a visible sector of an imaging system, the visible sector including a region of interest displayed to a user, and acquiring a second set of imaging data at a region offset from the visible sector during the acquiring the first set of imaging data. The second set of imaging data is used to compensate for gross motion of the region of interest, resulting in at least a portion of the region of interest being displaced outside the visible sector.

Description

TECHNICAL FIELD

[0001] Embodiments of the subject matter disclosed herein relate to ultrasound imaging. BACKGROUND

[0002] Ultrasound imaging utilizes high frequency sound waves to produce images of organs, tissues, or blood flow. The sound waves are generated by an ultrasound probe or transducer and transmitted in pulses. The probe detects reflections of the sound waves at boundaries between organs, tissues, bone, etc., and the reflections are relayed to a control unit where the reflected waves are converted to two- or three-dimensional images.

[0003] Information provided by images produced by ultrasound imaging systems can be affected by conditions during image acquisition. More specifically, analysis of images for diagnosing a patient's condition can be affected by image quality and completeness of the imaging data set. Conditions can include movement occurring during image collection, such as gross motion. For example, an ultrasound probe can move during acquisition due to movement of an operator holding the probe. Additionally or alternatively, a patient being imaged can move and / or an anatomical region of interest to the patient can move within a visible sector of the ultrasound probe displayed to a user. SUMMARY

[0004] In one embodiment, a method for motion compensation in real-time images includes acquiring a first set of imaging data from a visible sector of an imaging system, the visible sector including a region of interest displayed to a user; acquiring a second set of imaging data at a region offset from the visible sector during the acquiring the first set of imaging data; and using the second set of imaging data to compensate for gross motion of the region of interest, resulting in displacement of at least a portion of the region of interest outside of the visible sector. In this way, images can be displayed in a more stable and reliable manner without losing data due to gross motion.

[0005] In one example, acquiring the second set of imaging data from a region offset from the visible sector includes determining a positioning of a buffer region relative to the visible sector. The acquisition beam angle can be adjusted in accordance with the buffer region to collect additional data from a region outside of the visible sector in which gross motion can occur. The additional data can be collected regardless of whether the additional data is used or included in an image frame displayed to a user. However, the visible sector displayed to the user and the image of the region of interest can remain unchanged, thus maintaining an amount of detail of the region of interest that can be easily observed by the user.

[0006] It is to be understood that the above brief description is provided to introduce in simplified form a selection of concepts that are further described in the detailed description. It is not meant to identify key or essential features of the claimed subject matter, the scope of which is defined uniquely by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages mentioned above or in any part of this disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0007] The present application will be better understood by reading the following description of non-limiting embodiments, with reference to the appended drawings, in which:

[0008] Figure 1 An example of an imaging system is shown.

[0009] Figure 2 An example of a movie loop for displaying images from an imaging system is shown. Figure 1

[0010] Figure 3 An example of an imaging sector of an imaging system is shown, where additional data is collected from areas outside the imaging sector. Figure 1

[0011] An example of an imaging sector of an imaging system is shown, where additional data is collected from areas outside the imaging sector. Figure 4 Figure 1 An example of a method for collecting imaging data in addition to data acquired from a visible sector of an imaging system when total motion of an imaging region of interest is detected is shown.

[0012] Figure 5 An example of a method for processing and displaying imaging data collected according to the method of

[0013] Figure 6 An example of a method for processing and displaying imaging data collected according to the method of Figure 5 DETAILED DESCRIPTION

[0014] ​​​The following description relates to a method for acquiring images from an imaging system. In one example, the imaging system can be an ultrasound imaging system, and images can be acquired via an ultrasound probe. The method can provide a motion compensation strategy by tracking the total motion of an imaged object, where total motion is movement that affects the relative position of the imaged object. Thus, total motion is the overall motion of the object, excluding motion differences within the imaged object, such as contractions within the ventricular muscle of the heart. The imaged object can be a region of interest (ROI) of a region displayed as an image, where the region displayed as an image is also referred to herein as a visible sector or an imaging sector. For example, the ROI can be an anatomical feature, such as an organ, tissue, artery, and vein, etc. In some cases, the total motion can cause at least a portion of the ROI to be displaced outside of the visible / imaging sector of the ultrasound probe. To mitigate data loss that results in incomplete display or analysis of the acquired images, the movement of the ROI can be tracked, and additional data can be collected from outside the visible sector to compensate for the movement of the ROI. The additional data can be obtained from a buffer region that is offset from the visible sector, as shown in Figure 3 and Figure 4 The buffer region can be applied according to a custom method for motion tracking during image acquisition, as depicted in Figure 5 The image data acquired via the method of Figure 5 may be processed to provide a stable and complete display of the image according to the method shown in Figure 6

[0015] The custom motion tracking techniques described herein can be applied to various types of imaging modalities, as well as imaging device types and geometries used to obtain images. Further, the techniques can be used for both two-dimensional (2D) imaging and three-dimensional (3D) imaging. In some examples, the techniques can be selectively applied according to observed and / or expected motion of the object to be imaged, e.g., the techniques can include real-time application. Thus, details of the image can not be lost due to motion of the object or at the imaging device. While examples of motion tracking of ultrasound systems and images obtained by ultrasound systems are shown below, object tracking and motion compensation can be applied to other types of images, such as x-ray fluoroscopy images.

[0016] Before further discussing the method for compensating for total motion in ultrasound imaging, Figure 1 ​An exemplary platform for implementing the method is shown in FIG. 1. Therein, a block diagram of a system 100 is depicted in accordance with one embodiment. In the illustrated embodiment, the system 100 is an imaging system, and more particularly an ultrasound imaging system. As shown, the system 100 includes a number of components. The components can be coupled to one another to form a single structure. In one example, the system 100 is an integrated system that is capable of moving from one room (e.g., portably) to another. For example, the system 100 can include one or more components that are configured to couple the system 100 to a wheeled cart. However, in other examples, at least portions of the system 100 can be configured to remain stationary and / or fixed in place.

[0017] In the illustrated embodiment, the system 100 includes a transmit beamformer 101 and a transmitter 102 that drives an array of elements 104 (e.g., piezoelectric crystals) within a diagnostic ultrasound probe 106 (or transducer) to transmit pulsed ultrasound signals into a subject's body or volume (not shown). The elements 104 and probe 106 can have a variety of geometrical shapes. For example, the probe 106 can be a fan-shaped probe, a linear probe, a convex probe, a curved probe, a phased array probe, etc. The ultrasound signals are backscattered from structures (e.g., vessels and surrounding tissue) within the body to produce echoes that return to the elements 104. The echoes are received by a receiver 108. The received echoes are provided to a receive beamformer 110 that performs beamforming and outputs RF signals. The RF signals are then provided to an RF processor 112 that processes the RF signals. Alternatively, the RF processor 112 can include a complex demodulator (not shown) that demodulates the RF signals to form IQ data pairs representative of the echo signals. The RF or IQ signal data can then be directly provided to a memory 114 for storage (e.g., temporary storage).

[0018] The system 100 also includes a system controller 116 that includes a plurality of modules that can be part of a single processing unit (e.g., a processor) or distributed across multiple processing units. The system controller 116 is configured to control the operation of the system 100. For example, the system controller 116 can include an image processing module 122 that receives image data (e.g., ultrasound signals in the form of RF signal data or IQ data pairs) and processes the image data. For example, the image processing module 122 can process the ultrasound signals to generate slices or frames of ultrasound information (e.g., ultrasound images) for display to an operator. The image processing module 122 can be configured to perform one or more processing operations according to a plurality of selectable ultrasound modalities on the acquired ultrasound information. By way of example only, the ultrasound modalities can include color flow, acoustic radiation force imaging (ARFI), B-mode, A-mode, M-mode, spectral Doppler, acoustic streaming, tissue Doppler module, C-scan and elastography, speckle tracking, and chamber quantification. The generated ultrasound images can be 2D or 3D. When multiple 2D images are obtained, the image processing module can also be configured to stabilize or register the images.

[0019] When the echo signals are received, the acquired ultrasound information can be processed in real-time during an imaging session (or scan session). Additionally or alternatively, the ultrasound information can be temporarily stored in the memory 114 during the imaging session and processed in a less than real-time manner in real-time or offline operations. An image memory 120 is included for storing processed slices of acquired ultrasound information that are not scheduled for immediate display. The image memory 120 can include any known data storage medium, such as a permanent storage medium, a removable storage medium, or the like. Additionally, the image memory 120 can be a non-transitory storage medium.

[0020] In operation, the ultrasound system can acquire data, such as a volumetric data set, by various techniques (e.g., 3D scanning, real-time 3D imaging, volumetric scanning, 2D scanning using a probe with a positioning sensor, freehand scanning using voxel correlation techniques, scanning using a 2D or matrix array probe, etc.). Ultrasound images of the system 100 can be generated (at the controller 116) from the acquired data and displayed to an operator or user on a display device 118.

[0021] The system controller 116 is operably connected to a user interface 123 that enables an operator to control at least some of the operations of the system 100. The user interface 123 can include hardware, firmware, software, or a combination thereof that enables a person (e.g., an operator) to directly or indirectly control the operation of the system 100 and its various components. As shown, the user interface 123 includes a display device 118 having a display area 117. In the examples described herein, the display device 118 is a touchscreen display that enables an operator to adjust operational parameters of the system 100 by directly interacting with (e.g., touching) the display device 118. For example, the display device 118 can be configured such that when a user moves a finger / glove / stylus across the face of the display device 118, a cursor on the ultrasound image on the display area 117 moves in a corresponding manner. The display device 118 can detect the presence of a touch by an operator on the display area 117 and can also identify the location of the touch in the display area 117. The touch can be applied by, for example, at least one of a person’s hand, a glove, a stylus, etc. As such, the touch-sensitive display can also be characterized as an input device configured to receive input from an operator. The display device 118 also communicates information from the controller 116 to the operator by displaying information to the operator. The display device 118 and / or the user interface 123 can also communicate in audio. The display device 118 is configured to present information to the operator during an imaging session. The presented information can include ultrasound images, graphical elements, user-selectable elements, and other information (e.g., administrative information, personal information of a patient, etc.). In some embodiments, the user interface 123 can also be configured to interface with (e.g., electrically couple to) one or more user interface input devices 115, such as a physical keyboard, a mouse, and / or a touchpad.

[0022] In addition to the image processing module 122, the system controller 116 can also include a graphics module 124, an initialization module 126, a tracking module 128, and an image recognition module 130. The image processing module 122, the graphics module 124, the initialization module 126, the tracking module 128, and the image recognition module 130 can coordinate with one another to present information to the operator during and / or after an imaging session. For example, the image processing module 122 can be configured to display acquired images on the display device 118, and the graphics module 124 can be configured to display designated graphics, such as a graphical outline, with the ultrasound images that represent a lumen or vessel wall in the acquired images. The image processing module 122 and / or the graphics module 124 within the system controller 116 can also be configured to generate a 3D rendering or image (not shown) of the entire vascular structure.

[0023] The image processing module 122 can further include various sub-modules for image processing. For example, a 2D video processor sub-module can be used to combine one or more frames generated from different types of ultrasound information. Consecutive frames of images can be stored in the memory 114 as a movie loop, as described below with reference to Figure 2 Further described. This movie loop represents a first-in-first-out circular image buffer to capture image data displayed in real-time to a user. The user can freeze the movie loop by entering a freeze command at the user interface 123.

[0024] The image processing module 122 can also include a 3D processor sub-module that can be controlled by the user interface 123 and have access to the memory 114 for 3D ultrasound image data. Thus, for example, three-dimensional images can be generated by volume rendering or surface rendering algorithms and techniques such as ray casting, maximum intensity pixel projection, and the like.

[0025] The tracking module 128 can also be controlled by the user interface 123 and retrieve ultrasound information from the memory 114 to generate motion tracking information for display. Thus, the tracking module 128 can receive processed data from the image recognition module 130, which can include algorithms for detecting and recognizing objects, such as ROIs, within an imaging sector (or visible sector) of the probe 106. The imaging sector can be a region of the FOV of the probe 106 defined by the sector beam of the probe 106 displayed to the user at the user interface 123, and frames presented in the movie loop can be taken from the imaging sector. The image recognition module 130 can include the ability to analyze the imaging sector, recognize ROIs within the imaging sector, such as the heart, liver, lungs, blood vessels, and / or other organs, tissues, and / or structures. The data processed by the image recognition module 130 and sent to the motion tracking module 128 can include an indicator identifying the location of the recognized ROIs in a set of real-time images taken during an imaging session. The motion tracking module can include motion tracking algorithms for monitoring movement in the recognized ROIs.

[0026] In one example, the motion tracking capability of the tracking module can be implemented by speckle tracking. In speckle tracking, speckle information in the acquired images is used to track the motion of the ROIs. For example, the naturally occurring speckle pattern in the myocardium of the heart can be tracked, where the tracking can be implemented by various block matching techniques. Block matching techniques can include similarity measures such as sum of absolute differences or cross-correlation. Other motion tracking techniques can include contour tracking, where a contour can be detected in an image and tracked over time, and optical flow, where the changes in the intensity pattern over time are analyzed.

[0027] The tracking module 128 can also include an algorithm for motion compensation that enables removal of gross motion from the displayed images. The motion compensation algorithm can provide instructions for scaling, rotating, and / or translating the images to reduce gross motion from the images. Gross motion can be detected based on the speckle tracking or border detection algorithms implemented at the tracking module 128. In some cases, the tracking module can receive signals from one or more motion sensors 107 embedded into the probe 106 that detect motion at the ROI. For example, the probe 106 can include optical markers and one or more cameras that photograph the probe 106. The processing unit can be used to analyze the images from the one or more cameras to estimate the motion of the probe 106. As another example, the probe 106 can include an accelerometer to provide signals that can be used to estimate the motion of the probe. In yet another example, a magnetic position sensor can be used. Additionally, a hybrid variant using both image analysis and motion sensors is possible.

[0028] In some cases, the tracking module 128 can be configured to determine the boundary of the ROI, which can be, for example, labeled and / or tracked to monitor the relative position of the ROI within the sector. In another example, a segmentation algorithm can be applied to the images captured within the sector to identify and track the ROI. For example, various segmentation techniques, such as region, threshold, edge, cluster, etc. detection, can be used to segment the images into segments, where each segment includes image pixels that share a common attribute.

[0029] The image recognition module 130 can access the stored images or videos from either or both of the memory 114 and the memory 120 prior to analyzing the stored images / videos (e.g., image library). For example, with the parameters of the performed protocol known (type of ultrasound, scan plane, tissue being imaged, etc.), the image recognition module 130 can compare the real-time images on the display area 117 to the images stored in the memory 120 in order to analyze the images, thereby improving the accuracy of placement and utilization of the analysis tools. In alternative embodiments, instead of utilizing the image recognition module 130 and image library, the system controller 116 can host instructions for analyzing the acquired imaging data (e.g., ultrasound images / videos acquired with the probe) and automatically determining the desired placement of one or more analysis tools, such as regions of interest.

[0030] The screen of the display area 117 of the display device 118 is composed of a series of pixels that display data acquired with the probe 106. The acquired data includes one or more imaging parameters computed for each pixel of the display or a group of pixels (e.g., a group of pixels assigned the same parameter value) of the display, where the one or more computed image parameters include one or more of intensity, velocity, color flow velocity, texture, granularity, contractility, deformation, and deformation rate values. The series of pixels then make up a display image generated from the acquired ultrasound data.

[0031] As described above, ultrasound images can be collected continuously, and the continuous images can be displayed to a user as an imaging loop, such as a cine loop. The cine loop presents a set number of consecutive image frames as a dynamic display of a region of interest (ROI), for example, similar to a short video. For example, in Figure 2 A cine loop 200 is represented in FIG. 2. The cine loop 200 relies on a set of image frames obtained, for example, from an ultrasound imaging system.

[0032] The set of image frames is arranged according to a time sequence. For example, a first image frame 202 can be a first captured image of the set of image frames (e.g., x = 1, where x is the temporal order of the images according to acquisition time). A second image frame 204 can be acquired after the first image frame 202, and a third image frame 206 can be acquired after the second image frame 204. The sequence continues until a target number (n) of images are obtained, as indicated by a final image frame 208 of the set of image frames. The cine loop 200 is displayed to a user as a continuous loop at a user interface (e.g., the user interface 123 of FIG. 1) when requested by the user. For example, each image frame of the set of image frames is sequentially displayed according to a time frame (e.g., a duration between image acquisitions), similar to an image acquisition frequency for capturing the images. When the final image frame 208 is displayed, the cine loop returns to the first image frame 202. Figure 1

[0033] ​The ROI 210 can be a target imaging object within an imaging sector 212 for each image frame in the set of image frames. In one example, the ROI 210 can be an anatomical feature associated with gross motion. The gross motion can cause the ROI 210 to move relative to the boundaries of the imaging sector 212 of the first image frame 202 in subsequently acquired image frames. However, motion compensation algorithms can be used to account for the gross motion such that the ROI 210 remains in a stationary position within the imaging sector 212 on each image frame shown in the cine loop 200. For example, in addition to changing the position of the ROI 210, the ROI 210 can also experience gross motion that can temporarily change the dimensions of the ROI 210 (as shown in the second image frame 204) or change the shadow of the ROI 210 (as shown in the third image frame 206) in addition to changing the position of the ROI 210. Changing / altering the position of the ROI 210 can include a translation of the ROI 210 in space relative to the imaging sector 212. However, while a change in dimensions is shown in the second image frame 204 and a change in shadow is shown in the third image frame 206, a change in the relative position of the ROI 210 within each of the image frames is not shown. The relative position of the ROI 210 within the imaging sector 212 remains consistent between the set of image frames of the cine loop 200.

[0034] By keeping the position of the ROI 210 stationary in the cine loop 200, a user can easily observe and evaluate the ROI 210. However, the gross motion of the ROI 210 can temporarily displace the ROI 210 beyond the boundaries of the imaging sector 212. In such cases, at least a portion of the data specific to the ROI 210 can be lost in the image frames of the cine loop 200 where at least a portion of the ROI 210 moves outside of the imaging sector 212. Thus, the cine loop 200 can include a point or instant in time where the ROI 210 appears to skip or miss data.

[0035] Other issues can arise in gross motion during image acquisition. As described above, the gross motion can be caused by movement of the imaging object. In other examples, the gross motion can be due to movement of the imaging device, such as an ultrasound probe. In addition to the adverse effects on image / data display (e.g., as a cine loop), such movement can also present challenges to post-processing of the ultrasound images when the ROI moves within the image frames (e.g., the relative position of the ROI changes). To effectively analyze and process data from the ROI over time, it is desirable to remove the gross motion, for example, by a motion compensation algorithm. For example, removing the gross motion causes the ROI to appear in a fixed position in the cine loop. Additionally, for systems that apply thermal therapy (e.g., high intensity focused ultrasound) to a target tissue region, gross motion of the target tissue can cause the thermal therapy to affect tissue regions that do not need the thermal therapy.

[0036] In one example, the aforementioned problem can be addressed using a motion tracking strategy that allows the collection of additional imaging data outside the imaging area displayed to the user (e.g., outside the imaging sector). This motion tracking strategy avoids skipping data analysis and / or display at time points or image frames where at least a portion of the ROI is outside the imaging sector. A more complete set of results can be obtained, and / or a more stable display of the results can be achieved.

[0037] Furthermore, motion tracking strategies can eliminate the reliance on modeling the total motion and / or data display of at least a portion of the ROI outside the imaging sector at certain time points or in image frames. Therefore, users can obtain more complete and accurate results. Similarly, image acquisition using sector sizes similar to the ROI is enabled. Thus, images of the ROI can be displayed on display devices (such as...) Figure 1 The display device (118) appears large enough to allow users to easily assess the ROI during real-time imaging, while collecting additional data from outside the imaging sector for total motion compensation.

[0038] As mentioned above, total motion during image acquisition may cause the ROI to move outside the imaging sector of the imaging system or device (e.g., the visible sector displayed to the user). Figure 3 An example of such movement is depicted. A first example of an imaging dataset 300 showing a representative time-lapse image of a ROI 302 within an imaging sector 304 of an ultrasound imaging system is shown. In one example, ROI 302 can be a heart. The imaging dataset 300 includes a first location 306 of ROI 302 captured in a first frame and a second location 308 of ROI 302 captured in a second frame.

[0039] In the first frame, the first position 306 of ROI 302 depicts ROI 302 completely positioned within the boundary of imaging sector 304. Therefore, the complete image of ROI 302 is displayed to the user. However, in the second position 308, ROI 302 is shifted to the right within imaging sector 304, such that a portion of ROI 302 (within...) Figure 3 The area indicated in the middle (as a shadow area) is located outside imaging sector 304.

[0040] Shifting of the ROI within the imaging sector can occur intermittently across a set of frames displayed in a movie loop. In frames where at least a portion of the ROI has shifted beyond the boundaries of the imaging sector, for example, as... Figure 3As shown in the second position 308 of the ROI 302, an amount of data corresponding to a portion of the ROI positioned outside of the imaging sector can be lost. This can result in skipping analysis and / or display of data in frames where the ROI is shifted outside of the imaging sector 304. Thus, a set of results can be incomplete and display of results can be unstable. In some examples, the lost data can be accounted for by extrapolating available data and / or modeling the missing region. However, the accuracy of the estimated data can be low.

[0041] In one example, movement of the ROI 302 outside of the boundaries of the imaging sector 304 can be addressed by a custom strategy for tracking motion of the ROI 302 during post-processing of imaging data for real-time display. Real-time display of imaging data includes presenting a cine loop to a user. The strategy can include obtaining a larger data set than the data set displayed to the user at the imaging sector 304. For example, a data set acquired via a wider beam angle than required by data constrained by the boundaries of the imaging sector 304 can be captured. A portion of the wider angle data set (e.g., a central portion) can be presented to the user and a buffer region 310 can be included in the wider angle data set, where the buffer region 310 is outside of the imaging sector 304. Figure 3 The buffer region 310 is shown in shadow in order to allow collection of data outside of the data corresponding to the imaging sector 304. However, the buffer region 310 can not be displayed to the user. Thus, the additional data can provide hidden available imaging results that can be used to supplement the displayed imaging data in the imaging sector 304. Additionally, the buffer region 310 can be applied to define where additional data will be acquired, regardless of whether the additional data is actually used to fill in missing data within the imaging sector 304. In other words, in the event that data is lost during an imaging session due to gross motion, additional data can be acquired upon detection of the gross motion and, depending on the gross motion at the ROI 302, the additional data can or can not be displayed with the imaging data within the imaging sector 304.

[0042] The buffer region 310 can be offset from the imaging sector 304. For example, the buffer region 310 can cover an area outside of the area of the imaging sector 304. Thus, the imaging sector 304 can have a first cross-sectional width 312 and the buffer region can have a second cross-sectional width 314 that is greater than the first cross-sectional width 312.

[0043] In addition to image acquisition from within the imaging sector 304, data can also be collected simultaneously with the image acquisition. As described above, in addition to the imaging sector 304, data can also be obtained from the buffer region 310 using a wider beam angle. For example, the beams can be transmitted in a manner that are spaced further apart from one another, e.g., a low beam density, which can reduce lateral resolution, but can allow the frame rate to be maintained. Alternatively, the beam density can be maintained constant while the frame rate is reduced, thereby maintaining the desired resolution of the image. In some examples, a balance between applying a lower beam density and a lower frame rate can be used. In some cases, as described further below, additional data can be dynamically collected from the buffer region 310. In such examples, the dynamic collection can vary the frame rate, which can not be permissible depending on the image acquisition parameters, and thus the beam resolution can be adjusted accordingly rather than the frame rate.

[0044] While adjusting the beam angle for image acquisition can simultaneously reduce the beam density relative to a narrower angle in some cases, the buffer region 310 can be selected to minimize the loss of resolution. The size of the imaging sector 304 displayed to the user can remain consistent despite the additional available data, and thus the frames presented to the user in cine loops can maintain a high level of resolution (e.g., beam density). The amount of additional data obtained from the buffer region 310 can be adjusted depending on the type of probe or application used to acquire the image. The additional data can be processed via a motion compensation algorithm of a tracking module (e.g., tracking module 128 of Figure 1 The additional data can be processed via a motion compensation algorithm of a tracking module (e.g., tracking module 128 of Figure 1 The additional data can be processed via a motion compensation algorithm of a tracking module (e.g., tracking module 128 of

[0045] The amount of additional data to be obtained from the buffer region 310, e.g., the size of the buffer region, can be determined based on a set estimate of the total motion at the ROI 302. For example, an analysis of previous analyses of the same type of ROI can be used to infer an average amount of expected movement of the ROI type. The buffer region 310 can then be set to accommodate the set estimate of the total motion and applied throughout the imaging session. For example, when the total motion is estimated frame-by-frame in real time, the size of the buffer region 310, e.g., the additional sector size, can be increased starting from the next acquired frame to keep the ROI 302 entirely within the buffer region 310. If the total motion is estimated to be within a smaller sector, the size of the buffer region 310 can be decreased starting from the subsequent frame.

[0046] As another example, the total motion can be estimated over a longer period of time, such as one full cardiac cycle or longer, and the maximum estimated motion can be used to set a new buffer region size. Further, in yet another example, a study of previously recorded images can be used to set a fixed buffer region size for a given probe / application to cover expected motion outside of the imaging sector 304. A combination of the above methods can also be used, e.g., a fixed buffer region size is initially applied and the size is adjusted during image acquisition.

[0047] In some examples, the additional image data can be collected in a dynamic manner. In other words, Figure 3 The positioning of the buffer region 310 relative to the imaging sector 304 can be adjusted depending on how the ROI is estimated to move within a particular imaging session rather than a set estimate. For example, due to movement of the imaging device, e.g., at the ultrasound probe, or due to movement of the patient, total motion can occur in an unexpected manner at the ROI. In such cases, the total motion can deviate from the set estimate of the total motion. In one example, when total motion is detected during image acquisition, the location of the total motion can be estimated, e.g., based on speckle tracking and / or signals from motion sensors at the tracking module. Depending on the extent of the total motion, the buffer region 310 can be adjusted to focus the additional data to be obtained in the region where the total motion is estimated to occur.

[0048] For example, Figure 4 A second example of an imaging data set 400 is shown in FIG. 4. The imaging data set 400 includes Figure 3 The elements shown, such as the imaging sector 304 and the ROI 302 in the first position 306 and the second position 308, where the second position 308 results in a portion of the ROI 302 being positioned outside of the imaging sector 304. Relative to Figure 3of the first example of the imaging data set 300, when the ROI 302 is in the second position 308, the buffer region 310 is offset to an area outside the boundary of the imaging sector 304, providing a complete image of the ROI 302.

[0049] Additional data, e.g., other than data collected from within the imaging sector 304, can be obtained from a local area outside the imaging sector 304 in order to obtain a complete imaging data set from the ROI 302. Various techniques can be implemented for detecting motion outside the imaging sector 304. For example, speckle tracking or boundary tracking can be used to track the overall motion of the ROI 302, where portions of the ROI 302 can be tracked outside the imaging sector 304. As another example, prior knowledge of the shape of the ROI 302 can be used in conjunction with boundary tracking to estimate the position of the ROI 302. It can be observed which portions of the prior known shape of the ROI 302 are not fully detected within the imaging sector 304, and which portions of the ROI 302 are missing to indicate that portions of the ROI 302 are located outside the imaging sector 304.

[0050] Adjustment of the buffer region 310 can be performed dynamically, e.g., in response to an estimate of the overall motion of the ROI 302. Thus, the area covered by the buffer region 310 can not remain uniform between imaging sessions of the same type of ROI. The amount of additional data can be tailored according to the overall motion conditions that occur during a particular imaging session.

[0051] As another example, a tailored motion tracking strategy can be applied in real time. For example, as described above, a motion tracking algorithm or a motion sensor at the ultrasound probe can detect motion as images are acquired. Thus, the image acquisition process can be adjusted in response to detecting motion. In one example, acquisition of additional data outside the imaging sector can be collected only when motion is detected. Thus, application of a buffer region to collect additional imaging data can be intermittent and selective. Adjustments to the applied image acquisition process, such as modifications to frame rate and image quality / resolution, can be needed to adapt the motion tracking strategy for real-time application.

[0052] Figure 5 An example of a method 500 for real-time imaging data acquisition with tailored motion tracking is shown in FIG. 5, and Figure 6 A method 600 for post-processing and display of real-time imaging data is depicted in FIG. 6. The method 500 can be applied to the process of data acquisition, e.g., image capture by a device such as an ultrasound probe, an imaging system such as an ultrasound imaging system. Instructions for performing the methods 500 and 600 can be executed by a system controller, such as the system controller 116 of Figure 1 The system controller 116 of FIG. 1 can be configured to perform the methods 500 and 600. The system controller 116 can be configured to perform the methods 500 and 600 based on instructions stored on a memory of the controller in combination with data from sensors of the imaging system, such as the ultrasound probe 102 of FIG. 1.Figure 1 the signals received by the motion sensor 107 of the ultrasound probe 101.

[0053] Turning first to the method 500, the method includes identifying an ROI within an imaging sector of an ultrasound probe at 502. For example, the ROI can be identified by an image recognition module, such as the image recognition module 130 of the ultrasound probe 101. The imaging sector can define an image displayed to a user in real-time and post-processing, and can be selected to produce an image with the highest level of beam density for a given beam angle, where the beam angle is related to the size of a target region or object to be imaged. In one example, an image processing algorithm of the image recognition module can be used in real-time to locate a boundary, contour, or surface within an image or set of images that define the shape of the ROI. The ROI can thereby be tracked in real-time during real-time image acquisition. Figure 1

[0054] At 504, the method includes confirming whether gross motion is detected. For example, gross motion of the ROI or the probe itself can be detected. In one example, the ROI can move without moving the probe, such as a shift of an internal organ. In another example, the probe can be moved such that the ROI appears to move within the imaging sector. In other examples, both the ROI and the probe can move. Regardless of the source of movement, the ROI can move within the imaging sector, which can be detected by analyzing the images, such as by speckle tracking, or by motion sensors of the probe, such as the motion sensor 107 of the ultrasound probe 101. Figure 1

[0055] If gross motion is not detected, the method continues to 506 to collect data from within the imaging sector of the probe. Collecting data can include acquiring images of the ROI according to a selected acquisition mode, frame rate, resolution / beam density, etc. The method proceeds to 514 to store the data, as further described below.

[0056] Returning to 504, if gross motion is detected, the method continues to 508 to determine whether a magnitude of the gross motion is estimated to be greater than a threshold. The threshold can be an amount of movement at the ROI, such as a translation of the ROI, that is estimated to keep the ROI within the boundaries of the imaging sector. In other words, the threshold can be an estimated magnitude of allowable movement that the ROI can exhibit without at least partially displacing outside of the imaging sector. For example, the threshold can be an angle or lateral distance between an outer edge of the ROI and an outer edge of the imaging sector in a frame of an initial acquisition, where the ROI is centered in the image.

[0057] ​​If the estimated and / or detected amount of total motion is not greater than a threshold, the method proceeds to 506 to collect data from within the imaging sector. If the estimated and / or detected amount of total motion is greater than the threshold, the method proceeds to 510 to determine a suitable buffer region applicable to the imaging sector. For example, the amount of total motion can be estimated based on signals from a motion sensor and / or based on real-time speckle tracking, and a target cross-sectional area of ​​the buffer region can be selected that extends beyond, for example, the boundary of the initial imaging sector. The target cross-sectional area can be a balance between providing a sufficiently large area (e.g., beam angle) to keep the ROI within the buffer region regardless of the total motion, while minimizing a corresponding reduction in beam density.

[0058] In one example, the maximum area of ​​the buffer region can be determined based on the impact of the increase in area on image resolution and / or frame rate. For instance, the frame rate can be kept constant, and the spacing between the transmit beams can be increased by 10%, corresponding to a 10% reduction in resolution. Therefore, the maximum area can increase by 10%. Alternatively, the beam spacing (and resolution) can be kept constant, and the frame rate can be reduced by up to 10%. By reducing the frame rate, more beams can be added to the imaging sector, allowing the imaging sector to increase by up to 10% (by adding the buffer region). Furthermore, the maximum area of ​​the buffer region can be determined by changing a combination of resolution and frame rate.

[0059] In one example, a buffer region can be applied to each image frame collected during image acquisition. For instance, total motion can be initially detected and analyzed to determine settings for subsequent image acquisition, including the beam angle and density used. The buffer region can be a static parameter; for example, a suitably wide beam angle can be determined that magnifies the area over which imaging data is collected without localization, such as... Figure 3 As shown. In other words, data collection can be increased with equal probability along each dimension.

[0060] For example, a buffer region can be alternatively applied dynamically with a bias to collect additional data from specific regions where the total motion is estimated to occur. For instance, it can be as follows: Figure 4 The buffer region is adjusted to include the area to which the ROI is estimated to be at least temporarily shifted. The biased buffer region can be applied to each image frame acquired during image acquisition. By applying a biased buffer region, resolution loss can be reduced compared to using a buffer region similarly applied to each dimension. However, the initial analysis may be longer and more complex. In some examples, the application of the biased buffer region can vary, for example, whether the frame rate or resolution is adjusted, or whether the buffer region is fixed or dynamically applied, depending on how the acquired imaging data is used and can be selected by the user.

[0061] In another example, the buffer region can be applied in real-time to image frames where the ROI has moved outside the imaging sector. Real-time deployment of the buffer region can achieve maximum image resolution and frame rate while reducing data volume and corresponding memory storage requirements. However, real-time application may cause image resolution and / or frame rate to vary over time.

[0062] At 512, the method includes collecting data via a wider beam angle (e.g., from both the imaging sector and the buffer region). Collecting data may include acquiring images of the ROI according to a selected acquisition mode, frame rate, resolution / beam density corresponding to the wider beam angle, etc. At 514, the data is stored. Storing the data may include, for example, storing the collected data in a suitable format relative to the image processing module (e.g., image processing module 122) of the system controller (such as...). Figure 1 The method ends at memory location 114 and / or 120.

[0063] Turn now Figure 6 Method 600 describes how imaging data (such as those obtained via...) can be processed. Figure 5 This is an example of post-processing the data collected and stored by method 500. At 602, the method includes obtaining imaging data collected for a region of interest (ROI, e.g., the ROI of method 500). Obtaining the imaging data may include processing the data from the memory of the system controller (e.g., Figure 1 The memory 114 and / or 120 retrieves data and sends the data to the tracking module of the system controller, such as... Figure 1 The tracking module 128. In one example, the imaging data can be transmitted via... Figure 5 The imaging data acquired by method 500 may include a first set of imaging data obtained from within the imaging sector and a second set of imaging data collected from a buffer region offset from the imaging sector, the second set of data being collected in response to the detection of motion at the ROI.

[0064] At 604, the method includes tracking motion at the ROI based on the retrieved first set of imaging data. For example, the ROI can be identified via an analysis module of the system controller, as described above at 502 of reference method 500. The position of the ROI within the imaging sector of the imaging data can be monitored across a set of images corresponding to the retrieved imaging data.

[0065] At 606, the method includes confirming whether the ROI is shifted outside, for example, the boundary of an imaging sector on that set of images. This can be done by an analysis module (such as...) Figure 1the image recognition module 130) provides a determination of whether the ROI is moving outside the boundaries of the imaging sector. The ROI is displaced outside the boundaries can include at least a portion of the ROI moving outside the imaging sector. If it is determined that the ROI is not moving outside the imaging sector, then the method proceeds to 610 to remove gross motion from the ROI in the imaging data, as described below, without adding information from the second set of data to the first set of data.

[0066] If it is confirmed that the ROI is moving outside the imaging sector in any of the images of the set, then the method continues to 608 to add information from the second set of imaging data. For example, as described in the method 500, the detection of gross motion during image acquisition can result in a widening of the beam angle to collect additional data outside the imaging sector during the imaging session. For example, the additional data collected as provided by the second set of imaging data can be used to supplement the data corresponding to the image frames in which the ROI moved outside the imaging sector. In other words, the additional data can provide missing data within the imaging sector relative to obtaining a complete image of the ROI while the ROI remains in a fixed position for display. Figure 5

[0067] At 610, the method includes removing gross motion, if present, from the ROI in the acquired images. Removing gross motion can include using a motion compensation algorithm implemented at the tracking module of the system controller. For example, the first set of imaging data from within the imaging sector supplemented by the additional data from the second set of imaging data can be fed to the motion compensation algorithm. A displacement field can be used to determine the overall gross motion of the ROI, which can estimate the amount of deformation of the shape change of the ROI. The deformation or shape change can represent the gross motion of the ROI. Gross motion can be compensated for by subtracting the overall motion or deformation from the set of image data, thereby removing the average motion from the images of the ROI to be displayed. In some examples, the images of the ROI can be scaled, rotated, and / or translated based on the gross motion. In one example, an affine transformation of the images can be performed, which includes a transformation between two dimensional spaces that includes a linear transformation followed by a translation. However, other methods and techniques for motion compensation can be used without departing from the scope of the present disclosure.

[0068] At 612, the method includes displaying the images with the gross motion removed. For example, the images can be presented as a cine loop, and the removal of gross motion allows the ROI to appear stationary, e.g., not moving and in a fixed position within the imaging sector, as the image frames of the cine loop are displayed and looped. The buffer region is not shown in the image frames of the cine loop. By selectively incorporating the additional imaging data provided by the application of the buffer region, the ROI appears stable throughout the cine loop, e.g., the ROI images do not skip or appear temporarily as missing data.​

[0069] At 614, post-processing of the image can be performed. For example, speckle tracking, border detection, etc. can be applied to the image with the gross motion removed. Alternatively, in other examples, post-processing can be performed without removing the gross motion. Conversely, post-processing can be applied directly to the imaging data with the data from the buffer region added (e.g., omitting 610 and 612). The method ends.

[0070] In this manner, a complete imaging data set can be obtained, enabling a stable real-time display of the imaging data set to be presented to a user. By collecting additional imaging data outside the visible sector of the FOV of the imaging device, gross motion of the imaging ROI can be tracked, and the additional imaging data can be used to supplement data corresponding to portions of the ROI that displace outside the visible sector during real-time image acquisition. Compensation for missing data via modeling is eliminated, and an image presentation enabling effective analysis and evaluation is achieved.

[0071] A technical effect of collecting additional data from a buffer region outside the visible sector of the imaging system is that, in response to detecting gross motion causing the ROI to displace at least partially outside the visible sector, the acquisition beam angle of the imaging system is widened, enabling a larger imaging data set to be collected while maintaining consistent display of the ROI in the imaging loop presented to a user.

[0072] As used herein, an element or step recited in the singular and preceded with the word "a" or "an" should be understood as not excluding plural of said elements or steps, unless explicitly stated that

[0073] The present disclosure also provides support for a method for motion compensation in real-time images, the method comprising: acquiring a first set of imaging data from a visible sector of an imaging system, the visible sector comprising a region of interest displayed to a user; acquiring a second set of imaging data at a region offset from the visible sector during the acquiring the first set of imaging data; and using the second set of imaging data to compensate for gross motion of the region of interest, resulting in displacement of at least a portion of the region of interest outside the visible sector. In a first example of the method, using the second set of imaging data to compensate for gross motion of the region of interest comprises feeding the second set of imaging data to a motion compensation algorithm implemented at a system controller of the imaging system. In a second example of the method, which optionally includes the first example, the method further comprises: estimating the gross motion of the region of interest in the first set of imaging data, and wherein estimating the gross motion of the region of interest comprises detecting motion at a motion sensor of an imaging device of the imaging system. In a third example of the method, which optionally includes one or both of the first example and the second example, the method further comprises: estimating the gross motion of the region of interest in the first set of imaging data based on an analysis of a real-time image, and wherein the analysis of the real-time image is performed using a motion compensation algorithm. In a fourth example of the method, which optionally includes one or more or each of the first through third examples, using the second set of imaging data to compensate for gross motion of the region of interest comprises using the second set of imaging data to supplement missing portions of data in the first set of imaging data, the missing portions resulting from the displacement of at least the portion of the region of interest outside the visible sector. In a fifth example of the method, which optionally includes one or more or each of the first through fourth examples, acquiring the second set of data at the region offset from the visible sector comprises widening a beam angle of an imaging device used to acquire both the first set of data and the second set of data.

[0074] The present disclosure also provides support for an imaging system including an imaging device configured to acquire images of a region of interest within a visible sector of the imaging device, and a system controller configured with executable instructions stored on a non-transitory memory that, when executed, cause the system controller to collect a first set of imaging data from a region offset from the visible sector in response to detecting gross motion at the region of interest in addition to a second set of imaging data collected from the visible sector, and utilize the first set of imaging data to supplement data missing from the second set of imaging data to display a processed image based on the second set of imaging data. In a first example of the system, the imaging device further includes a motion sensor embedded at the imaging device, and wherein the motion sensor is configured to detect the gross motion at the region of interest. In a second example of the system, which optionally includes the first example, the gross motion at the region of interest is detected by a motion tracking algorithm implemented at a tracking module of the system controller, and wherein the motion tracking algorithm includes one or more of speckle tracking, contour tracking, and optical flow. In a third example of the system, which optionally includes one or both of the first example and the second example, the region offset from the visible sector includes a region extending outside a boundary of the visible sector. In a fourth example of the system, which optionally includes one or more or each of the first through third examples, the region offset from the visible sector is adjusted to collect the first set of imaging data from a region estimated to have movement of the region of interest, and wherein the movement is estimated based on one or more of speckle tracking, boundary tracking, and prior knowledge of a shape of the region of interest. In a fifth example of the system, which optionally includes one or more or each of the first through fourth examples, the processed image displayed based on the second set of imaging data is displayed in a cine loop, and wherein the second set of imaging data is processed to present the region of interest in a fixed position in the cine loop. In a sixth example of the system, which optionally includes one or more or each of the first through fifth examples, the first set of imaging data is used to supplement the data missing from the second set of imaging data during display of the processed image in the cine loop as the region of interest moves outside the visible sector. In a seventh example of the system, which optionally includes one or more or each of the first through sixth examples, the first set of data is collected concurrently with the second set of data, and wherein only the second set of data is displayed, the second set of data showing the region of interest in the visible sector with the gross motion removed. In an eighth example of the system, which optionally includes one or more or each of the first through seventh examples, the gross motion is removed using a motion compensation algorithm.In a ninth example of the system, which optionally includes one or more or each of the first through eighth examples, the imaging device is an ultrasound probe, and wherein a beam density of data provided by the first set of imaging data and used to acquire the image of the region of interest is adjusted based on a type of the ultrasound probe.

[0075] The present disclosure also provides support for an ultrasound imaging system including a first set of imaging data acquired in addition to a second set of imaging data, the first set of imaging data acquired in response to detecting gross motion at a region of interest shown in the second set of imaging data, and wherein the first set of imaging data is fed to a motion compensation algorithm to supplement data omission in the second set of imaging data caused by the gross motion at the region of interest. In a first example of the system, the first set of imaging data is acquired from a buffer region outside of a visible sector from which the second set of imaging data is acquired, and wherein the buffer region is determined based on an estimate of a magnitude of the gross motion and a target image resolution. In a second example of the system, which optionally includes the first example, the second set of imaging data is shown as a processed image of the region of interest within the visible sector presented to a user, and the buffer region is not shown. In a third example of the system, which optionally includes one or both of the first and second examples, the second set of imaging data is shown as the processed image in the region of interest as a cine loop, and wherein the cine loop is shown without skipping, and the region of interest is shown in each image frame in the cine loop in a size similar to that of the region of the visible sector.

[0076] This written description uses examples to disclose the application, including the best mode, and also to enable any person skilled in the art to practice the application, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the application is defined by the claims, and can include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent

Claims

1. A method for motion compensation in real-time images, the method comprising: acquiring a first set of imaging data from a visible sector of an imaging system, the visible sector comprising a region of interest displayed to a user; acquiring a second set of imaging data at a region offset from the visible sector during the acquiring the first set of imaging data; and compensating for total motion of the region of interest using the second set of imaging data, the total motion causing at least a portion of the region of interest to be displaced outside the visible sector; wherein compensating for total motion of the region of interest using the second set of imaging data comprises supplementing missing portions of data in the first set of imaging data with the second set of imaging data, the missing portions caused by the displacement of at least the portion of the region of interest outside the visible sector.

2. The method of claim 1, wherein compensating for total motion of the region of interest using the second set of imaging data comprises feeding the second set of imaging data to a motion compensation algorithm implemented at a system controller of the imaging system.

3. The method of claim 1, the method further comprising estimating the total motion of the region of interest in the first set of imaging data, and wherein estimating the total motion of the region of interest comprises detecting motion at a motion sensor of an imaging device of the imaging system.

4. The method of claim 1, the method further comprising estimating the total motion of the region of interest in the first set of imaging data based on an analysis of the real-time images, and wherein the analysis of the real-time images is performed using a motion compensation algorithm.

5. The method of claim 1, wherein acquiring the second set of imaging data at the region offset from the visible sector comprises widening a beam angle of an imaging device used to acquire both the first set of imaging data and the second set of imaging data.

6. An imaging system, the imaging system comprising: an imaging device configured to acquire images of a region of interest within a visible sector of the imaging device; and a system controller configured with executable instructions stored on a non-transitory memory that, when executed, cause the system controller to: collect a first set of imaging data from a region offset from the visible sector in response to detecting total motion at the region of interest, the total motion causing at least a portion of the region of interest to be displaced outside the visible sector, in addition to a second set of imaging data collected from the visible sector; and supplement missing data from the second set of imaging data with the first set of imaging data to display a processed image based on the second set of imaging data, the missing data caused by the displacement of at least the portion of the region of interest outside the visible sector.

7. The imaging system of claim 6, wherein the imaging device further comprises a motion sensor embedded at the imaging device, and wherein the motion sensor is configured to detect the total motion at the region of interest.

8. The imaging system of claim 6, wherein the total motion at the region of interest is detected by a motion tracking algorithm implemented at a tracking module of the system controller, and wherein the motion tracking algorithm comprises one or more of speckle tracking, contour tracking, and optical flow.

9. The imaging system of claim 6, wherein the region offset from the visible sector comprises a region extending outside a boundary of the visible sector.

10. The imaging system of claim 6, wherein the region offset from the visible sector is adjusted to collect the first set of imaging data from a region where movement of the region of interest is estimated to occur, and wherein the movement is estimated based on speckle tracking, boundary tracking, and prior knowledge of a shape of the region of interest.

11. The imaging system of claim 6, wherein the processed image displayed based on the second set of imaging data is displayed in a cine loop, and wherein the second set of imaging data is processed to present the region of interest in a fixed position in the cine loop.

12. The imaging system of claim 11, wherein the first set of imaging data is used to supplement the data missing from the second set of imaging data during display of the processed image in the cine loop as the region of interest moves outside the visible sector.

13. The imaging system of claim 6, wherein the first set of imaging data is collected simultaneously with the second set of imaging data, and wherein only the second set of imaging data is displayed, the second set of imaging data showing the region of interest in the visible sector with the total motion removed.

14. The imaging system of claim 13, wherein the total motion is removed using a motion compensation algorithm.

15. The imaging system of claim 6, wherein the imaging device is an ultrasound probe, and wherein a beam density used to acquire the image of the region of interest and an amount of data provided by the first set of imaging data is adjusted based on a type of the ultrasound probe.

16. An ultrasound imaging system, the ultrasound imaging system comprising: a first set of imaging data acquired in addition to a second set of imaging data collected from a visible sector, the first set of imaging data acquired from a region offset from the visible sector in response to detecting a total motion at a region of interest displayed in the second set of imaging data, the total motion causing at least a portion of the region of interest to displace outside the visible sector, and wherein the first set of imaging data is fed to a motion compensation algorithm to supplement data omitted in the second set of imaging data caused by the total motion at the region of interest, the data omission caused by the displacement of at least the portion of the region of interest outside the visible sector.

17. The ultrasound imaging system of claim 16, wherein the first set of imaging data is acquired from a buffer region acquired outside the visible sector, and wherein the buffer region is determined based on an estimate of a magnitude of the total motion and a target image resolution.

18. The ultrasound imaging system of claim 17, wherein the second set of imaging data is displayed as the processed images of the region of interest within the visible sector presented to a user, and the buffer region is not displayed.

19. The ultrasound imaging system of claim 18, wherein the second set of imaging data is displayed as the processed images in the region of interest as a cine loop, and wherein the cine loop is displayed without skipping, and the region of interest is shown in each image frame in the cine loop in a size similar to that of the region of the visible sector.

Images