Ultrasound imaging system and method
By automatically identifying the fragments of interest in the ultrasound imaging system and applying time scaling technology, the problem of inconsistent video playback cycles in the panoramic view is solved, and the effect of playing the same amount of time at the same time in the panoramic view is achieved, reducing visual confusion and improving The efficiency of clinicians' interpretation and diagnosis of panoramic videos.
Patent Information
- Application Number
- CN202110423173.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-05-08
- Filing Date
- 2021-04-20
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-04-20
AI Technical Summary
In ultrasound imaging systems, when generating panoramic views, inconsistent playback cycles of each video lead to visual confusion by clinicians and difficulty identifying important features.
By automatically identifying multiple fragments of interest in an ultrasound imaging system and applying time scaling techniques, each video spends the same amount of time in a panoramic view to play.
All videos in the panoramic view are realized to play the same amount of time at the same time, reducing visual discontinuity, helping clinicians to view and interpret panoramic videos more easily, focusing on important diagnostic aspects.
Smart Images

Figure CN113616237B_ABST
Abstract
Description
Technical Field
[0001] The subject matter disclosed herein generally relates to ultrasound imaging systems and methods of ultrasound imaging. Background Art
[0002] Ultrasound imaging is an imaging modality capable of acquiring ultrasound image data that can be viewed in real time or stored and reviewed from time to time after acquisition. Ultrasound imaging systems also have the ability to acquire and store one or more videos, each video containing multiple image frames acquired at different points in time. Displaying ultrasound image data as a video allows clinicians to view how a person's anatomy changes over time. The size of the field of view (FOV) of an ultrasound probe used with an ultrasound imaging system determines the maximum area or volume that can be acquired in a single frame of ultrasound data. Due to the size of the FOV, during an ultrasound examination, a clinician operating the ultrasound imaging system may miss one or more items of interest, particularly when attempting to examine large organs or large areas of a person's body.
[0003] In order to minimize the possibility of missing one or more items of interest in the ultrasound image data, particularly when the imaged organ is relatively large compared to the FOV, some conventional ultrasound imaging systems are configured to generate a panoramic view. A panoramic view may be generated by multiple ultrasound images acquired at different spatial locations. Due to the constraints imposed by the size of the FOV, ultrasound images may be combined or stitched together to provide a single panoramic view covering a larger area or volume than a single ultrasound image may cover.
[0004] The ultrasound images used to generate the panoramic view can be still frames, each representing the anatomical structure of a person at a single point in time, or each of the ultrasound images in the panoramic view can be a video. For the embodiment in which each of the ultrasound images in the panoramic view is a video, there is a challenge in displaying the video in a way that is easy for clinicians to interpret. Since each individual video is collected at a different time, various patient vital signs (such as heart rate or respiratory rate) may be different between the videos. It is known that each video can be synchronized in time based on parameters such as respiratory rate or heart rate. However, when the ultrasound image is displayed as a video in the panoramic view, this does not solve the basic problem. Each video in the video can be displayed as a repeating loop so that it is played continuously. This means that each video can have a different period. When multiple different video loops each having a different period are combined into a panoramic view and played simultaneously, the result may be visually confusing for the clinician. The mismatch between the various video loops can distract the clinician's attention. The visual changes caused by the mismatch between the video lengths may attract the clinician's attention, making it more difficult to identify clinically important features in the video. Therefore, there is a need for an improved ultrasound imaging system and ultrasound imaging method in which videos in a panoramic view are automatically time-scaled so that all videos take the same amount of time to play. Summary of the invention
[0005] In one embodiment, a method of ultrasound imaging includes acquiring ultrasound image data while moving an ultrasound probe. The method includes automatically identifying a plurality of segments of interest in the ultrasound data. The method includes automatically applying time scaling to at least one of the plurality of segments of interest. The method includes displaying the ultrasound image data as a panoramic view including a plurality of videos, wherein each of the plurality of videos is based on a different segment of interest in the plurality of segments of interest, and wherein each of the plurality of videos in the panoramic view takes the same amount of time to play based on the time scaling.
[0006] In one embodiment, an ultrasound imaging system includes a display device, an ultrasound probe configured to acquire ultrasound image data, and a processor in electronic communication with the ultrasound probe and the display device. The processor is configured to automatically identify multiple segments of interest in the ultrasound image data, and automatically apply time scaling to at least one of the multiple segments of interest. The processor is configured to display the ultrasound image data on the display device as a panoramic view including multiple videos, wherein each of the multiple videos is based on a different segment of interest in the multiple segments of interest, and wherein each of the multiple videos in the panoramic view takes the same amount of time to play based on the time scaling. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The inventive subject matter described herein will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:
[0008] Figure 1 is a schematic diagram of an ultrasound imaging system according to one embodiment;
[0009] Figure 2 shows a thorax according to one example;
[0010] Figure 3 An ultrasound probe according to one embodiment is shown;
[0011] Figure 4 A flow chart illustrating an embodiment of a method according to one embodiment;
[0012] Figure 5A An example of ultrasound image data of lungs and ribs is shown;
[0013] Figure 5B is a graph showing intensity / time according to one embodiment;
[0014] Figure 5C is a graph of centroid / time according to one embodiment; and
[0015] Figure 6 is an illustration of a panoramic image according to one embodiment. DETAILED DESCRIPTION
[0016] At least one technical effect of the inventive subject matter described herein includes displaying multiple segments of interest as a panoramic view including multiple videos, wherein each of the multiple videos in the panoramic view takes the same amount of time to play based on a time scaling applied to at least one of the multiple segments of interest.
[0017] Figure 1 1 is a schematic diagram of an ultrasound imaging system 100 according to one embodiment of the inventive subject matter described herein. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102, which drive elements 104 within an ultrasound probe 106 to transmit pulsed ultrasound signals into a body (not shown). According to one embodiment, the ultrasound probe 106 can be a linear probe, a curved probe, a phased array probe, a linear phased array probe, a curved phased array probe, a two-dimensional matrix array probe, a curved two-dimensional matrix array probe, a mechanical 3D probe, or any other type of ultrasound probe capable of acquiring diagnostic ultrasound images.
[0018] The pulsed ultrasound signal is backscattered from in vivo structures such as blood cells or muscle tissue to produce an echo that returns to the element 104. The echo is converted into an electrical signal by the element 104, and the electrical signal is received by the receiver 108. The electrical signal representing the received echo passes through the receive beamformer 110 that outputs ultrasound image data. The ultrasound probe 106 may contain electronic circuits to perform all or part of the transmit and / or receive beamforming. For example, in other embodiments, the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be located in the ultrasound probe 106. Scanning may include collecting data through the process of transmitting and receiving ultrasound signals. The ultrasound image data collected by the ultrasound probe 106 may include one or more data sets collected with the ultrasound imaging system 100. The user interface 115 can be used to control the operation of the ultrasound imaging system 100, including input for controlling data, for changing scanning or display parameters, etc. The user interface may include, for example, one or more of a touch screen, a keyboard, a touch pad, a trackball, a mouse, one or more knobs, one or more hard keys, and one or more soft keys.
[0019] The ultrasound imaging system 100 also includes a processor 116 that controls the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110. The processor 116 is in electronic communication with the probe 106 via one or more wired and / or wireless connections. The processor 116 can control the ultrasound probe 106 to acquire data. The processor 116 controls which of the elements 104 are active and the shape of the beam emitted from the ultrasound probe 106. The processor 116 is also in electronic communication with a display device 118, and the processor 116 can process the data into an image for display on the display device 118. According to one embodiment, the processor 116 may include one or more central processors. According to other embodiments, the processor 116 may include one or more other electronic components capable of performing processing functions, such as one or more digital signal processors, field programmable gate arrays, graphics boards, and / or integrated circuits. According to other embodiments, the processor 116 may include multiple electronic components capable of performing processing functions. Other embodiments may use two or more separate processors to perform the processing according to Figure 1 The exemplary embodiment shown illustrates functions performed by processor 116. According to another embodiment, processor 116 may also include a complex demodulator (not shown) that demodulates the RF data and generates raw data. In another embodiment, demodulation may be performed earlier in the processing chain.
[0020] The processor 116 is adapted to perform one or more processing operations according to a plurality of selectable ultrasound modalities on the data. The data may be processed in real time during a scanning session as echo signals are received, such as by processing the data without any intentional delay, or while acquiring additional data during the same imaging session of the same person.
[0021] The data may be temporarily stored in a buffer (not shown) during a scanning session and processed in a less-than-real-time manner in real-time or off-line operation. Some embodiments of the inventive subject matter may include multiple processors (not shown) to handle the processing tasks handled by processor 116 according to the exemplary embodiments described above. For example, a first processor may be used to demodulate and extract RF signals, while a second processor may be used to further process the data before displaying an image. It should be understood that other embodiments may use different arrangements of processors.
[0022] The ultrasound imaging system 100 can continuously acquire ultrasound data at a frame rate of, for example, 10 to 30 Hz. Images generated from the data can be refreshed at a similar frame rate. Other embodiments can acquire and display ultrasound data at different rates. For example, some embodiments can acquire ultrasound data at a frame rate less than 10 Hz or greater than 30 Hz.
[0023] A memory 120 is included for storing frames of processed acquisition data. In one embodiment, the memory 120 has sufficient capacity to store at least several seconds of ultrasound image data. The data frames are stored in a manner that facilitates retrieval based on their acquisition order or time. The memory 120 may include any known data storage medium, such as one or more tangible and non-transitory computer-readable storage media (e.g., one or more computer hard drives, disk drives, universal serial bus drives, solid-state drives, etc.).
[0024] In various embodiments of the present invention, the processor 116 can process data through other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, etc.) to form two-dimensional or three-dimensional image data. For example, one or more modules can generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, and combinations thereof. Timing information indicating the time when data was collected in the memory can be recorded. These modules may include, for example, a scan conversion module for performing a scan conversion operation to convert the image volume from beam space coordinates to display space coordinates. The video processor module can read image frames from the memory and display the image in real time when surgery is performed on a person. The video processor module can store images in an image memory, read and display images from the image memory.
[0025] Figure 2 A thorax 200 of a person 204 is shown according to an example. Acquired ultrasound image data (as described herein) may represent a portion of the thorax 200, including lungs 208, a plurality of ribs, and a sternum 210 of the person 204. The plurality of ribs may include a first rib 221, a second rib 222, a third rib 223, a fourth rib 224, a fifth rib 225, a sixth rib 226, a seventh rib 227, an eighth rib 228, a ninth rib 229, and a tenth rib 230. Figure 2 Also shown are a plurality of intercostal spaces between the ribs. For example, the first intercostal space 231, the second intercostal space 232, the third intercostal space 233, the fourth intercostal space 234, and the fifth intercostal space 235 are all located in Figure 2 . The first intercostal space 231 is located between the first rib 221 and the second rib 222; the second intercostal space 232 is located between the second rib 222 and the third rib 223; the third intercostal space 233 is located between the third rib 223 and the fourth rib 224; and the fourth intercostal space 234 is located between the fourth rib 224 and the fifth rib 225. The chest cavity 200 of the person 204 includes additional intercostal spaces; however, these additional intercostal spaces have not yet been Figure 2 When obtaining ultrasound image data, Figure 1 The ultrasound probe 106 shown may remain in contact with the outer surface of the skin of the person 204 and move longitudinally along the person 204 (e.g., in a direction that is closer to parallel to the length or height of the person 204 than one or more other directions). The movement also causes the ultrasound probe 106 to move laterally relative to the plurality of ribs. For example, the ultrasound probe 106 may move in a direction that is parallel or substantially parallel to the sagittal plane 202 of the person 204 (e.g., within ten degrees of parallel, within 15 degrees of parallel, etc.). When the ultrasound probe 106 moves in this direction during the acquisition of ultrasound image data, the ultrasound probe 106 moves transversely or substantially transversely to the direction in which the plurality of ribs extend.
[0026] Figure 3 Shows Figure 1 An embodiment of an ultrasound probe 106 of an ultrasound imaging system 100 is shown. The ultrasound probe 106 may have a retaining element 104 ( Figure 3 The housing 300 of the ultrasound probe 106 interfaces (e.g., contacts) with the person 204 along a facing surface 302 of the housing 300. The facing surface 302 is elongated along a first direction 304 relative to an orthogonal (e.g., vertical) direction 306.
[0027] The ultrasound probe 106 can be moved along the thorax 200 along the outside of the person 204 to acquire ultrasound image data of the lungs 208 of the person 204. In one embodiment, the ultrasound probe 106 is moved transversely to the direction in which the plurality of ribs are elongated. For example, the ultrasound probe 106 can be moved along the outside of the person 204 in a direction that is more parallel to the sagittal plane 202 than perpendicular to the sagittal plane 202.
[0028] The ultrasound probe 106 may be maintained in an orientation in which the elongated direction 304 of the housing 300 of the ultrasound probe 106 is parallel to (or more parallel than perpendicular to) the orientation of multiple ribs of the person 204 while the ultrasound probe 106 moves in a direction substantially parallel to the sagittal plane 202. This orientation of the ultrasound probe 106 may be referred to as a sagittal position or orientation of the ultrasound probe 106. Alternatively, the ultrasound probe 106 may be maintained in an orientation perpendicular to the sagittal orientation. This orientation results in the ultrasound probe 106 being oriented such that the elongated direction 304 of the housing 300 of the probe 106 is perpendicular to (or more perpendicular than parallel to) the multiple ribs of the person 204 while the probe 106 moves in a direction substantially parallel to the sagittal plane 202. This orientation of the probe 106 may be referred to as a lateral position or orientation of the probe 106.
[0029] Figure 4 A flow chart of one embodiment of a method 400 for acquiring ultrasound image data and displaying a panoramic image based on the ultrasound image data is shown. The technical effect of the method 400 is to acquire ultrasound image data of a body (e.g., lungs or another body part) and display a panoramic view that displays multiple time-scaled videos of the ultrasound image data. The method 400 will be described with respect to an embodiment of acquiring ultrasound image data of a human lung, but it should be understood that the method can be used to acquire different anatomical regions according to other embodiments.
[0030] At step 402, ultrasound image data is acquired while moving the ultrasound probe 106. While acquiring the ultrasound image data, the ultrasound probe 106 may remain in contact with the outer surface of the skin of the person 204 and move transversely to the ribs of the person. For example, the ultrasound probe 106 may move in a direction parallel to or substantially parallel to the sagittal plane 202 of the person 204 (e.g., within ten degrees of parallel, within 15 degrees of parallel, etc.). When the ultrasound probe 106 moves in this direction during the acquisition of ultrasound image data, the ultrasound probe 106 moves transversely to or substantially transversely to the direction in which each rib is elongated. Alternatively, the ultrasound probe 106 may move in a direction parallel to the direction in which the ribs are elongated. According to other embodiments, the ultrasound probe 106 may move in a direction other than parallel to the direction in which the ribs are elongated, or move substantially transversely to the direction in which the ribs are elongated. The ultrasound image data may, for example, include a plurality of image frames acquired while the ultrasound probe 106 is moved along the body. Each of the plurality of image frames represents ultrasound image data acquired at a unique point in time. Also, because the ultrasound probe 106 is moving along the body during acquisition of ultrasound image data at step 402 , each of the image frames may also be acquired from a slightly different location on the patient's body.
[0031] In one embodiment, ultrasound image data is acquired while the ultrasound probe 106 is maintained in the same orientation (e.g., only the sagittal orientation or only the transverse orientation) and is moved in a single direction (e.g., only toward the head of the person 204 or only away from the head of the person 204). In another embodiment, ultrasound image data is acquired while the ultrasound probe 106 is maintained in different orientations and is moved in a single direction (e.g., a portion of ultrasound image data is acquired while the probe 106 is maintained in the sagittal orientation, and another portion of ultrasound image data is acquired while the probe 106 is maintained in the transverse orientation). In another embodiment, ultrasound image data is acquired while the ultrasound probe 106 is maintained in the same or different orientations and is moved in two or more different directions (e.g., opposite directions, transverse directions, orthogonal directions, etc.).
[0032] Figure 5A An example of ultrasound image data 500 of the lungs 208 and ribs of a person 204 acquired with the ultrasound probe 106 held in a sagittal orientation is shown. Figure 5A Contains multiple segments of interest and multiple rib shadows. The rib shadows indicate where the passage of the pulsed ultrasound signal is blocked by the ribs. For example, Figure 5AContains a first interesting segment 531, a second interesting segment 532, a third interesting segment 533, and a fourth interesting segment 534. FIG5 also contains a second rib shadow 522, a third rib shadow 523, and a fourth rib shadow 524. The second rib shadow 522 is caused by the second rib 222; the third rib shadow 523 is caused by the third rib 223; and the fourth rib shadow 524 is caused by the fourth rib 224.
[0033] In one embodiment, the processor 116 may be configured to automatically identify segments of interest in the ultrasound image data. The segments of interest may be subsets or portions of the combined image data selected based on characteristics of the image data. The processor 116 may examine characteristics of pixels (or other subsets of the image data) to identify the segments of interest, such as color, intensity, brightness, etc. of pixels in the image data.
[0034] The image data 500 may be a video showing movement of one or more portions of the intercostal spaces 504 and / or rib shadows 506. Such movement may cause one or more features of interest to appear at times and disappear at other times from the ultrasound image data 500. For example, B-lines or other features in the image data 500 that indicate pneumonia infection, air bronchograms, or other injuries may be visible in some but not all image frames.
[0035] The processor 116 may examine the image data acquired by the ultrasound probe 106 to determine how quickly the probe 106 is moving relative to the body of the person 204. For example, the processor 116 may determine that the ultrasound probe 106 is moving when new or additional ultrasound image data of new or different areas of the lungs 208, ribs, etc. are acquired. These new or different areas may contain image data of additional intercostal spaces and / or rib shadows. Figure 4 4. Description of the flowchart of method 400 shown, after acquiring ultrasound data, such as during a scan of the ultrasound probe 106, the method 400 proceeds to step 404. According to some embodiments, the clinician may indicate that all ultrasound image data for the panoramic image has been acquired through input from the user interface 115. According to some embodiments, the clinician may actuate a button, such as a freeze button, to indicate that the acquisition of ultrasound data has been completed. At step 404, the processor 116 identifies segments of interest in the ultrasound image data. According to an exemplary embodiment, it may be desirable to identify segments of interest corresponding to intercostal spaces, such as the first intercostal space 231, the second intercostal space 232, the third intercostal space 233, the fourth intercostal space 234, and the fifth intercostal space 235. The processor 116 may identify segments of interest such as intercostal spaces based on changes in characteristics of the image data, such as changes in intensity (e.g., an increase in intensity when additional intercostal spaces are imaged, or a decrease in brightness when ribs are imaged). Reference will be made to Figure 5BAn exemplary embodiment is described that shows how the processor 116 may use intensity information in ultrasound image data to identify segments of interest.
[0036] Figure 5B It is shown that the Figure 5A Graph 540 of intensity / position of ultrasound image data of image 500 is shown. Graph 540 includes a first portion 541, a second portion 542, a third portion 543, a fourth portion 544, a fifth portion 545, a sixth portion 546, and a seventh portion 547. The first portion 541 represents a portion of ultrasound image data acquired from the first intercostal region 231 and displayed as a first segment of interest 531 on the image 500; the second portion 542 represents a portion of ultrasound image data acquired from the second intercostal region 232 and displayed as a second segment of interest 532 on the image 500; the third portion 543 represents a portion of ultrasound image data acquired from the third intercostal region 233 and displayed as a third segment of interest 533 on the image 500; and the fourth portion 544 represents a portion of ultrasound image data acquired from the fourth intercostal region 234 and displayed as a fourth segment of interest 534 on the image 500. The fifth portion 545 represents the second rib shadow 522 ; the sixth portion 546 represents the third rib shadow 523 ; and the seventh portion 547 represents the fourth rib shadow 524 .
[0037] Figure 5B The difference between the portion of ultrasound image data acquired from the intercostal spaces and the portion of ultrasound image data acquired from the region containing the ribs (i.e., the portion that will be represented as the rib shadows in the image) is shown. The portion of ultrasound image data acquired from the intercostal spaces has a much higher intensity than the ultrasound image data acquired from the region obscured by the ribs. The presence of the ribs blocks all or nearly all of the acoustic energy and results in an intensity of zero or near zero. On image 500, the portion of the ultrasound image data containing the ribs is depicted as rib shadows, such as the second rib shadow 522, the third rib shadow 523, and the fourth rib shadow 524.
[0038] According to one embodiment, the processor 116 may identify the segment of interest based on the intensity of the ultrasound image data. For example, the processor 116 may compare the ultrasound image data with a threshold: if the ultrasound image data is above the threshold, the ultrasound image data has the segment of interest; if the ultrasound image data is below the threshold, the ultrasound image data represents a rib shadow and does not contain the segment of interest. An exemplary threshold 550 is shown in FIG. Figure 5BAccording to an embodiment, the processor 116 may use other image attributes to identify segments of interest from the ultrasound image data. For example, the processor 116 may use information such as Doppler data, spot tracking, or color flow data to identify segments of interest within the ultrasound image data. The processor 116 may also use artificial intelligence, such as by using a neural network to identify segments of interest in the ultrasound image data.
[0039] According to another embodiment, the processor 116 may use a center of mass (COM) calculation, in which the pixel intensity in the image is considered to be the equivalent of mass. The processor 116 may calculate the vertical position of the COM at each location in the direction of translation or scanning of the ultrasound probe 106. At locations where ribs and rib shadows are present, the COM tends to be close to the surface of the ultrasound probe 106 (i.e., at a shallower depth in the image), while for portions of the image with intercostal spaces, the COM tends to be deeper. The processor 116 may determine the locations of the rib shadows and intercostal spaces based on the COM calculation relative to time or distance. For example, according to an exemplary embodiment, the processor 116 may identify the locations of ribs and rib shadows by identifying areas in the image where the COM calculation is relatively high; and the processor 116 may identify the locations of intercostal spaces or pleural areas in the image by identifying areas in the image where the COM calculation is relatively low. For example, Figure 5C An example of a COM graph 560 is shown. The COM graph 560 shows the vertical position of the COM relative to the horizontal position in the image. The processor 116 can identify relative peaks 562 in the COM graph 560 and relative valleys 564 in the COM graph 560. The relative peaks 562 correspond to areas of the image with ribs and rib shadows, while the relative valleys 564 correspond to areas of the image obtained from the intercostal space / pleural region.
[0040] At step 406, the processor 116 identifies the Figure 1 According to one embodiment, the processor 116 may determine the acquisition duration of each segment of interest in the ultrasound image data and use the acquisition duration to determine the target video duration. According to one embodiment, the processor 116 may use intensity information from the ultrasound image data (such as Figure 5B intensity information represented in ) or calculated from COM (such as Figure 5CCOM calculation represented in ) is used to identify the start time and end time of each of the segments of interest. According to one embodiment, the segment of interest corresponds to an intercostal space and displays data acquired from the lungs of a person. As described with respect to step 402, when acquiring ultrasound image data, the ultrasound probe 106 moves relative to the person. For example, the ultrasound probe 106 may be translated in a direction substantially parallel to the longitudinal plane 202 to acquire ultrasound image data. The ultrasound image data includes a plurality of ultrasound image data frames, each frame being acquired at a different time. As long as the ultrasound probe 106 is translated during the acquisition of ultrasound image data, each of the image frames will be acquired from a different spatial position relative to the anatomical structure of the person. According to one embodiment, the processor 116 may use intensity information to identify the start time and end time associated with each segment of interest. The processor 116 may be configured to identify the time associated with the transition from the rib shadow to the intercostal space and the transition from the intercostal space to the rib shadow. For example, as Figure 5B As shown, the intensity associated with the rib shadows is very low, and the intensity associated with the intercostal spaces is much higher. When the probe is translated during acquisition of ultrasound data during step 402, the anatomical structures acquired in each frame of the ultrasound image data are different. The processor 116 may be configured to identify a frame in the ultrasound image data where the ultrasound image data first transitions from a relatively low intensity associated with the rib shadows to a relatively high intensity associated with a particular intercostal space. Likewise, the processor 116 may be configured to identify a frame in the ultrasound image data where the ultrasound image data transitions from a particular intercostal space having a relatively high intensity to an adjacent (next) rib shadow having a relatively low intensity.
[0041] According to another embodiment, the processor 116 may use information from the COM calculation to determine the acquisition duration of each segment of interest in the ultrasound image data. For example, the processor 116 may use the COM calculation to identify ribs and / or rib shadow locations, as previously described with respect to Figure 5C Once the location of the ribs and / or rib shadows is determined, the processor 116 can use this information to determine the acquisition time of the first frame containing a particular intercostal space and the acquisition time of the last frame containing a particular intercostal space. The processor 116 can use the COM information to identify the frames in the ultrasound data that cover each segment of interest.
[0042] Each of the frames identified by the processor 116 may have a timestamp indicating an acquisition time. The processor 116 may use the identified frames and / or associated timestamps to identify a start time and an end time of each segment of interest, which according to an exemplary embodiment is an intercostal space. Based on the identified start time and end time of each segment of interest in the ultrasound image data, the processor 116 may calculate the acquisition duration of each segment of interest by, for example, calculating the amount of time between the start time and the end time of each segment of interest (e.g., each intercostal region).
[0043] According to other embodiments, the processor 116 may rely on a fixed or predetermined target video duration. The processor 116 may, for example, apply time scaling to all or at least some of the segments of interest so that the videos based on the segments of interest will each have a target video duration. This results in a panoramic view in which each video takes the same amount of time to play. In other words, after time scaling, each video in the panoramic view will have a target video duration.
[0044] At step 408, the processor 116 applies time scaling to one or more of the segments of interest. The processor 116 may time scale the ultrasound image data by expanding the one or more segments of interest and / or contracting the one or more segments of interest. For example, the processor 116 may time scale the segments of interest by temporally expanding the ultrasound image data in the segments of interest so that the video takes a longer amount of time to play. Alternatively, the processor 116 may time scale the segments of interest by contracting the ultrasound image data in the segments of interest so that the video takes a shorter amount of time to play. Temporally extending the segments of interest causes the associated video to play at a frame rate slower than the acquisition frame rate; and temporally contracting the segments of interest causes the associated video to play at a frame rate faster than the acquisition frame rate.
[0045] In addition to or in lieu of temporally contracting or expanding the ultrasound image data, the processor 116 may utilize additional techniques to perform temporal scaling of one or more segments of interest. For example, the processor 116 may temporally expand some segments of interest while temporally contracting other segments of interest. If the ultrasound image data associated with a particular segment of interest has significantly fewer frames than the ultrasound image data associated with other segments of interest, the processor 116 may play the video with fewer frames more than once, while only playing the video with a greater number of frames a single time. For example, if the first segment of interest has 10 frames and the second segment of interest has 30 frames, the processor 116 may play the first video three times based on the first segment of interest, while only playing the second video a single time based on the second segment of interest, so as to provide appropriate temporal scaling of the first segment of interest and the second segment of interest.
[0046] According to another embodiment, the processor 116 may generate one or more interpolated frames to ensure that the playback of each video in the panoramic image takes the same amount of time. For example, the processor 116 may insert interpolated frames to adjust the time taken for video playback based on a specific segment of interest. For example, the processor 116 may add interpolated frames to a segment of interest that is shorter than desired to provide appropriate time scaling of the ultrasound image data so that all videos take the same amount of time to play. The processor 116 may be configured to use a combination of the techniques described above to provide time scaling. For example, the processor 116 may be configured to use two or more of the following techniques: temporally extending, temporally contracting, playing the video of the segment of interest more than once, or adding interpolated frames to the video associated with the segment of interest. According to other embodiments, time scaling may include relative time scaling. For example, the processor 116 may play back each video at a speed different from that at which the ultrasound image data is acquired. Therefore, the processor 116 may provide relative time scaling between videos based on the extent to which the playback speed of each video is adjusted relative to other videos in the panoramic view. In other words, time scaling is performed by adjusting the relative playback speed of each video in the panoramic view.
[0047] According to another embodiment, applying time scaling to the segment of interest may include repeating one or more of the frames of the segment of interest when displaying the video based on the segment of interest. For example, assuming that the video is played at the same frame rate, by repeating one or more of the frames in the video, the video may be time scaled to take more time to play.
[0048] The segments of interest may be time scaled to achieve a target video duration. The target video duration may be based on the length of one or more of the segments of interest, the target video duration may be selected by the clinician, or the target video duration may be a predetermined length of time. For example, ultrasound image data associated with one or more segments of interest may be time scaled to match the length of a video associated with a particular segment of interest. According to one embodiment, the processor 116 may be configured to apply time scaling to the ultrasound image data such that each video in the panoramic view takes the same amount of time to play. According to one embodiment, the processor 116 may time scale the ultrasound image data such that each video in the video takes the same amount of time to play as the longest video in the video. According to one embodiment, the processor 116 may time scale the ultrasound image data associated with one or more segments of interest such that each video in the video takes the same amount of time to play as the shortest video in the video. Additional details regarding time scaling ultrasound image data will be discussed below. At step 410, the processor 116 Figure 6 The illustrated panoramic views, such as panoramic view 600, display ultrasound image data. Additional details regarding panoramic view 600 will be provided below.
[0049] As previously described, the panoramic view may be acquired by obtaining different portions of ultrasound image data as the probe 106 moves over the person 204 and then stitching or otherwise combining these different portions of ultrasound image data together to form the panoramic view. The processor 116 may use image information such as brightness values, shape / pattern identification, or other matching algorithms to combine portions of ultrasound image data associated with various segments of interest in the panoramic view 600.
[0050] The panoramic view 600 of ultrasound image data may display or include more image data of the patient than may be available in a single field of view (FOV) of the ultrasound probe 106. For example, the FOV of the ultrasound probe 106 may be much smaller than the panoramic view 600.
[0051] refer to Figure 6, the panoramic view 600 includes a first part 602, a second part 604, a third part 606, a fourth part 608, a fifth part 610, a sixth part 612, a seventh part 614, and an eighth part 616. Videos based on different segments of interest are displayed in each part of the panoramic view 600. For example, a first video based on a first segment of interest may be displayed in the first part 602; a second video based on a second segment of interest may be displayed in the second part 604; a third video based on a third segment of interest may be displayed in the third part 606; a fourth video based on a fourth segment of interest may be displayed in the fourth part 608; a fifth video based on a fifth segment of interest may be displayed in the fifth part 610; a sixth video based on a sixth segment of interest may be displayed in the sixth part 612; a seventh video based on a seventh segment of interest may be displayed in the seventh part 614; and an eighth video based on an eighth segment of interest may be displayed in the eighth part 616. According to Figure 6 In the illustrated embodiment, each of the interest segments may be ultrasound image data acquired from a different intercostal space of a lung region of the patient.
[0052] According to one embodiment, displaying each of the plurality of videos in a panoramic view may include displaying each of the plurality of videos as a repeating loop or a film loop. When displayed as a repeating loop, each video is replayed continuously. In other words, when displayed as a repeating loop, each video transitions from the last frame in the video to the first frame in the video. Time scaling ultrasound image data refers to adjusting the length of time each video takes to play before looping or repeating. The length of time each video takes to play in a panoramic view may also be referred to as the period of the repeating loop. Time scaling may be used so that each video displayed as a repeating loop in a panoramic view has the same period.
[0053] Time scaling of ultrasound image data enables display of a panoramic view, such as panoramic view 600, in which each video takes the same amount of time to play before repeating. The processor 116 can also synchronize all videos in the panoramic view so that each of the videos transitions from the end of the loop to the beginning of the loop at the same time (i.e., loops). A panoramic view containing videos that have been time scaled and synchronized reduces distractions for the user. This allows the clinician to focus on the diagnostically important aspects of the panoramic view, rather than being distracted by the discontinuity caused by videos looping at different time points.
[0054] By automatically performing time scaling on the ultrasound image data, the processor 116 is able to present a panoramic view to the clinician in which each video is of the same length without any additional user input. Performing time scaling additionally has the advantage of presenting a panoramic video that is easier for the clinician to view and interpret because it is less visually distracting to the clinician.
[0055] As used herein, an element or step listed in the singular and beginning with the word "one" or "a kind of" should be understood as not excluding a plurality of said elements or steps, unless such exclusion is explicitly stated. In addition, reference to "an embodiment" is not intended to be interpreted as excluding the existence of other embodiments that also include the described features. In addition, unless explicitly stated to the contrary, an embodiment that "includes," "comprises," or "has" an element or a plurality of elements having a particular attribute may include other such elements that do not have that attribute.
[0056] It should be understood that the above description is intended to be illustrative rather than restrictive. For example, the above embodiments (and / or aspects thereof) can be used in combination with each other. In addition, without departing from the scope of the present invention, many modifications may be made to adapt specific situations or materials to the teachings of the present invention. Although the size and type of materials described herein are intended to define the parameters of the present invention, they are by no means restrictive but exemplary embodiments. After reviewing the above description, many other embodiments will be apparent to those skilled in the art. Therefore, the scope of the present invention should be determined with reference to the appended claims and the full range of equivalents granted by such claims. In the appended claims, the terms "including" and "in..." are used as the colloquial Chinese equivalents of the corresponding terms "including" and "wherein". In addition, in the following claims, the terms "first", "second", and "third", etc. are used only as marks and are not intended to impose numerical requirements on their objects. In addition, the limitations of the following claims are not written in a device-plus-function format, nor are they intended to be interpreted based on 35 U.S.C. §112 (f), unless and until these claim limitations explicitly use the phrase "device for...", followed by a functional statement without other structures.
[0057] This written description uses examples to disclose the invention, including the best mode, and also to enable those skilled in the art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with minor differences from the literal language of the claims.
Claims
1. An ultrasound imaging method, include: Acquiring ultrasound image data while moving the ultrasound probe; automatically identifying a plurality of segments of interest in the ultrasound image data; identifying a target video duration for the panoramic view by determining the acquisition duration of each segment of interest; automatically applying time scaling to at least one segment of interest among the plurality of segments of interest; as well as The ultrasound image data is displayed as a panoramic view including a plurality of videos, wherein each of the plurality of videos is based on a different segment of interest among the plurality of segments of interest, and wherein based on the time scaling, each of the plurality of videos in the panoramic view takes the same target video duration to play.
2. The method of claim 1, wherein the time scaling comprises at least two of the following: extending one of the multiple segments of interest; shrinking one of the multiple segments of interest; playing one of the multiple videos more than once to achieve a target video duration; performing relative time scaling between two of the multiple segments of interest; or generating multiple interpolated frames and inserting the multiple interpolated frames into one of the multiple videos to achieve the target video duration. The method of claim 1 , wherein the temporal scaling comprises extending a segment of interest among the plurality of segments of interest. The method of claim 1 , wherein the temporal scaling comprises shrinking a segment of interest among the plurality of segments of interest. The method of claim 1 , wherein the time scaling comprises playing a video of the plurality of videos more than once to achieve a target video duration. The method of claim 1 , wherein the time scaling comprises performing a relative time scaling between two segments of interest among the plurality of segments of interest. 7 . The method of claim 1 , wherein the temporal scaling comprises generating a plurality of interpolated frames and inserting the plurality of interpolated frames into one of the plurality of videos to achieve a target video duration.
8. The method of claim 1, wherein the ultrasound image data is acquired from the lungs and comprises both a plurality of rib shadows and a plurality of intercostal spaces, wherein each of the plurality of segments of interest represents one of the plurality of intercostal spaces.
9. The method of claim 8, wherein each of the plurality of intercostal spaces is determined by identifying one of: at least one rib shadow among the plurality of rib shadows in the ultrasound image data; at least one rib shadow among the plurality of intercostal spaces in the ultrasound image data; both at least one rib shadow among the plurality of rib shadows in the ultrasound image data and at least one rib space among the plurality of intercostal spaces.
10. The method of claim 8, wherein each of the plurality of intercostal spaces is identified in the ultrasound image data by using a centroid calculation, wherein image intensity is considered an equivalent of mass.
11. An ultrasound imaging system, include: Display devices; an ultrasound probe configured to acquire ultrasound image data; and A processor in electronic communication with the ultrasound probe and the display device, wherein the processor is configured to perform the method according to any one of claims 1-10.
Citation Information
Patent Citations
Automatic analysis of cardiac m-mode views
CN102348417A
Ultrasound image / ultrasound video transmission method and device
CN104055534A
Three dimensional mapping display system for diagnostic ultrasound machines
CN105025799A
Ultrasonic imaging system and method
CN107708571A
Method and system for enhanced visualization and selection of a representative ultrasound image by automatically detecting b lines and scoring images of an ultrasound scan
CN109788939A
