Imaging system and method utilizing real-time inspection completeness monitor
By using artificial intelligence in an ultrasound imaging system to automatically identify and store specified views in image data and display completion progress indicators, the problem of operators having difficulty tracking the required views is solved, improving the efficiency and integrity of the imaging system.
Patent Information
- Application Number
- CN202210607970.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-06-22
- Filing Date
- 2019-06-24
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2039-06-24
AI Technical Summary
In ultrasound imaging systems, it may be difficult for the operator to track and ensure that all the required views are obtained during the imaging session, resulting in the imaging session not being fully concluded and additional imaging sessions are required to interrupt and delay imaging efforts of others.
Ensure that the imaging protocol is completed by using artificial intelligence in an ultrasound imaging system and displaying a graphical completion progress indicator on the display device.
It realizes automatic identification and capture of the views required by the imaging protocol while acquiring image data in real time, reducing the interruption and delay of imaging sessions and improving imaging efficiency.
Smart Images

Figure CN114983468B_ABST
Abstract
Description
[0001] This application is a divisional application of the invention patent application with application number 201910547364.4 and invention name “Imaging system and method using real-time inspection completion monitor”. Technical Field
[0002] The subject matter disclosed herein relates generally to imaging systems. Background Art
[0003] The imaging system can generate image data representative of the imaged body. Some imaging systems are real-time imaging systems that can generate and display images of the body while image data is continuously being acquired. An ultrasound imaging system is an example of such a real-time imaging system. These types of imaging systems are distinguished from other imaging systems that capture image data of the body, where the image data is subsequently displayed to an operator of the imaging system after the imaging session is complete (e.g., all desired image data of the body has been acquired).
[0004] During an imaging session, an operator of an imaging system may wish to obtain certain views of the body being imaged. For example, an operator using an ultrasound imaging system may wish to obtain an apical two-chamber view of the human heart, an apical four-chamber view of the human heart, and an apical long-axis view of the human heart to complete the imaging session. However, the operator may forget which views have been obtained, may forget which views have not yet been obtained, and / or may be distracted by one view and begin to obtain other views that are not required to complete the imaging session (while not obtaining the views that are required to complete the imaging session). This may occur when the operator is required to complete several different imaging protocols on a single person during a single imaging session. When the operator is required to obtain many different views simultaneously under different parameters of multiple different protocols, it may be difficult for the operator to keep track of the parameters and the views that have been obtained. Therefore, the imaging session may terminate without obtaining all desired views of the body. This may require performing additional imaging sessions, which may interrupt and delay the imaging sessions of others. Summary of the invention
[0005] In one embodiment, an ultrasound imaging method includes accessing an imaging protocol for an ultrasound imaging session. The imaging protocol includes one or more specified views to be obtained to complete the imaging protocol. The specified view may refer to a specified imaging view, a specified acquisition view, or a specified acoustic transmission view. The method also includes acquiring image data using an ultrasound imaging system. The image data includes a plurality of different acquired views from a plurality of different positions. The method also includes automatically identifying (using artificial intelligence) a portion of the image data corresponding to one of the one or more specified views of the imaging protocol, automatically storing (in a memory) the portion of the image data corresponding to one of the one or more specified views of the imaging protocol, and displaying (on a display device) a graphical completion progress indicator of the imaging protocol, which indicates one or more of the following: one of the one or more specified views of the imaging protocol has been acquired, or one or more additional specified views of the imaging protocol have not yet been acquired.
[0006] In one embodiment, an ultrasound imaging system includes an ultrasound imaging probe configured to acquire image data during an ultrasound imaging session. The image data includes a plurality of different acquired views from a plurality of different positions. The imaging system also includes one or more processors configured to access an imaging protocol for an ultrasound imaging session. The imaging protocol includes one or more specified views to be acquired to complete the imaging protocol. The one or more processors are also configured to automatically identify a portion of the image data corresponding to one of the one or more specified views of the imaging protocol. The imaging system also includes a memory configured to automatically store the portion of the image data corresponding to one of the one or more specified views of the imaging protocol. The one or more processors are configured to instruct a display device to display a graphical completion progress indicator of the imaging protocol, indicating one or more of the following: one of the one or more specified views of the imaging protocol has been acquired, or one or more additional specified views of the imaging protocol have not yet been acquired.
[0007] In one embodiment, an imaging method includes accessing an imaging protocol for an ultrasound imaging session. The imaging protocol includes one or more specified views to be obtained to complete the imaging protocol. The method also includes acquiring image data using an imaging system. The image data includes multiple different acquired views from multiple different locations. The method also includes automatically identifying (using artificial intelligence) a portion of the image data corresponding to one of the one or more specified views of the imaging protocol, automatically storing (in a memory) the portion of the image data corresponding to one of the one or more specified views of the imaging protocol, and displaying (on a display device) a graphical completion progress indicator of the imaging protocol and the portion of the image data corresponding to one of the one or more specified views of the imaging protocol, the graphical completion progress indicator indicating one or more of the following: one of the one or more specified views of the imaging protocol has been acquired, or one or more additional specified views of the imaging protocol have not yet been acquired. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The inventive subject matter described herein will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:
[0009] Figure 1 is a schematic diagram of an ultrasound imaging system according to one embodiment;
[0010] Figure 2 A flow chart illustrating a method of automatically capturing imaging body views required by an imaging protocol while continuously acquiring image data of the body;
[0011] Figure 3 Shown in Figure 1 An example of a graphical completion progress indicator displayed on a user interface shown in;
[0012] Figure 4 Shown in Figure 1 Additional examples of graphical completion progress indicators displayed on the user interface shown in ; and
[0013] Figure 5 Shown in Figure 1 Additional example of a graphical completion progress indicator displayed on the user interface shown in . DETAILED DESCRIPTION
[0014] The subject matter described herein relates to an imaging system and method for accessing an imaging protocol that specifies which views of a body are to be obtained. The imaging protocol optionally specifies conditions for obtaining these views, such as physiological parameters of the imaged body and / or acquisition parameters (e.g., settings) of the imaging system. The imaging system is controlled to obtain image data, and the imaging system can automatically determine whether any portion of the image data obtained by the imaging system contains a view required by the imaging protocol. The imaging system can optionally determine whether the conditions required by the protocol are also met. The imaging system can automatically determine whether the required view is obtained without the operator of the imaging system recognizing or identifying the view in the image data. The imaging system can automatically save a portion of the image data containing the required view in a memory, and can update a graphical completion progress indicator displayed on a user interface (e.g., an electronic display device). The indicator can indicate which views required by the imaging protocol have been captured and / or which views required by the imaging protocol are still to be captured. This can help train operators on what additional body views are needed.
[0015] At least one technical effect of the subject matter described herein is to automatically identify and capture (e.g., save in memory) portions of image data containing body views required by an imaging protocol while image data is continuously being acquired. Another technical effect is to track and communicate which views required by the protocol have been acquired and / or which additional views required by the protocol remain to be acquired.
[0016] Figure 1 1 is a schematic diagram of an ultrasound imaging system 100 according to one embodiment. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102, which drives an element 104 within a probe 106 to transmit a pulsed ultrasound signal into the body (not shown). According to one embodiment, the probe 106 can be a two-dimensional matrix array probe. However, according to other embodiments, another type of probe capable of acquiring four-dimensional ultrasound data can be used. The four-dimensional ultrasound data may include ultrasound data, such as multiple three-dimensional volumes acquired over a period of time. The four-dimensional ultrasound data may include information showing how the three-dimensional volume changes over time. Alternatively, a 1D array probe or a linear array probe can be used. Optionally, the system 100 may not acquire four-dimensional ultrasound data, but may obtain another type of imaging data, such as a time-motion ultrasound mode, a planar video loop, etc.
[0017] The pulsed ultrasound signal is backscattered from in vivo structures such as blood cells or muscle tissue to produce an echo that returns to the element 104. The echo is converted into an electrical signal or ultrasound data by the element 104, and the electrical signal is received by the receiver 108. The electrical signal representing the received echo passes through the receive beamformer 110 that outputs ultrasound data. The probe 106 may include electronic circuits to perform all or part of the transmit and / or receive beamforming. For example, all or part of the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be located within the probe 106. Scanning may include collecting data through the process of transmitting and receiving ultrasound signals. The data generated by the probe 106 may include one or more data sets collected with the ultrasound imaging system. The user interface 115 may be used to control the operation of the ultrasound imaging system 100, including input for controlling patient data, for changing scanning or display parameters, etc. An example of the user interface 115 may be an electronic display device, such as a monitor, a touch screen, etc. The user interface 115 may optionally include one or more input devices, such as a keyboard, an electronic mouse, a speaker, etc.
[0018] The ultrasound imaging system 100 also includes one or more processors 116 that control the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110. The processor 116 communicates electronically with the probe 106 via one or more wired and / or wireless connections. The processor 116 can control the probe 106 to acquire data. The processor 116 controls which of the elements 104 are active and the shape of the beam emitted from the probe 106. The processor 116 also communicates electronically with the display device 118, and the processor 116 can process the data into an image for display on the display device 118. The processor 116 may include one or more central processing units (CPUs) according to one embodiment. According to other embodiments, the processor 116 may include one or more other electronic components capable of performing processing functions, such as one or more digital signal processors, field programmable gate arrays (FPGAs), graphics boards, and / or integrated circuits. According to other embodiments, the processor 116 may include multiple electronic components capable of performing processing functions. For example, the processor 116 may include two or more electronic components selected from the list of electronic components, including: one or more central processing units, one or more digital signal processors, one or more field programmable gate arrays, and / or one or more graphics boards. According to another embodiment, the processor 116 may also include a complex demodulator (not shown) that demodulates the RF data and generates raw data. In another embodiment, the demodulation may be performed earlier in the processing chain.
[0019] The processor 116 is adapted to perform one or more processing operations according to a plurality of optional ultrasound modalities on the data. In one embodiment, the processor 116 may include one or more graphics processing units (GPUs), or may be communicatively connected to one or more GPUs for performing analysis of image data as described herein. When the echo signal is received, the data can be processed in real time during the scanning session, for example, by processing the data without any intentional delay, or processing the data while collecting additional data during the same imaging session of the same patient. For example, one embodiment can collect images at a real-time rate of 7 to 20 volumes per second. However, the real-time volume rate may depend on the length of time required to collect each volume data for display. Therefore, when a relatively large volume of data is collected, the real-time volume rate may be slower. Some embodiments may have a real-time volume rate significantly faster than twenty volumes per second, while other embodiments may have a real-time volume rate lower than seven volumes per second.
[0020] The data may be temporarily stored in a buffer (not shown) during a scanning session and processed in a less-than-real-time manner in real-time or off-line operation. Some embodiments of the inventive subject matter may include multiple processors (not shown) to handle processing tasks handled by processor 116 according to the exemplary embodiments described above. For example, a first processor may be used to demodulate and extract RF signals, while a second processor may be used to further process the data before displaying an image. It should be understood that other embodiments may use different arrangements of processors.
[0021] The ultrasound imaging system 100 may continuously acquire data at a rate of, for example, ten to 200 Hz. Images generated from the data may be refreshed at a similar frame rate. Other embodiments may acquire and display data at different rates. For example, some embodiments may acquire data at a rate of less than ten Hz or greater than 200 Hz, depending on the volume size and intended application.
[0022] A memory 120 is included for storing processed image data. In one embodiment, the memory 120 has sufficient capacity to store a large amount of ultrasound data for at least a few seconds or minutes. The image data is stored in a manner that facilitates retrieval based on the order or time of its acquisition. The memory 120 may include any known data storage medium, such as one or more tangible and non-transitory computer-readable storage media (e.g., one or more computer hard drives, disk drives, universal serial bus drives, etc.).
[0023] Optionally, one or more embodiments of the subject matter of the present invention described herein can be implemented using contrast agents. When using ultrasound contrast agents including microbubbles, contrast imaging produces enhanced images of anatomical structures and blood flow in the body. After collecting data using contrast agents, image analysis includes separating harmonic components and linear components, enhancing harmonic components, and generating ultrasound images by utilizing the enhanced harmonic components. Separation of harmonic components from the received signal is performed using a suitable filter.
[0024] In various embodiments of the present invention, the processor 116 can process data through other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, etc.) to form two-dimensional or three-dimensional image data. For example, one or more modules can generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, and combinations thereof, etc. Image beams and / or volumes are stored, and timing information indicating the time when data is collected in the memory can be recorded. These modules may include, for example, a scan conversion module for performing a scan conversion operation to convert the image volume from beam space coordinates to display space coordinates. The video processor module can read the image volume from the memory and display the image in real time while the patient is undergoing surgery. The video processor module can store the image in an image memory, read and display the image from the image memory.
[0025] Figure 2 A flow chart of a method 200 for automatically capturing imaging body views required by an imaging protocol while continuously acquiring image data of the body is shown. The method 200 may represent operations performed by the processor 116 to automatically track which views required by an imaging protocol have been acquired and / or which views required by the protocol remain to be captured. At 202, an imaging protocol is accessed. The imaging protocol may be accessed by downloading or otherwise obtaining a copy of the protocol from the memory 120. Alternatively, the protocol may be provided from an operator of the imaging system 100 via the user interface 115.
[0026] An imaging protocol may be a list, table, or other memory structure that indicates or otherwise specifies body views to be obtained to complete the imaging protocol. The views in an imaging protocol may specify the orientation of images of anatomical structures in the body being imaged. For example, an imaging protocol may require obtaining an apical two-chamber view, an apical four-chamber view, an apical long-axis view, and a parasternal short-axis view at the level of the papillary muscles (SAX), etc.
[0027] An imaging protocol may require that some or all of the specified views be obtained in a specified order. For example, an imaging protocol may require that an apical two-chamber view be obtained before an apical four-chamber view, which is obtained before a subcostal view. Alternatively, an imaging protocol does not require that the views be obtained in a specified order. For example, the protocol may require that an aortic valve view, a subcostal view, and a 4CH view be obtained in any order or sequence.
[0028] The imaging protocol may specify views to be obtained to successfully complete the imaging session. The imaging session may begin when the imaging system 100 is activated and the probe 106 begins capturing image data of the body. The imaging session may continue as long as the body continues to be imaged by the probe 106, and may terminate when the probe 106 stops capturing image data of the body and / or the imaging system 100 is deactivated. For example, the imaging session may continue as long as the body continues to be imaged by the imaging system 100.
[0029] Optionally, the imaging protocol may specify prerequisites that must be met before or at the same time as one or more specified views of the acquisition protocol. These conditions may include one or more physiological parameters of the person being imaged. The physiological parameters may include a specified heart rate, a specified heart rate range, a specified respiratory rate, a specified respiratory rate range, etc. For example, the imaging protocol may require that the person's heart rate is at least one hundred beats per minute when obtaining a 4CH view, the person's heart rate is between eighty and ninety beats per minute when obtaining a subcostal view, and the person's respiratory rate is not greater than 20 breaths per minute, etc. These conditions may include one or more acquisition parameters of the imaging system 100. The acquisition parameters may include frame rate, frame rate range, resolution, ultrasound line density, ultrasound line density range, imaging width, imaging depth, ultrasound frequency, ultrasound pulse repetition frequency, ultrasound pulse length, power, etc. The acquisition parameters may include imaging modes, such as ultrasound imaging modes. For example, the acquisition parameters may specify that image data is obtained using ultrasound color flow mode, ultrasound pulse wave Doppler mode, etc.
[0030] The protocol may specify conditions between multiple views. For example, the protocol may require that two or more views (the same or different views) be obtained when the change in a physiological parameter (e.g., heart rate) does not exceed a specified amount (e.g., 20%). If any view is obtained when the physiological parameter changes by more than that amount, the view does not meet the requirements of the protocol.
[0031] If a view required by the agreement is not obtained under or during the conditions required by the agreement, then the view does not satisfy or conform to the requirements of the agreement. However, if the view is obtained under or during the conditions required by the agreement, then the view does satisfy or conform to the requirements of the agreement.
[0032] At 204, the method 200 optionally includes displaying a completion progress indicator. The indicator notifies the operator of the imaging system 100 of the completion of the imaging protocol for the imaging session, and can be displayed and updated (and displayed to the operator) while the imaging system 100 is acquiring image data. Thus, the indicator can provide a real-time completeness monitor for an examination of a person using the imaging system 100.
[0033] The indicator may be a graphical representation of the degree of completion of the imaging protocol and / or how much of the imaging protocol remains to be completed. Alternatively, the indicator may be a graphical representation of protocol views that have been acquired, protocol views that have not yet been acquired, conditions required to capture one or more views, etc. The indicator may be associated with the image data and / or views acquired by the probe 106. Figure 1 The indicator may be displayed together with the image data when the image data is being acquired, for example.
[0034] Figure 3 An example of a graphical completion accuracy indicator 300, 302, 304 is shown shown on the user interface 115. The indicators 300, 302, 304 can be displayed simultaneously with and alongside a partial view of the image data 306. In the illustrated example, the indicator 300 represents how many images in the imaging protocol are specified to be associated with an automated functional imaging (AFI) protocol.
[0035] Indicator 302 indicates how many images have been captured in the imaging protocol that are specified for examination or assessment of myocardial infarction (MI) according to the guidelines established by the American Society of Echocardiography (ASE). For example, one-third of the images required by the guidelines established by the ASE to assess a person for MI have been acquired, and two-thirds of the images required by the guidelines remain to be acquired.
[0036] Indicator 304 indicates how many images have been captured in the imaging protocol that are designated for examining or evaluating mitral valve (MV) prolapse according to the guidelines established by the ASE. For example, half of the images required by the guidelines established by the ASE to evaluate a person for MV prolapse have been acquired, and half of the images required by the guidelines remain to be acquired.
[0037] Figure 4An additional example of a graphical completion progress indicator 400 shown on the user interface 115 is shown. The indicator 400 can be displayed simultaneously with and alongside a partial view 406 of the image data. In the illustrated example, the indicator 400 is a text list of views to be obtained, such as 2CH, 4CH, apical long axis (APLAX) view, parasternal short axis view (PSAX), and parasternal long axis (PLAX) view. Some of the views in the list of the indicator 400 can be shown in different colors, brightness, fonts, etc. to indicate which views have been obtained and which views have not yet been obtained. For example, the indicator 400 shows that the terms 2CH, 4CH, and PLAX have different text (e.g., brighter or different color text) than the terms APLAX and PSAX, thereby indicating that the 2CH, 4CH, and PLAX views required by the imaging protocol have been obtained, and the APLAX and PSAX views required by the imaging protocol have not yet been obtained. The interface 115 can optionally display a smaller version (e.g., thumbnail) of the saved portion 402 of the image data corresponding to the required views of the imaging protocol.
[0038] Figure 5 An additional example of a graphical completion progress indicator 500 shown on the user interface 115 is shown. The indicator 500 can be displayed simultaneously with and alongside the partial view 406 of the image data. In the example shown, the indicator 500 is a textual checklist of views to be obtained, such as 2CH, 4CH, APLAX, PSAX, and PLAX. The indicator 500 is a checklist with a circular graphic icon (e.g., square, circle, etc.) next to each view in the checklist. An "X", check mark, or other symbol can be placed in the icon corresponding to the protocol views that have been obtained. Those views of the protocol that have not yet been obtained can be shown as having no symbol in the graphic icon.
[0039] Returning to the description of the flowchart of method 200, at 206, image data is acquired. An operator of the imaging system 100 may move the probe 106 around the person being imaged to obtain image data of one or more anatomical structures (e.g., organs, blood vessels, bones, etc.) of the person being imaged. The image data may be presented on the user interface 115 while additional image data is being acquired during the same imaging session. For example, the user interface 115 may display a real-time view of the ultrasound image data as the processor 116 acquires and processes the image data. The probe 106 of the imaging system 100 may be moved relative to the subject being imaged so that the image data generated by the processor 116 of the imaging system 100 includes several different views of the imaged subject from different positions. These different views may display different orientations of physiological structures in the body.
[0040] At 208, a determination is made as to whether any portion of the acquired image data includes a specified view required for the imaging protocol. If the image data includes a specified view for the imaging protocol, then at least a portion of the imaging protocol may have been completed. Thus, flow of method 200 may proceed toward 210. However, if the image data acquired thus far does not include a specified view for the imaging protocol, then the imaging protocol has not been completed. Thus, flow of method 200 may return to 206. For example, additional image data may be acquired and examined in a loop to determine when the specified view for the protocol has been obtained.
[0041] In one embodiment, determining whether a portion of the acquired image data includes a specified view of the imaging protocol is performed automatically by the processor 116. For example, the processor 116 may use artificial intelligence or other machine-based learning techniques to automatically determine whether the image data represents a specified view. The artificial intelligence of the processor 116 may be embodied in one or more neural networks formed by at least some of the processors 116.
[0042] The artificial neural network formed by at least some of the processors 116 includes artificial neurons or nodes that receive input image data and perform operations (e.g., functions) on the image data, selectively passing the results of the operations to other neurons. The neural network can be operated to classify frames of acquired image data. For example, the neural network can examine features of a frame of image data and determine whether the frame belongs to one or more different categories of frames, such as an apical two-chamber view, an apical four-chamber view, an apical long-axis view, etc.
[0043] Alternatively, the neural network may identify objects in frames of image data and determine what view the frame represents based on which identified objects appear in the frame. Weight values may be associated with each vector (described below) and neuron in the neural network, and these values constrain how input image data relates to the output of the neuron. The weight values may be determined by an iterative flow of training image data through the neural network. For example, weight values are established during a training phase in which the neural network learns how to recognize a particular object class through typical input image data features of the objects being trained or ground truth images.
[0044] The labeled training images may be image data in which all or most of the pixels or voxels forming the image data are associated with object classes. Object classes are types or categories of objects that appear in the image data. For example, human tissue may be one object class, human bones may be another object class, blood vessels may be another object class, and so on. Pixels or voxels may be labeled (e.g., associated) with the probability that the pixels or voxels represent various object classes by a vector [abcd], where the values of a, b, c, and d indicate the probability that the pixel or voxel represents each different class of object or thing. In a labeled training image, a pixel or voxel labeled [1 0 0 0] may indicate that the probability that the pixel or voxel represents at least a portion of a first class of objects is 100% (e.g., object class human tissue represented by probability a), the probability that the pixel or voxel represents at least a portion of a different second class of objects is zero (e.g., object class human bone represented by probability b), the probability that the pixel or voxel represents at least a portion of a different third class of objects is zero (e.g., object class blood vessels represented by probability c), and the probability that the pixel or voxel represents at least a portion of a different fourth class of objects is zero (e.g., object class representing no body part represented by probability d).
[0045] Artificial neurons in the neural network can examine individual pixels or voxels in the input image data. The processor 116 can use linear classification to calculate scores for different categories of object classes. These scores can indicate the probability that the pixel or voxel represents different classes. For example, the score of a pixel or voxel can be represented as one or more of the above vectors. Each artificial neuron can apply a mathematical function (e.g., an activation function) to the same pixel or voxel, where the functions applied by different neurons affect the functions applied by other neurons, and different neurons apply different weights to different terms in the function than one or more or all other neurons. The application of the function generates a classification score for the pixel or voxel, which can be used to identify objects in the input image data.
[0046] Neurons in the neural network examine features of pixels or voxels, such as intensity, color, etc., to determine scores for various pixels or voxels. After the neural network determines the score vectors for the pixels or voxels, the neural network examines the score vectors for each pixel or voxel and determines which object class has the highest probability for each pixel or voxel, or which object class has a higher probability than the probabilities of one or more or all other object classes for each pixel or voxel. For example, a pixel or voxel with a score vector of [0.6 0.15 0.05 0.2] indicates that the neural network has calculated that the probability that the pixel or voxel represents human tissue is 60%, the probability that the pixel or voxel represents human bone is 15%, the probability that the pixel or voxel represents a blood vessel is 5%, and the probability that the first pixel or voxel does not represent anything is 20% (e.g., not tissue, blood, or blood vessels). The processor 116 can determine that the pixel or voxel represents the object class with the highest or maximum value among these probabilities. For example, since the probability is 60%, the processor can determine that the pixel or voxel represents human tissue. This process may be repeated for several or all other pixels or voxels in the image data.
[0047] Once the neural network identifies possible object classes represented by different pixels or voxels in the image data, the neural network can identify shapes formed by pixels or voxels representing the same object class. These identified shapes can be compared to template shapes for different views associated with different anatomical structures (e.g., stored in memory 120). If the shape of the identified object class (e.g., a blood vessel) more closely matches the shape template associated with a specified view of the blood vessel, the processor 116 can determine that the image data displays a view of the blood vessel.
[0048] At 210, it is determined whether the conditions of the imaging protocol are satisfied. As described above, the imaging protocol may require that one or more physiological parameters and / or acquisition parameters be satisfied before capturing a view of the imaged body (e.g., stored in the memory 120). For example, the imaging protocol may require that the patient's heart rate be within a specified range and that the frame rate of the imaging system 100 be a specified rate. The patient's heart rate or other physiological parameters may be measured by one or more sensors or input by an operator of the imaging system 100. The frame rate or other acquisition parameters may be determined by the processor 116 when the processor 116 controls the operation of the imaging system 100. If the imaging protocol includes one or more physiological conditions, acquisition conditions, or other conditions, and these conditions are not satisfied, the desired view of the imaging protocol may not have been obtained. For example, if the patient's heart rate has not risen to a specified heart rate range required by the imaging protocol, the view of the imaging protocol may not have been obtained. Therefore, the flow of the method 200 may proceed toward 212. However, if the imaging protocol includes one or more physiological conditions, acquisition conditions, or other conditions, and these conditions are satisfied, the desired view of the imaging protocol may be captured and stored. For example, if the patient's heart rate rises to within a specified heart rate range required by the imaging protocol, then a view of the imaging protocol may be obtained. Thus, flow of method 200 may proceed toward 214.
[0049] At 212, physiological and / or acquisition parameters are changed. For example, physiological parameters of the imaged person and / or acquisition parameters of the imaging system 100 may be changed to be within the range or equal to the conditions required by the imaging protocol. With respect to physiological parameter conditions, this may involve the processor 116 instructing the imaged person to increase (or decrease) their heart rate, such as by walking on a treadmill, sitting still, etc. With respect to acquisition parameter conditions, this may involve the processor 116 changing one or more settings of the imaging system 100 to be consistent with the conditions of the imaging protocol. Once one or more conditions of the protocol are met or satisfied, flow of the method 200 may proceed toward 214.
[0050] At 214, a portion of the image data corresponding to one or more specified views of the imaging protocol is stored. The processor 116 can automatically (e.g., without operator intervention) save a digital copy of the portion of the image data containing the views required by the imaging protocol in the memory 120. Alternatively, the processor 116 can instruct the interface 115 to display a notification in response to determining that the specified views of the imaging protocol have been obtained. The operator of the system 100 can then provide input instructing the processor 116 to save the specified views in the memory 120. Because image data can be continuously acquired and / or displayed to the operator while the probe 106 continues to acquire more body views, not all of the image data can be saved in the memory 120. Instead, a subset or portion of the image data having the views required by the imaging protocol can be saved in the memory 120.
[0051] Optionally, the processor 116 can be configured to receive input from the operator to delete or remove the acquired imaging protocol views from the memory 120. For example, although the specified views of the imaging protocol can be obtained, the operator may not be satisfied with the appearance or other characteristics of the acquired views. The operator can provide input (e.g., via the interface 115) indicating the rejection of the acquired views and instructing the processor 116 to delete the acquired views. The processor 116 can then request the operator to capture the views again before determining that the protocol is complete.
[0052] At 216 , the method 200 optionally includes displaying a specified view of the imaging protocol from the acquired image data. For example, as described above, a graphical representation (such as a thumbnail view) of the saved portion 402 of the image data may be displayed on the user interface 115 .
[0053] At 218, the graphical completion progress indicator is updated or otherwise modified. The graphical completion progress indicator may be updated to show that at least one additional view required for the imaging protocol has been captured and saved. For example, the processor 116 may change one or more of the indicators 300, 302, 304 to indicate that a greater percentage of the corresponding imaging protocol has been completed. The manner in which one or more additional terms or words in the indicator 400 are displayed may be changed by the processor 116 to show that more views required for the imaging protocol have been obtained. The processor 116 may check one or more boxes or circles in the checklist of the indicator 500 to show that more views required for the imaging protocol have been obtained.
[0054] At 220, it is determined whether the imaging protocol is complete. The processor 116 may check the automatically saved portion of the views and image data required for the imaging protocol (e.g., at 214) to determine whether all views required for the imaging protocol have been obtained. If all views of the imaging protocol have been obtained, the imaging protocol may be complete. Therefore, the flow of method 200 may proceed toward 222. However, if one or more additional views of the imaging protocol need to be obtained, the imaging protocol may not be completed. Therefore, the flow of method 200 may return to 210. For example, method 200 may return to obtain additional image data to determine whether more views required for the imaging protocol have been obtained.
[0055] Optionally, if an operator of system 100 attempts to end the imaging session before one or more imaging protocols are completed, processor 116 may generate a warning that is displayed on interface 115. For example, if one or more views of an imaging protocol have not yet been acquired but the operator attempts to close or exit interface 115, processor 116 may instruct interface 115 to generate a visual and / or audible warning indicating that the operator still needs to acquire one or more views of the imaging protocol.
[0056] At 222, a notification of the completion of the imaging protocol may be provided to the operator. For example, the processor 116 may change the manner in which the completion progress indicator is displayed on the user interface 115 to indicate that the imaging protocol is complete. Optionally, the method 200 may terminate after 222. Alternatively, the method 200 may return to one or more other operations described above.
[0057] In one embodiment, an ultrasound imaging method includes accessing an imaging protocol for an ultrasound imaging session. The imaging protocol includes one or more specified views to be obtained to complete the imaging protocol. The method also includes acquiring image data using an ultrasound imaging system. The image data includes multiple different acquired views from multiple different locations. The method also includes automatically identifying (using artificial intelligence) a portion of the image data corresponding to one of the one or more specified views of the imaging protocol, automatically storing (in a memory) the portion of the image data corresponding to one of the one or more specified views of the imaging protocol, and displaying (on a display device) a graphical completion progress indicator of the imaging protocol, which indicates one or more of the following: one of the one or more specified views of the imaging protocol has been acquired, or one or more additional specified views of the imaging protocol have not yet been acquired.
[0058] Optionally, the one or more specified views of the imaging protocol include one or more specified orientations of images of anatomical structures in the imaged body.
[0059] Optionally, a neural network is used to automatically identify a portion of the image data corresponding to one of the one or more specified views.
[0060] Optionally, the method further comprises displaying on a display device portions of the image data corresponding to one or more specified views of the imaging protocol on the display device.
[0061] Optionally, a graphical completion progress indicator and portions of the image data corresponding to one or more designated views of the imaging protocol are simultaneously displayed on the display device.
[0062] Optionally, the imaging protocol also includes one or more prerequisites that must be met when acquiring at least one or one or more specified views.
[0063] Optionally, the one or more prerequisites include physiological parameters of the person being imaged.
[0064] Optionally, the physiological parameter is one or more of a specified heart rate, a specified heart rate range, a specified breathing rate, or a specified breathing rate range.
[0065] Optionally, the one or more prerequisites include acquisition parameters of the ultrasound imaging system.
[0066] Optionally, the acquisition parameters include one or more of a specified frame rate, a specified frame rate range, a specified ultrasound line density, or a specified ultrasound line density range.
[0067] In one embodiment, an ultrasound imaging system includes an ultrasound imaging probe configured to acquire image data during an ultrasound imaging session. The image data includes a plurality of different acquired views from a plurality of different positions. The imaging system also includes one or more processors configured to access an imaging protocol for an ultrasound imaging session. The imaging protocol includes one or more specified views to be acquired to complete the imaging protocol. The one or more processors are also configured to automatically identify a portion of the image data corresponding to one of the one or more specified views of the imaging protocol. The imaging system also includes a memory configured to automatically store the portion of the image data corresponding to one of the one or more specified views of the imaging protocol. The one or more processors are configured to instruct a display device to display a graphical completion progress indicator of the imaging protocol, indicating one or more of the following: one of the one or more specified views of the imaging protocol has been acquired, or one or more additional specified views of the imaging protocol have not yet been acquired.
[0068] Optionally, the one or more specified views of the imaging protocol include one or more specified orientations of images of anatomical structures in the imaged body.
[0069] Optionally, the one or more processors are further configured to instruct the display device to display a portion of the image data corresponding to one or more specified views of the imaging protocol on the display device.
[0070] Optionally, the imaging protocol also includes one or more prerequisites that must be met when acquiring at least one or one or more specified views.
[0071] Optionally, the one or more prerequisites include one or more of physiological parameters of the person being imaged or acquisition parameters of the imaging probe.
[0072] In one embodiment, an imaging method includes accessing an imaging protocol for an ultrasound imaging session. The imaging protocol includes one or more specified views to be obtained to complete the imaging protocol. The method also includes acquiring image data using an imaging system. The image data includes multiple different acquired views from multiple different locations. The method also includes automatically identifying (using artificial intelligence) a portion of the image data corresponding to one of the one or more specified views of the imaging protocol, automatically storing (in a memory) the portion of the image data corresponding to one of the one or more specified views of the imaging protocol, and displaying (on a display device) a graphical completion progress indicator of the imaging protocol and the portion of the image data corresponding to one of the one or more specified views of the imaging protocol, the graphical completion progress indicator indicating one or more of the following: one of the one or more specified views of the imaging protocol has been acquired, or one or more additional specified views of the imaging protocol have not yet been acquired.
[0073] Optionally, a neural network is used to automatically identify a portion of the image data corresponding to one of the one or more specified views.
[0074] Optionally, the imaging protocol also includes one or more prerequisites that must be met when acquiring at least one or one or more specified views.
[0075] Optionally, the one or more prerequisites include physiological parameters of the person being imaged.
[0076] Optionally, the one or more prerequisites include acquisition parameters of the ultrasound imaging system.
[0077] As used herein, an element or step described in singular form and beginning with the word "one" or "an" should be understood as not excluding a plurality of said elements or steps, unless such exclusion is explicitly stated. In addition, reference to "an embodiment" is not intended to be interpreted as excluding the existence of other embodiments that also include the described features. In addition, unless explicitly stated to the contrary, an embodiment that "comprises," "includes," or "has" an element or a plurality of elements having a particular attribute may include other such elements that do not have that attribute.
[0078] It should be understood that the above description is intended to be illustrative rather than restrictive. For example, the above embodiments (and / or their various aspects) can be used in combination with each other. In addition, without departing from the scope of the present invention, many modifications can be made to adapt specific situations or materials to the teachings of the present invention. Although the size and type of materials described herein are intended to limit the parameters of the present invention, they are by no means restrictive but exemplary embodiments. After reading the above description, many other embodiments will be obvious to those skilled in the art. Therefore, the scope of the present invention should be determined with reference to the appended claims and the full range of equivalents given by these claims. In the appended claims, the terms "including" and "in..." are used as the colloquial Chinese equivalents of the corresponding terms "including" and "wherein". In addition, in the following claims, the terms "first", "second" and "third", etc. are used only as marks and are not intended to impose numerical requirements on their objects. In addition, the limitations of the following claims are not written in a device-plus-function format, nor are they intended to be interpreted based on 35 U.S.C. §112 (f), unless and until these claim limitations explicitly use the phrase "device for...", followed by a functional statement without other structures.
[0079] This written description uses examples to disclose the invention, including the best mode, and also to enable those skilled in the art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements that are insubstantially different from the literal language of the claims.
Claims
1. An ultrasonic imaging method, comprising: accessing an imaging protocol for an ultrasound imaging session, the imaging protocol comprising a plurality of specified views acquired in any order to complete the imaging protocol, wherein the imaging protocol comprises one or more prerequisites that must be satisfied when acquiring the plurality of specified views, the one or more prerequisites comprising at least one of an acquisition parameter or a physiological parameter; displaying a graphical completion progress indicator of the imaging protocol on a display device, the indicator indicating at least one of: how much of the imaging protocol has been completed, how much of the imaging protocol remains to be completed, one or more of the plurality of specified views have been acquired, or one or more additional specified views of the imaging protocol have not yet been acquired; Acquiring image data using an ultrasound imaging system; automatically identifying, using artificial intelligence, a portion of the image data corresponding to one of the plurality of specified views of the imaging protocol; automatically determining whether one or more prerequisites are satisfied when acquiring a portion of the image data; automatically rejecting a portion of the image data if the one or more preconditions are not met when acquiring the portion of the image data; and If the one or more prerequisites are satisfied while acquiring the portion of the image data, the portion of the image data is automatically stored and a graphical completion progress indicator displayed on the display device is updated to show that one of a plurality of specified views of the imaging protocol has been stored. 2 . The ultrasound imaging method of claim 1 , further comprising displaying one or more additional graphical completion progress indicators simultaneously with displaying the graphical completion progress indicator on the display device. 3 .
3. The ultrasound imaging method of claim 1, wherein the one or more preconditions include acquisition parameters, the method further comprising automatically changing the acquisition parameters in response to rejecting a portion of the image data. 4 . The ultrasound imaging method according to claim 3 , wherein the acquisition parameters include one or more of a specified frame rate, a specified frame rate range, a specified ultrasound line density, or a specified ultrasound line density range.
5. The ultrasound imaging method of claim 1, wherein the one or more preconditions include a physiological parameter, the method further comprising automatically providing instructions to the imaged person to change the physiological parameter in response to rejecting a portion of the image data. 6 . The ultrasound imaging method according to claim 5 , wherein the physiological parameters include one or more of a specified heart rate, a specified heart rate range, a specified respiratory rate, or a specified respiratory rate range. 7 . The ultrasound imaging method of claim 1 , wherein the protocol specifies the same prerequisite in two or more of the plurality of specified views.
8. The ultrasound imaging method according to claim 1, wherein the one or more prerequisites include acquisition parameters and physiological parameters.
9. The ultrasound imaging method of claim 1, wherein the imaging protocol comprises a first prerequisite for a first one of a plurality of designated views and a second prerequisite for a second one of the plurality of views, wherein the second prerequisite is different from the first prerequisite.
10. The ultrasound imaging method according to claim 1, further comprising: determining that an operator of the imaging system terminated an imaging session before completion of an imaging protocol; as well as A warning is displayed on a display device notifying the operator that one or more specified views of the imaging protocol have not been acquired.
11. An ultrasound imaging system, comprising: an ultrasound imaging probe configured to acquire image data during an ultrasound imaging session, the image data comprising a plurality of different acquired views from a plurality of different positions; Display devices; as well as A processor, wherein the processor is configured to execute the method according to any one of claims 1 to 10.
12. A non-transitory computer-readable medium storing a computer program, wherein the computer program has at least one code segment, and the at least one code segment can be executed by a machine to enable the machine to perform the steps of the method according to any one of claims 1 to 10.
