Increasing image quality in ultrasound images due to bad face rendering

Abstract

The disclosure relates to increasing image quality in ultrasound images due to poor face rendering. Methods and systems are provided for identifying a median sagittal plane (MSP) (310) in a medical image. In one example, a method for an image processing system (202) includes obtaining an input volume (302) acquired with an ultrasound imaging system (100); logging the input volume (302) as input to a segmentation model (304, 418) trained to output a segmentation mask identifying the segmented fetal head (306, 420), the first segmented trajectory (802), and the second segmented trajectory (804); identifying a median sagittal plane (MSP) (310) of the fetal head using the segmented first track (502) and the segmented second track (504); in response to determining that the MSP (310) is not the acquired plane, visually displaying the MSP (310) on the segmented fetal head (306, 420); alerting the user of undesirable face rendering or likelihood of undesirable face rendering and prompting the user to redirect the ultrasound probe (106, 236) and reacquire the input volume (302); and displaying the input volume (302) and / or storing the input volume (302) in the memory (120).

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to ultrasound imaging, and more particularly, to improving image quality of ultrasound imaging. Background Art

[0002] Medical ultrasound is an imaging modality that uses ultrasound to detect the internal structures of a patient's body and produce corresponding images. For example, an ultrasound probe including multiple transducer elements emits ultrasound pulses, which are reflected or transmitted back, refracted or absorbed by structures in the body. The ultrasound probe then receives the reflected echoes, which are processed into images. For example, medical imaging devices such as ultrasound imaging devices can be used to obtain images of a patient's heart, uterus, liver, lungs, and various other anatomical regions. Ultrasound images of internal structures can be saved for later analysis by clinicians to aid in diagnosis and / or displayed on a display device in real time or near real time. However, ultrasound images of a fetal head may easily exhibit poor image quality due to poor facial rendering. Summary of the invention

[0003] In one example, a method includes: obtaining an input volume acquired using an ultrasound imaging system; entering the input volume as input to a model that is trained to output a mask that identifies a fetal head, a first track, and a second track; identifying a midsagittal plane (MSP) of the fetal head using the first track and the second track; in response to determining that the MSP is not a plane of acquisition, visually displaying the MSP on the fetal head; alerting a user of poor facial rendering or the possibility of poor facial rendering; and prompting the user to re-orient the ultrasound probe and re-acquire the input volume; and displaying the input volume and / or saving the input volume in a memory.

[0004] It should be understood that the above brief description is provided to introduce in a simplified form selected concepts that are further described in the detailed description. It is not meant to identify key features or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Various aspects of the present disclosure may be better understood by reading the following detailed description and referring to the accompanying drawings, in which:

[0006] Figure 1 A block diagram of an exemplary embodiment of an ultrasound system is shown;

[0007] Figure 2 is a schematic diagram illustrating a system for generating an ultrasound image according to an exemplary embodiment;

[0008] Figure 3 is an example process for identifying a midsagittal plane (MSP) of a fetal head of a subject and providing feedback to a user;

[0009] Figure 4 is an example procedure for training a segmentation model to identify the fetal head and tracks;

[0010] Figure 5 is an example image of the midsagittal plane (MSP) passing between the orbits;

[0011] Figure 6 is a flow chart illustrating a method for identifying a midsagittal plane (MSP) of a fetal head of a subject and providing feedback to a user;

[0012] Figure 7 is a flow chart illustrating a method for training a segmentation model to identify a fetal head and track;

[0013] Figure 8 are example images of a first segmented track and a second segmented track constructed from more than two identified tracks;

[0014] Fig. 9 is the first example image of the midsagittal plane (MSP) relative to the canonical plane;

[0015] Fig.10 is a second example image of the midsagittal plane (MSP) relative to the canonical plane;

[0016] Fig.11 is the third example image of the midsagittal plane (MSP) relative to the canonical plane;

[0017] Fig.12 is an example image of visualization of the midsagittal plane (MSP) when the fetal head is in a first orientation;

[0018] Fig.13 is an example image of visualization of the midsagittal plane (MSP) when the fetal head is in a second orientation;

[0019] Fig.14 is an example image of visualization of the midsagittal plane (MSP) when the fetal head is in a third orientation; and

[0020] Fig.15 are example images of visualization of the midsagittal plane (MSP) when the fetal head is in the fourth orientation. DETAILED DESCRIPTION

[0021] Medical ultrasound imaging generally includes placing an ultrasound probe including one or more transducer elements onto an imaging subject (such as a patient) at the location of a target anatomical feature (e.g., a fetal head, etc.). Images are acquired by the ultrasound probe and displayed on a display device in real time or near real time (e.g., the image is displayed as soon as it is generated without intentional delay). An operator of the ultrasound probe can view the image and adjust various acquisition parameters and / or positions of the ultrasound probe in order to obtain a high quality image of the target anatomical feature (e.g., a fetal head).

[0022] In order to achieve an acceptable facial rendering of the fetal head, and more specifically, the fetal face, the anterior surface of the fetal head is captured at high resolution. By identifying the mid-sagittal plane (MSP), an image can be acquired in which the anterior surface of the fetal head is captured at high resolution. The MSP is perpendicular to the anterior or posterior surface of the head. In addition, the MSP is the axis of symmetry of the fetal face that passes between the orbits. Therefore, this makes the MSP a reliable anatomical landmark for determining the orientation of the fetal head. Therefore, it is desired that the MSP is the plane of acquisition rather than the tilted plane / reconstruction plane.

[0023] However, changing the position of the ultrasound probe to identify the MSP of the fetal head to acquire the best image (e.g., an image of desired quality) may be challenging and may prove to be time consuming. Often, the MSP is not correctly identified by the operator, resulting in poor image quality due to poor fetal facial rendering. By identifying the MSP of the fetal head based on the output from the segmentation model, the frequency of obtaining images with poor fetal facial rendering can be reduced. Although this example describes a segmentation method for identifying tracks, alternative methods may include general detection and / or positioning. The segmentation model may output a segmented fetal head and a segmented track. The segmented track may be used to analytically determine the MSP and visualize the MSP on the segmented fetal head. Visualizing the MSP on the segmented fetal head may enable the MSP to be classified as a plane of acquisition or an inclined plane / reconstruction plane. In response to the MSP being an inclined plane / reconstruction plane, feedback and recommendations may be provided to the user. More specifically, it may be suggested that the user redirect the probe and reacquire the volume when poor facial rendering is expected.

[0024] Figure 1 An example ultrasound system is shown in FIG. 1 , which includes an ultrasound probe, a display device, and an image processing system. Through the ultrasound probe, an ultrasound image can be acquired and displayed on the display device. Figure 2 As shown, the image processing system includes a segmentation module in a non-transitory memory, which may include code that, when executed, segments an input volume acquired using an ultrasound imaging system. Figure 3An example process for identifying the midsagittal plane is shown. Figure 4 An example process for training a segmentation model to identify the fetal head and track is shown. Figure 5 An example image depicting the midsagittal plane positioned between the orbits is shown in . Figure 6 A method for identifying a mid-sagittal plane (MSP) based on output from a segmentation model and providing feedback in response to the MSP being an oblique plane is shown. Figure 7 A method for training a segmentation model to identify the fetal head and track of an object is shown. Figure 9-11 Example images of the midsagittal plane (MSP) relative to the canonical plane are shown. Figure 12-Figure 15 Example images of the midsagittal plane are shown when the fetal head is in various orientations.

[0025] See now Figure 1 , which shows a schematic diagram of an ultrasound imaging system 100 according to an embodiment of the present disclosure. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102, which drives an element (e.g., a transducer element) 104 within a transducer array (referred to herein as a probe 106) to transmit a pulsed ultrasound signal (referred to herein as a transmit pulse) into a body (not shown). The probe 106 can be a one-dimensional transducer array probe, or the probe 106 can be a two-dimensional matrix transducer array probe. As further explained below, the transducer element 104 can be made of a piezoelectric material. When a voltage is applied to a piezoelectric crystal, the crystal physically expands and contracts, thereby emitting an ultrasonic spherical wave. In this way, the transducer element 104 can convert an electronic transmit signal into an acoustic transmit beam.

[0026] After the elements 104 of the probe 106 transmit the pulsed ultrasound signals into the body (of the patient), the pulsed ultrasound signals are backscattered from structures inside the body (such as blood cells or muscle tissue) to produce echoes that return to the elements 104. The echoes are converted into electrical signals or ultrasound data by the elements 104, and the electrical signals are received by the receiver 108. The electrical signals representing the received echoes pass through the receive beamformer 110, which outputs the ultrasound data. In addition, the transducer elements 104 may generate one or more ultrasound pulses based on the received echoes to form one or more transmit beams.

[0027] According to some embodiments, the probe 106 may contain electronic circuits to perform all or part of the transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be located within the probe 106. In the present disclosure, the term "scan" or "scanning" may also be used to refer to the process of transmitting and receiving ultrasound signals to collect data. In the present disclosure, the term "data" may be used to refer to one or more data sets collected by the ultrasound imaging system. In one embodiment, the data collected via the ultrasound imaging system 100 can be used to train a machine learning model. The user interface 115 can be used to control the operation of the ultrasound imaging system 100, including input for controlling patient data (e.g., patient medical history), for changing scanning or display parameters, for starting a probe repolarization sequence, etc. The user interface 115 may include one or more of the following: a rotating element, a mouse, a keyboard, a trackball, a hard key linked to a specific action, a soft key that can be configured to control different functions, and / or a graphical user interface displayed on a display device 118.

[0028] The ultrasound imaging system 100 also includes a processor 116 that controls the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110. The processor 116 is in electronic communication (e.g., communicatively connected) with the probe 106. For purposes of this disclosure, the term "electronic communication" may be defined to include both wired communication and wireless communication.

[0029] The processor 116 can control the probe 106 to collect data according to instructions stored in the processor's memory and / or the memory 120. The processor 116 can control which control elements in the element 104 are active and the shape of the beam emitted from the probe 106. The processor 116 also communicates electronically with the display device 118, and the processor 116 can process the data (e.g., ultrasound data) into an image for display on the display device 118. According to an embodiment, the processor 116 may include a central processing unit (CPU). According to other embodiments, the processor 116 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 116 may include multiple electronic components capable of performing processing functions. For example, the processor 116 may include two or more electronic components selected from a list of electronic components, which include: a central processing unit, a digital signal processor, a field programmable gate array, and a graphics board. According to another embodiment, the processor 116 may also include a composite demodulator (not shown) that demodulates the RF data and generates raw data. In another embodiment, the demodulation may be performed earlier in the processing chain.

[0030] The processor 116 is adapted to perform one or more processing operations according to a plurality of selectable ultrasound modalities on the data. In one example, the data may be processed in real time during a scanning session because the echo signals are received by the receiver 108 and transmitted to the processor 116. For the purposes of this disclosure, the term "real time" is defined as including a program executed without any intentional delay. For example, an embodiment may acquire images at a real-time frame rate of 7 frames / second to 20 frames / second. The ultrasound imaging system 100 may acquire 2D data of one or more planes at a significantly faster rate. However, it should be understood that the real-time frame rate may depend on the length of time it takes to acquire each frame of data for display. Therefore, when a relatively large amount of data is acquired, the real-time frame rate may be slower. Therefore, some embodiments may have a real-time frame rate significantly faster than 20 frames / second, while other embodiments may have a real-time frame rate lower than 7 frames / second.

[0031] The data may be temporarily stored in a buffer (not shown) during a scanning session and processed in less than real time in real time or off-line operation. In some embodiments, multiple processors (not shown) may be included to handle processing tasks handled by processor 116 according to the exemplary embodiments described above. For example, a first processor may be utilized to demodulate and extract the RF signal before displaying an image, while a second processor may be utilized to further process the data (e.g., by augmenting the data as further described herein). It should be appreciated that other embodiments may use different processor arrangements.

[0032] The ultrasound imaging system 100 can continuously acquire data at a frame rate of, for example, 10 Hz to 30 Hz (e.g., 10 to 30 frames per second). The image generated based on the data can be refreshed at a similar frame rate on the display device 118. Other embodiments can acquire and display data at different rates. For example, depending on the size of the frame and the intended application, some embodiments can acquire data at a frame rate less than 10 Hz or greater than 30 Hz. A memory 120 is included to store frames of processed acquisition data. In an exemplary embodiment, the memory 120 has sufficient capacity to store at least a few seconds of ultrasound data frames. The data frames are stored in a manner that is easy to retrieve according to their acquisition order or time. The memory 120 may include any known data storage medium.

[0033] In various embodiments of the present disclosure, the processor 116 may process data through different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, etc.) to form spatiotemporal 2D, 3D, or 4D data (e.g., where time is included in one dimension). For example, one or more modules may generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, and combinations thereof, etc. As an example, one or more modules may process color Doppler data, which may include traditional color blood flow Doppler, power Doppler, HD flow, etc. The image lines and / or frames are stored in the memory and may include timing information indicating the time when the image lines and / or frames are stored in the memory. These modules may include, for example, a scan conversion module that performs a scan conversion operation to convert the acquired image from beam space coordinates to display space coordinates. A video processor module may be provided that reads the acquired image from the memory and displays the image in real time while a procedure (e.g., ultrasound imaging) is performed on the patient. The video processor module may include a separate image memory, and the ultrasound image may be written to the image memory for reading and display by the display device 118 .

[0034] In various embodiments of the present disclosure, one or more components of the ultrasound imaging system 100 may be included in a portable handheld ultrasound imaging device. For example, the display device 118 and the user interface 115 may be integrated into the external surface of the handheld ultrasound imaging device, which may further contain a processor 116 and a memory 120. The probe 106 may include a handheld probe that electronically communicates with the handheld ultrasound imaging device to collect raw ultrasound data. The transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be included in the same or different parts of the ultrasound imaging system 100. For example, the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be included in a handheld ultrasound imaging device, a probe, and a combination thereof.

[0035] After performing an ultrasound scan, a two-dimensional data block including scan lines and their samples is generated for each row of transducers included in the ultrasound probe (e.g., one data block for a 1D probe, or n data blocks for a 2D probe with n rows of transducers). After applying the back-end filter, a process called scan conversion is performed to transform the two-dimensional data block into a displayable bitmap image with additional scanning information (such as depth, angle of each scan line, etc.). During scan conversion, interpolation techniques are applied to fill in missing holes (e.g., pixels) in the resulting image. These missing pixels occur because each element of the two-dimensional block should generally cover many pixels in the resulting image. For example, in current ultrasound imaging systems, bicubic interpolation is applied, which utilizes adjacent elements of the two-dimensional block. Therefore, if the two-dimensional block is relatively small compared to the size of the bitmap image, the scan-converted image will include areas of poor or low resolution, especially for areas with greater depth.

[0036] The ultrasound data acquired by the ultrasound imaging system 100 may be further processed at various stages before, during, or after image formation. In some embodiments, as described in more detail below, the ultrasound data generated by the ultrasound imaging system 100 may be transmitted to an image processing system, such as the following reference system. Figure 2 The image processing system described herein, in which ultrasound data can be processed by one or more data-driven models. For example, an ultrasound image and a corresponding reference truth image can be used to train a neural network model to segment an input volume of a region of interest (ROI) generated from ultrasound data. More specifically, the ROI can be a fetal head. The segmentation output can be used to identify the midsagittal plane (MSP) of the fetal head. As used herein, a reference truth output refers to an expected or "correct" output based on a given input into a machine learning model.

[0037] Although described herein as separate systems, it should be understood that in some embodiments, the ultrasound imaging system 100 includes an image processing system. In other embodiments, the ultrasound imaging system 100 and the image processing system may include separate devices. In some embodiments, the images generated by the ultrasound imaging system 100 may be used as a training data set for training one or more machine learning models, where the machine learning model may be used to perform one or more steps of ultrasound image processing as described below.

[0038] refer to Figure 2, block diagram 200 shows an image processing system 202 according to an embodiment. In some embodiments, the image processing system 202 is incorporated into the ultrasound imaging system 100. For example, the image processing system 202 may be provided in the ultrasound imaging system 100 as a processor 116 and a memory 120. In some embodiments, at least a portion of the image processing system 202 is disposed at a device (e.g., an edge device, a server, etc.) that is communicatively coupled to the ultrasound imaging system via a wired connection and / or a wireless connection. In some embodiments, at least a portion of the image processing system 202 is disposed at a separate device (e.g., a workstation) that can receive images from the ultrasound imaging system or from a storage device that stores images / data generated by the ultrasound imaging system. The image processing system 202 may be operably / communicatively coupled to a user input device 232 and a display device 234. At least in some examples, the user input device 232 may include a user interface 115 of the ultrasound imaging system 100, and the display device 234 may include a display device 118 of the ultrasound imaging system 100. The image processing system 202 may also be operably / communicatively coupled to an ultrasound probe 236 .

[0039] The image processing system 202 includes a processor 204 configured to execute machine-readable instructions stored in a non-transitory memory 206. The processor 204 may be a single-core or multi-core processor, and the program executed thereon may be configured for parallel processing or distributed processing. In some embodiments, the processor 204 may optionally include separate components distributed on two or more devices, which may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 204 may be virtualized and performed by a remotely accessible networked computing device configured in a cloud computing configuration.

[0040] The non-transitory memory 206 may store a segmentation module 208, a training module 210, an inference module 212, an MSP determination module 214, and an image database 216. The segmentation module 208 may include at least a deep learning (DL) model and instructions for implementing the DL model to segment a fetal head and orbit from an ultrasound image (e.g., a 3D volume), as described in more detail below. The segmentation module 208 may include various types of models, including trained and / or untrained neural networks such as CNNs, statistical models, or other models, and may also include various data or metadata related to one or more models stored therein.

[0041] The non-transitory memory 206 may also store a training module 210, which may include instructions for training one or more of the models stored in the segmentation module 208. The training module 210 may include instructions that, when executed by the processor 204, cause the image processing system 202 to perform one or more steps of the method 700 for training a neural network model, which will be referred to below. Figure 7 In some embodiments, the training module 210 may include instructions for implementing one or more gradient descent algorithms, applying one or more loss functions, and / or training routines for adjusting parameters of one or more neural networks of the segmentation module 208. The training module 210 may include a training data set for one or more models of the segmentation module 208.

[0042] The non-transitory memory 206 stores the inference module 212. The inference module 212 may include a method for deploying the following with respect to Figure 6 Specifically, the inference module 212 may include instructions that, when executed by the processor 204, cause the image processing system 202 to perform one or more of the steps of the method 700, as discussed in further detail below.

[0043] The non-transitory memory stores the MSP determination module 214. The MSP determination module 214 may include instructions for analytically determining a mid-sagittal plane (MSP) that is perpendicular to a vector between two tracks of the fetal head.

[0044] The non-transitory memory 206 also stores an image database 216. The image database 216 may include, for example, ultrasound images acquired via an ultrasound probe. For example, the image database 216 may store images acquired via a handheld ultrasound probe placed on a subject's body. The image database 216 may include ultrasound images used in one or more training sets for training one or more neural networks of the segmentation module 208.

[0045] In some embodiments, non-transitory memory 206 may include components disposed on two or more devices that may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of non-transitory memory 206 may include a remotely accessible networked storage device configured in a cloud computing configuration.

[0046] The user input device 232 may include one or more of a touch screen, keyboard, mouse, trackpad, motion sensing camera, or other device configured to enable a user to interact with and manipulate data within the image processing system 202. In one example, the user input device 232 may enable a user to select images for training a machine learning model, or for further processing using a trained machine learning model.

[0047] The display device 234 may include one or more display devices utilizing almost any type of technology. In some embodiments, the display device 234 may include a computer monitor and may display ultrasound images. The display device 234 may be combined with the processor 204, the non-transitory memory 206, and / or the user input device 232 in a common housing, or may be a peripheral display device and may include a monitor, a touch screen, a projector, or other display devices known in the art that may enable a user to view ultrasound images generated by the ultrasound imaging system and / or interact with various data stored in the non-transitory memory 206.

[0048] It should be understood that Figure 2 The image processing system 202 shown is for illustration and not for limitation. Another suitable image processing system may include more, fewer, or different components.

[0049] Steering Figure 3 , shows a process 300 for identifying a midsagittal plane (MSP) of an input volume 302, which may be performed by Figure 2 The process 300 may include obtaining an input volume 302. The input volumes 302 may be generated from ultrasound data collected during a scanning session, and thus each input volume 302 may be mapped from a 2D slice generated from processed ultrasound data obtained when a user operates the 3D probe in 2D mode. The input volume 302 may include a fetal head and orbit of one or more objects in various orientations relative to three canonical planes (e.g., an axial plane, a sagittal plane, and a coronal plane).

[0050] Process 300 may include entering an input volume 302 into a segmentation model 304 (which may be the same as above with respect to Figure 2 304) to segment the input volume into a segmented fetal head 306 and a segmented track 308. In some embodiments, the segmentation model 304 can be a deep learning (DL) model. In this way, the segmentation model 304 can output a mask for the fetal head and track included in the input volume.

[0051] Process 300 may include determining a midsagittal plane (MSP) 310. The MSP 310 may be analytically determined based on the segmentation trajectory 308. After determining the MSP 310, process 300 may include performing an MSP visualization 312 on the segmented fetal head 306, which displays the MSP on the segmented fetal head. The segmented fetal head 306 is oriented relative to one of the three canonical planes. In this manner, the visualized MSP may be determined as a plane of acquisition or an inclined plane / reconstruction plane. The MSP visualization 312 may be displayed on a display device 314. In response to the MSP being an inclined plane, process 300 may include outputting a user warning 316 on the display device 314, which explains the cause and / or possibility of poor facial rendering of the fetal head. As an example, the cause or possibility of poor rendering may be due to the MSP being an inclined plane rather than an acquired plane. The process 300 may also include outputting a re-acquisition prompt 318 on the display device 314, the re-acquisition prompt advising the user to re-acquire the input volume 302 because the current input volume may result in an image with poor facial rendering of the fetal head. In this way, visualization of the fetal face of the subject may be increased without storing ultrasound data that generates an inappropriate 3D volume with poor facial rendering and does not enable a medical professional to monitor the development of the fetal head.

[0052] Go to Figure 4 , shows a process 400 for training a segmentation model 418. Figure 7 One or more operations described in more detail may train the segmentation model 418 to recognize Figure 1 The process 400 may be performed by one or more computing systems such as Figure 2 The image processing system 202 of FIG. 1 is implemented to train the segmentation model 418 to segment an input volume of an object (such as an input volume) into a fetal head and a track. Once trained, the segmentation model 418 can be used according to the following reference Figure 6 and Figure 7 One or more operations described in more detail may use the segmentation model 418 to analyze the image using an ultrasound imaging system (e.g., Figure 1 An input volume of an object acquired by an ultrasound imaging system 100 is segmented.

[0053] The process 400 includes obtaining an input volume 402 of one or more objects. For example, ultrasound data of a fetal head of each of the one or more objects can be obtained during a scanning session, where a user (e.g., a sonographer) adjusts the position of a 3D probe to identify a midsagittal plane (MSP) of the fetal head. When the 3D probe is stationary and operating in 2D mode, the user acquires ultrasound data, which is processed to generate 2D slices. The 2D slices are mapped to a 3D volume, which can be used as the input volume 402. The input volume 402 can include one or more objects in various orientations relative to a canonical plane, where each canonical plane is perpendicular to the other canonical planes.

[0054] As an example, in a first input volume in input volume 402, the MSP of the first input volume can be aligned with one of the canonical planes. In another example, in a second input volume in input volume 402, the MSP of the second input volume can be aligned with a canonical plane different from the first input volume. In yet another example, in a third input volume in input volume 402, the MSP of the third input volume can be an inclined plane, wherein the inclined plane is offset from one of the first canonical planes by an angle (e.g., 30°). In other embodiments in which the MSP is an inclined plane, the MSP can be offset from each of the canonical planes by a predetermined angle. For illustration, the MSP can be offset from the first canonical plane by 15°, from the second canonical plane by 25°, and from the third canonical plane by 17°.

[0055] Process 400 includes: using a data set generator 404 to generate a plurality of training data pairs. The plurality of training data pairs can be stored in a training module 406. The training module 406 can be used with Figure 2 The training module 210 of the image processing system 202 may be the same or similar. The plurality of training data pairs may be divided into training pairs 408 and test pairs 410. Each training pair of the training pairs 408 and the test pairs 410 may include an annotated input volume and a ground truth segmentation including a fetal head and a track.

[0056] After each pair is generated, each pair may be assigned to training pairs 408 or test pairs 410. In one embodiment, the pairs may be randomly assigned to training pairs 408 or test pairs 410 in a predetermined ratio (e.g., 90% to training pairs / 10% to test pairs, or 85% to training pairs / 15% to test pairs). It should be understood that the examples provided herein are for illustrative purposes, and the pairs may be assigned to the training pair 408 dataset or the test pair 410 dataset via different processes and / or in different ratios without departing from the scope of the present disclosure.

[0057] A plurality of training pairs 408 and test pairs 410 may be selected to ensure that sufficient training data is available to prevent overfitting, whereby the initial segmentation model 412 will learn to map features unique to samples of the training set that are not present in the test set. The process 400 includes training the initial segmentation model 412 based on the training pairs 408. The process 400 may include a validator 414 that validates the performance of the initial segmentation model 412 against the test pairs 410 (when the initial model is trained). The validator 414 may take as input a trained or partially trained model (e.g., the initial segmentation model 412, but after the model has been trained and updated) and a dataset of the test pairs 410, and may output an evaluation of the performance of the trained or partially trained segmentation model based on the dataset of the test pairs 410.

[0058] Therefore, the initial segmentation model 412 is trained on training pairs, wherein the training pairs include annotated input volumes and reference truth segmentations. Additional training pairs may be used to train the initial segmentation model 412, each training pair including annotated input volumes at various orientations relative to the canonical plane and reference truth segmentations. It should be understood that different pairs may be different objects. The corresponding annotations for each annotated input volume indicate the location of the fetal head and track in each annotated input volume. The corresponding annotations may be considered as reference truth segmentations. The reference truth segmentations may be compared to the segmentations output from the initial segmentation model to calculate a loss function for adjusting the model parameters of the initial segmentation model.

[0059] After the validator 414 determines that the segmentation model is sufficiently trained, the segmentation model can be stored in the segmentation module 208 and the inference module 416, which can be Figure 2 The segmentation model 418, when deployed, can segment the fetal head and orbit included in the input volume acquired using the ultrasound imaging system. The newly acquired input volume can be input as input to the segmentation model 418 to output a segmented fetal head 420 and a segmented orbit 422. According to the embodiments described herein, the segmented orbit 422 can be used to determine the mid-sagittal plane.

[0060] Figure 5 The midsagittal plane (MSP) 500 analytically determined based on two tracks, such as a first track 502 and a second track 504, is shown. In some embodiments, the first track 502 and the second track 504 can be output from a segmentation model, which can be Figure 3 The segmentation model 304 and Figure 4500 is an embodiment of a segmentation model 418 of the invention. MSP 500 is a plane perpendicular to a vector 506 connecting the centroids of a first track 502 and a second track 504. The first track 502 and the second track 504 may be segmented tracks output from a segmentation model. The plane is equidistant from the first track 502 and the second track 504. In particular, the centroid of the first track 502 may be at a first point 508, and the centroid of the second track 504 may be at a second point 510. Vector 506 may be analytically determined by subtracting the second point 510 from the first point. MSP 500 may be analytically determined using vector 506, which is an embodiment of vector 506, by using a vector equation including a point on the plane and a vector connecting the centroids of the first track 502 and the second track 504. In some embodiments, the point may be the midpoint of the first track 502 and the second track 504.

[0061] Figure 6 6 is a flow chart illustrating a method 600 for identifying a midsagittal plane (MSP) of an input volume and providing feedback and recommendations to a user based on whether the MSP is an oblique plane / reconstruction plane according to an embodiment of the present disclosure. The method 600 may be based on a non-transitory memory and a computing device (such as Figure 2 The image processing system 202 may be executed by executing instructions executed by one or more processors of the image processing system 202.

[0062] At 602, method 600 includes obtaining an input volume of a fetus acquired with an ultrasound imaging system. The input volume may be a three-dimensional (3D) image mapped from two-dimensional (2D) slices generated from ultrasound data collected during a scanning session of a fetal head, wherein the ultrasound data was acquired with a 3D probe operating in a 2D mode. In some embodiments, the ultrasound data may be acquired using a 3D probe. Figure 1 During scanning, ultrasound data used to generate the input volume is acquired using a fixed ultrasound probe.

[0063] At 604, method 600 includes segmenting a fetal head and a track by inputting an input volume into a segmentation model. The segmentation model includes a deep learning model trained using a plurality of training pairs, each training pair including an annotated training input volume and a ground truth segmentation. The segmentation model is trained to output a segmentation mask that identifies a fetal head without clutter, a first track, and a second track. By entering the input volume as input into the segmentation model, the input volume can be segmented into a segmented fetal head, a first segmented track, and a second segmented track. The segmentation model can be relative to Figure 3 and Figure 4The segmentation model described herein, or another suitable segmentation model configured to segment the input volume into the fetal head and the two tracks. The segmentation model may output a segmentation mask that identifies pixels in the input volume corresponding to each of the fetal head, the first track, and the second track.

[0064] In some examples, the segmentation model can identify more than two tracks, where the two or more tracks are generally located at two different locations. However, because the fetal head includes two tracks, the segmentation model can group potential tracks that are in close proximity together. Alternatively, in response to the segmentation model identifying more than two segmented tracks, all segmented tracks near a first region can be combined to form a first segmented track and all segmented tracks near a second region can be combined to form a second segmented track.

[0065] In detail, the first segmented track may be a first region, wherein the first region includes all identified tracks located near the first region. Therefore, the first segmented track located in the first region is a combination of pixels corresponding to a plurality of segmented tracks located in the first region. Similarly, the second segmented track may be a second region, wherein the second region includes all identified tracks located near the second region. In this way, the second segmented track located in the first region is a combination of pixels corresponding to a plurality of segmented tracks located in the second region.

[0066] An example of a pair of split tracks 800 is shown in Figure 8 . As shown, the pair of segmented tracks 800 includes a first segmented track 802 in the first region and a second segmented track 804 in the second region. The segmentation model may have initially identified four potential tracks near the first region, such as a first potential track 802a, a second potential track 802b, a third potential track 802c, and a fourth potential track 802d. Since the fetal head includes only two tracks, the first region including the four potential tracks may be considered to be the first segmented track 802. Similarly, the segmentation model may have initially identified three tracks, such as a fifth potential track 804a, a sixth potential track 804b, and a seventh potential track 804c. In this way, the second region including the three potential tracks may be considered to be the second segmented track 804.

[0067] In other examples, the segmentation model may identify more or fewer potential tracks than described above. Regardless of the number of potential tracks identified, potential tracks that focus on a particular location are combined to form a single segmented track. However, in order to determine the MSP, the segmentation model identifies at least two tracks. Otherwise, the MSP may not be analytically determined according to the embodiments described below. Instead, in response to the segmentation model outputting a segmented track, the user may be prompted to reorient the ultrasound probe and reacquire the input volume.

[0068] Generating a segmentation mask using a segmentation model may include identifying groups of pixels corresponding to anatomical regions, and each group of pixels may correspond to one anatomical region. For example, a first group of pixels may correspond to a first anatomical region, which may be a fetal head, a second group of pixels may correspond to a second anatomical region, which may be a first orbit, and a third group of pixels may correspond to a third anatomical region, which may be a second orbit. A segmentation mask may be generated based on groups of pixels corresponding to different anatomical regions. The segmentation mask may include a pixel value for each pixel of the image, the pixel value indicating the anatomy in the input volume, for example, a value of 0 indicates that no anatomical region was identified, a value of 1 indicates a fetal head, a value of 2 indicates a first orbit, and so on. However, other mechanisms for identifying anatomical regions in a segmentation mask are possible without departing from the scope of the present disclosure.

[0069] At 606, method 600 includes identifying the MSP by analytically determining a plane perpendicular to a vector connecting the centroids of the segmented tracks. Figure 5 As described, the plane may be determined analytically using a vector equation that includes points on the plane and a vector connecting the centroids of the first segmented track and the second segmented track.

[0070] At 608, method 600 includes visually displaying the MSP on the segmented fetal head on a display device. The display device may be Figure 2 Embodiment of display device 234. In this manner, the orientation of the fetal head and MSP relative to three canonical planes (eg, axial, coronal, and sagittal) can be visualized. Figure 12-Figure 15 Example images are shown in which the MSP is visualized on a segmented fetal head.

[0071] At 610, method 600 includes determining whether the MSP is a plane of acquisition. The MSP may be considered a plane of acquisition when the MSP is coplanar or relatively coplanar with one of the three canonical planes. More specifically, the MSP may be considered a plane of acquisition when the tilt angle of the MSP relative to one of the three canonical planes is within an angle threshold. The angle threshold may range from 0° to a maximum acceptable angle.

[0072] Determining whether the MSP is not a plane of acquisition may include: determining a tilt angle of the MSP relative to one of the three canonical planes; and determining whether the tilt angle of the MSP is within an angle threshold by comparing the tilt angle to a maximum acceptable tilt angle. In some embodiments, the MSP is stored in a computing device (such as Figure 2 Instructions in the image processing system 202 can analytically determine the tilt angle of the MSP relative to each of the three canonical planes. In particular, the tilt angle of the MSP is analytically determined based on an equation including a first normal vector of the MSP and a second normal vector of the corresponding canonical plane, the first normal vector being a vector between at least two segmented tracks. In other embodiments, a user can manually determine whether the tilt angle of the MSP is within an angle threshold when visually inspecting the MSP relative to the canonical planes. Examples of various tilt angles of the MSP relative to one of the three canonical planes are shown in Figure 9-11 Shown in.

[0073] exist Figure 9-11 , coordinate systems 900, 1000, and 1100 are shown, wherein a first midsagittal plane (MSP) 904 of a first orientation, a second MSP 1004 of a second orientation, and a third MSP of a third orientation are visualized on a segmented fetal head (not shown). Coordinate systems 900, 1000, and 1100 include an x-axis, a y-axis, and a z-axis. The three canonical planes may be an xy plane, an xz plane, and a yz plane of coordinate systems 900, 1000, and 1100. The three canonical planes are associated with the probe axis. Typically, the plane of acquisition is coplanar with the azimuth plane, or is at an inclination angle relative to the azimuth plane, the inclination angle being within an angle threshold. For simplicity, the first MSP 904, the second MSP 1004, and the third MSP 1104 may be considered to be the plane of acquisition when the corresponding MSP is coplanar with the yz plane (canonical plane) or the inclination angle of the corresponding MSP relative to the yz plane (e.g., the canonical plane) is within an angle threshold.

[0074] Go to Fig. 9, the coordinate system 900 includes a first MSP 904, a canonical plane 902 (not visible), and a theoretical MSP 906. The theoretical MSP 906 has a tilt angle 908 relative to the canonical plane 902. In some embodiments, the maximum tilt angle 908 can be in the range of 10°-15°. For the purpose of example, the tilt angle 908 of the theoretical MSP is 15°. The theoretical MSP 906 can be considered as an MSP with a maximum acceptable tilt angle 908. In other words, the tilt angle 908 of the theoretical MSP 906 is within the angle threshold of an MSP to be classified as an acquired plane. Since the first MSP 904 is coplanar with the canonical plane 902 and the tilt angle of the first MSP 904 is less than the tilt angle 908 of the theoretical MSP, the first MSP 904 can be considered as an acquired plane rather than a tilted plane / reconstructed plane.

[0075] like Fig.10 As shown, coordinate system 1000 includes a second MSP 1004 having a tilt angle 1008 relative to canonical plane 1002, canonical plane 1002, and theoretical MSP 1006. Second MSP 1004 is not coplanar with canonical plane 1002. As an example, tilt angle 1008 may be 20°. Similar to Fig. 9 Theoretical MSP 906, theoretical MSP 1006 has a tilt angle relative to canonical plane 1002, which can also be 15°, and can be considered as the MSP with the maximum acceptable tilt angle (e.g., within the angle threshold). Since the tilt angle 1008 of the second MSP 1004 exceeds the tilt angle of the theoretical MSP 1006 and is not within the acceptable angle threshold, the second MSP is considered to be a tilted plane / reconstruction plane.

[0076] like Fig.11 As shown, coordinate system 1100 includes third MSP 1104 having an inclined angle 1108 relative to canonical plane 1102, canonical plane 1102, and theoretical MSP 1106. Third MSP 1104 is not coplanar with canonical plane 1102. As an example, angle 1108 may be 10°. Similar to Fig. 9 and Fig.10 , the theoretical MSP 1106 has a tilt angle relative to the canonical plane 1102, which can also be 15°, and can be considered as the MSP with the maximum acceptable tilt angle (e.g., within the angle threshold). Since the tilt angle 1108 of the third MSP 1104 does not exceed the tilt angle of the theoretical MSP 1006 and is within the acceptable angle threshold, the third MSP is considered to be a tilted plane / reconstruction plane.

[0077] In response to determining that the MSP is an inclined plane rather than an acquired planar image, method 600 includes alerting a user of poor face rendering or the possibility of poor face rendering at 612. The user may be alerted that the MSP is not suitable for face rendering and that ultrasound images generated using the current input volume may exhibit reduced image quality. In particular, the user may be shown on a display device (such as Figure 2 A message is displayed on a display device 234) to warn the user of poor facial rendering or the possibility of poor facial rendering of the fetal head.

[0078] At 614, method 600 includes prompting the user to re-orient and re-acquire the input volume. More specifically, a message prompting the user to re-orient the probe and re-acquire the input volume can be displayed on the display device.

[0079] At 616, method 600 includes determining whether to request recapture of the input volume. In some embodiments, the user input may be received at a user interface of the display device. More specifically, the user input may be received at a user input device such as user input device 232. In this manner, the image processing system (which may be Figure 2 In some embodiments, the image processing system 202 may receive user input. In some embodiments, the image processing system may initiate a new scanning session to obtain ultrasound data to generate an input volume in response to receiving the user input. In other embodiments, the image processing system may terminate the scanning session and display the current input volume in response to receiving the user input.

[0080] In response to receiving user input requesting reacquisition of an input volume, method 600 includes obtaining an input volume of an object acquired using an ultrasound imaging system and determining an MSP until the determined MSP of the input volume is a plane of acquisition or the image processing system receives user input terminating a scanning session or acquisition process.

[0081] At 618, method 600 includes displaying and / or saving the input volume. The input volume may be displayed using a display device, such as a display device communicatively coupled to the image processing system, which may be Figure 2 The image processing system 202 is provided. In this way, the medical professional can visually assess the contents of the input volume and assess the development of the fetal head based on the contents of the input volume. By ensuring that the MSP is a plane of acquisition, the medical professional can more easily correctly assess the development of the fetal head because the frequency of poor facial rendering can be reduced and the image quality is not degraded. In addition, the final transformed image can be stored in a memory of the image processing system (e.g., Figure 2The input volume may be stored in a non-transitory memory 206 or in an image archive (such as a PACS) to enable a user or medical professional to access the input volume at a later time. The method 600 then ends.

[0082] Returning to 610, in response to determining that the MSP is the plane of acquisition, method 600 includes displaying and / or saving the segmented image at 618. The input volume can be displayed on and saved to the system as previously described to enable a medical professional to assess the development of the fetal head. Method 600 then ends.

[0083] In some examples, one or more aspects of method 600 may be performed in response to receiving user input at a user input device and / or as part of a scanning protocol. In some examples, as part of the scanning protocol, the input volume may be segmented to identify the MSP. In other examples, the input volume may be segmented to identify the MSP in response to a user request, wherein the user requests segmentation of the input volume and determination of the MSP by interacting with the user input device. For example, a user may specify when to segment the input volume to identify the MSP. In some examples, the user may initially view the input volume and request segmentation of the input volume to identify the MSP after viewing the input volume in order to determine a strategy for redirecting the ultrasound probe. Therefore, in some examples, the input volume may be displayed on a display device prior to segmentation of the input volume and identification of the MSP. However, in other examples, the image may not be displayed on the display device prior to segmentation of the input volume and identification of the MSP.

[0084] Furthermore, in other embodiments, one or more aspects of method 600 may be implemented by using alternative means other than segmentation and segmentation models to identify the fetal head, the first track, and the second track. As an example, the first track and the second track may be identified using a localization and detection method. Once identified, the first track and the second track may be used to identify the MSP, as described above in Figure 6 Described in .

[0085] Now turn to Figure 7 , which shows a flowchart illustrating an example method 700 for training a segmentation model to segment an input volume into a fetal head and a track of the fetal head. The method 700 is a flowchart of an example method 700 for training a segmentation model to segment an input volume into a fetal head and a track of the fetal head. Figure 1-Figure 2 The method 700 is described with reference to the system and components of the present invention, but it should be understood that the method 700 can be implemented with other systems and components without departing from the scope of the present disclosure. The method 700 can be based on a non-transitory memory such as a Figure 1 Memory 120 or Figure 2 in the non-transitory memory 206 and by a processor of a computing device such as Figure 1 The processor 116 or Figure 2The processor 206 executes the instructions to execute.

[0086] At 702, method 700 includes receiving a plurality of annotated training input volumes in various orientations, each of which may be annotated with a ground truth segmentation including a fetal head and two orbits. For example, the annotated training input volume may be annotated with a ground truth segmentation including a fetal head and two orbits. Figure 1 An input volume of one or more objects is obtained by an ultrasound imaging system of the present invention. Various orientations of the input volume include a fetal head at various positions relative to three canonical planes, each canonical plane being orthogonal to the other canonical planes.

[0087] The ground truth segmentation can be an expert generated label / annotation. For example, one or more experts (e.g., clinicians) can evaluate the training input volume and generate ground truth by manually (e.g., via a user input device) labeling / annotating the training input volume with a ground truth segmentation indicating, for example, the fetal head and the boundary of the orbit.

[0088] At 704, method 700 includes selecting an annotated training input volume (eg, training pair) from a plurality of annotated training input volumes. Instructions configured, stored, and executed by a processor in a memory may cause the processor to select the annotated training input volume from the plurality of annotated training input volumes.

[0089] The selected training pair may include the first annotated training input volume for the first object described above. Each training pair includes an annotated training input volume and a ground truth segmentation. The corresponding annotation of each annotated training input volume indicates the segmentation of the fetal head, the first track and the second track in each annotated training input volume. In particular, each annotated training input volume can be annotated with a ground truth segmentation.

[0090] At 706, method 700 includes inputting the annotated training input volume into the segmentation model. Figure 3 The one or more processors of the image processing system are configured, stored, and executed with instructions that cause the annotated training input volume to be entered as input into the segmentation model. The segmentation model may be a pre-trained Figure 4 An initial segmentation model 412 (e.g., initial segmentation model 412). In some embodiments, the segmentation model can be a deep learning model such as UNET.

[0091] At 708, method 700 includes receiving a segmented fetal head and at least two segmented tracks output from the segmentation model. The at least two segmented tracks output from the segmentation model may include a first segmented track located in the first region and a second segmented track located in the second region. In some embodiments, a segmentation mask output from the segmentation model may be utilized to identify the segmented fetal head, the first segmented track, and the second segmented track. The segmentation mask identifies pixels in the input volume corresponding to each of the segmented fetal head, the first segmented track, and the second segmented track as described herein.

[0092] At 710, method 700 includes comparing the ground truth segmentation with the output segmented fetus and segmented track and adjusting model parameters via back propagation. In this way, one or more weights, biases, gradients, etc. of the segmentation model can be updated based on one or more losses between the output of the segmentation model and the associated ground truth segmentation. The loss can be determined using a loss function such as a dice function. In some examples, the segmentation model can be trained to output a classification for each pixel in the input volume based on characteristics of the input image, wherein the classification indicates whether the pixel belongs to an anatomical ROI or whether the pixel belongs to background (or another suitable classification).

[0093] At 712, method 700 includes determining whether additional annotated training input volumes are included in the plurality of annotated training input volumes. As described above, a segmentation model may be trained using a plurality of training pairs, each training pair including an annotated training input volume and a ground truth segmentation. If fewer than all training pairs have been selected and used to train the segmentation model (e.g., at least some training images remain), or if the segmentation is otherwise determined to be incompletely trained, there may be additional annotated training input volumes to be entered as input into the segmentation model. Conversely, if it is determined that every triplet of training images has been selected and used to train the segmentation model (and no more training images remain), or if the segmentation model is otherwise determined to be fully trained, there are no additional annotated training input volumes to be entered as input into the segmentation model.

[0094] In response to determining that there are additional annotated training input images, method 700 includes selecting an annotated training input volume from a plurality of annotated training input volumes at 704, and continuing to train the segmentation model on the remaining annotated training input images. In response to determining that there are no additional annotated training input volumes, method 700 then ends.

[0095] Figure 12-Figure 15Additional examples of a mid-sagittal plane (MSP) visualized on a segmented fetal head of a subject at various orientations relative to a coordinate system defining three canonical planes are shown. The coordinate system may be related to the probe axis relative to the body surface. Figure 12-Figure 15 The image shown in is the fetal head volume of the subject.

[0096] Fig.12 A first example image 1200 of a fetal head volume at a first orientation generated by a segmentation model using ultrasound data is shown. The first example image 1200 can be a fetal head volume relative to a first axis 1202 (which can be a y-axis), a second axis 1204 (which can be an x-axis), and a third axis 1206 (which can be a z-axis). The three canonical planes can be coordinate planes (e.g., an xy plane, an xz plane, and a yz plane). The first example image 1200 can include an MSP 1208 visualized on the fetal head volume. By visualizing the MSP 1208 on the fetal head volume, the tilt angle of the MSP relative to the second axis 1204 can be visualized.

[0097] Fig.13 1300 is a second example image 1300 of a fetal head volume in a second orientation. The second example image 1300 can be a fetal head volume relative to a first axis 1302 (which can be a y-axis), a second axis 1304 (which can be an x-axis), and a third axis 1306 (which can be a z-axis). The three canonical planes can be coordinate planes (e.g., an xy plane, an xz plane, and a yz plane). The second example image 1300 can include an MSP 1308 visualized on the fetal head volume. By visualizing the MSP 1308 on the fetal head volume, the tilt angle of the MSP relative to the first axis 1302 can be visualized.

[0098] Fig.14 1400 is a third example image 1400 of a fetal head volume in a third orientation. The third example image 1400 can be a fetal head volume relative to a first axis 1402 (which can be a y-axis), a second axis 1404 (which can be an x-axis), and a third axis 1406 (which can be a z-axis). The three canonical planes can be coordinate planes (e.g., an xy plane, an xz plane, and a yz plane). The third example image 1400 can include an MSP 1408 visualized on the fetal head volume. By visualizing the MSP 1408 on the fetal head volume, the tilt angle of the MSP relative to the first axis 1402 can be visualized. As shown in the third example image 1400, the MSP 1408 is parallel to the yz plane.

[0099] Fig.151500 is a fourth example image 1500 of a fetal head volume in a fourth orientation. The fourth example image 1500 can be the fetal head volume relative to a first axis 1502 (which can be a y-axis), a second axis 1504 (which can be an x-axis), and a third axis 1506 (which can be a z-axis). The three canonical planes can be coordinate planes (e.g., an xy plane, an xz plane, and a yz plane). The fourth example image 1500 can include an MSP 1508 visualized on the fetal head volume. By visualizing the MSP 1508 on the fetal head volume, the tilt angle of the MSP relative to the first axis 1502 can be visualized. As shown by Figure 12-Figure 15 As will be appreciated, the tilt angle may be estimated via visual inspection or analytically determined and compared to a maximum acceptable tilt angle. In this manner, according to embodiments described herein, an MSP may be classified as either a plane of acquisition or a tilted plane.

[0100] The technical effect of identifying the midsagittal plane (MSP) of the input volume based on the segmented trajectory output from the segmentation model and predicting the quality of the facial rendering of the fetal head based on whether the identified MSP is the acquired plane is that memory storage can be managed more efficiently. Instead of dedicating addresses in the memory to generating ultrasound data of a 3D volume of a fetal head that produces poor facial rendering, the addresses can be dedicated to generating ultrasound data of a 3D volume of acceptable quality and enabling medical professionals to monitor the development of the fetal head. In addition, because the sonographer is warned when poor facial rendering is expected before displaying the 3D volume and can re-acquire ultrasound data accordingly, the duration of the scanning session to acquire ultrasound data can be reduced for both the patient and the sonographer.

[0101] The present disclosure also provides support for a method, the method comprising: obtaining an input volume acquired using an ultrasound imaging system, identifying a midsagittal plane (MSP) of a fetal head using a first track and a second track, visually displaying the MSP on the fetal head, in response to determining that the MSP is not the acquired plane, warning a user of poor facial rendering or the possibility of poor facial rendering and prompting the user to re-orient the ultrasound probe and re-acquire the input volume, and displaying the input volume and / or saving the input volume in a memory. In a first example of the method, ultrasound data used to generate the input volume is acquired using a fixed ultrasound probe. In a second example of the method, optionally including the first example, the fetal head is a segmented fetal head, and wherein the first track is a first segmented track and the second track is a second segmented track, and wherein identifying the MSP of the segmented fetal head includes: entering the input volume as input to a segmentation model, the segmentation model is trained to output a segmentation mask, the segmentation mask identifies the segmented fetal head, the first segmented track, and the second segmented track; and analytically determining a plane perpendicular to a vector connecting the centroids of the first segmented track and the second segmented track. In a third example of the method, optionally including one of the first example and the second example, the plane is equidistant from the first segmented track and the second segmented track, and is orthogonal to a vector connecting the centroids of the first segmented track and the second segmented track.

[0102] In a fourth example of the method, optionally including one or more or each of the first to third examples, a plane is analytically determined using a vector equation, wherein the vector equation includes points on the plane and a vector connecting the centroids of the first segmented track and the second segmented track. In a fifth example of the method, optionally including one or more or each of the first to fourth examples, the acquired plane is coplanar with the azimuth plane, or is at an angle of inclination relative to the azimuth plane, and the inclination angle is within an angle threshold. In a sixth example of the method, optionally including one or more or each of the first to fifth examples, the angle threshold ranges from 0° to a maximum acceptable angle. In a seventh example of the method, optionally including one or more or each of the first to sixth examples, the segmentation model includes a deep learning model trained using multiple training pairs, each training pair including an annotated training input volume and a ground truth segmentation, and wherein the corresponding annotation of each annotated training input volume indicates the segmentation of the fetal head, the first track, and the second track in each annotated training input volume.

[0103] The present invention also provides support for a system, the system comprising: one or more processors, and a memory storing instructions executable by the one or more processors to: obtain an image at a first energy level, the image reconstructed from projection data acquired at a single peak energy level; obtain an input volume acquired using an ultrasound imaging system; identify a midsagittal plane (MSP) of a segmented fetal head; visually display the MSP on the segmented fetal head; in response to determining that the MSP is not a plane of acquisition, warn a user of poor facial rendering or the possibility of poor facial rendering and prompt the user to re-orient the ultrasound probe and reacquire the input volume; and display the input volume and / or save the input volume in a memory. In a first example of the system, identifying the MSP of the segmented fetal head is based on output from a segmentation model that identifies the segmented fetal head, a first segmented track, and a second segmented track. In a second example of the system, optionally including the first example, determining whether the MSP is not a plane of acquisition may include: determining a tilt angle of the MSP relative to one of three canonical planes; and determining whether the tilt angle of the MSP is within an angle threshold by comparing the tilt angle to a maximum acceptable tilt angle.

[0104] In a third example of the system, optionally including one or both of the first and second examples, training of the segmentation model includes: receiving multiple annotated training input volumes in various orientations, each annotated training input volume annotated with a reference truth segmentation, the reference truth segmentation including a fetal head, a first track, and a second track; selecting an annotated training input volume from the multiple annotated training input volumes; inputting the annotated training input volume to the segmentation model; receiving a segmented fetal head, at least a first segmented track, and at least a second segmented track output from the segmentation model; and comparing the reference truth segmentation and the output segmented fetal head, the first segmented track, and the second segmented track; and adjusting model parameters via back-propagation.

[0105] In a fourth example of the system, optionally including one or more or each of the first to third examples, the training of the segmentation model further includes: in response to the segmentation model identifying more than two segmented tracks, combining all segmented tracks located near the first region to form a first segmented track, and combining all segmented tracks located near the second region to form a second segmented track. In a fifth example of the system, optionally including one or more or each of the first to fourth examples, the various orientations include the segmented fetal head being in various positions relative to three canonical planes, each canonical plane being orthogonal to the other canonical planes.

[0106] The present invention also provides support for a method comprising: obtaining an input volume of a fetal head, the input volume mapped from two-dimensional (2D) slices generated from ultrasound acquired using an ultrasound imaging system, entering the input volume as input into a segmentation model trained to output a segmented fetal head and at least two segmented tracks without clutter artifacts, identifying a midsagittal plane (MSP) of the fetal head by analytically determining a plane orthogonal to a vector between the at least two segmented tracks, visually displaying the MSP on the segmented fetal head and determining a tilt angle of the MSP relative to a canonical plane, the canonical plane being one of three canonical planes, in response to the MSP being identified as a tilted plane, warning a user of poor facial rendering or the possibility of poor facial rendering and prompting the user to re-orient the ultrasound probe and reacquire the input volume, and displaying the input volume on a display device and / or saving the input volume in a memory.

[0107] In a first example of the method, the method further includes: in response to the segmentation model outputting more than two segmented tracks, prompting the user to re-orient the ultrasound probe and re-acquire the input volume. In a second example of the method, optionally including the first example, the at least two segmented tracks output from the segmentation model include a first segmented track located in the first region and a second segmented track located in the second region. In a third example of the method, optionally including one or both of the first and second examples, the first segmented track located in the first region is a combination of pixels corresponding to more than one segmented track located in the first region.

[0108] In a fourth example of the method, optionally including one or more or each of the first to third examples, the second segmented track located in the first region is a combination of pixels corresponding to more than one segmented track located in the second region. In a fifth example of the method, optionally including one or more or each of the first to fourth examples, the tilt angle of the MSP is estimated based on a visual inspection of a user or analytically determined based on an equation including a first normal vector of the MSP and a second normal vector of a corresponding canonical plane, the first normal vector being a vector between at least two segmented tracks.

[0109] When introducing the elements of various embodiments of the present disclosure, the articles "one", "a kind of" and "the" are intended to mean that there are one or more such elements. The terms "first", "second", etc. do not represent any order, amount or importance, but are used to distinguish one element from another element. The terms "include", "comprises", and "have" are intended to be inclusive, and mean that additional elements may also exist in addition to the listed elements. As used herein, the terms "connected to", "coupled to", etc., an object (e.g., material, element, structure, member, etc.) may be connected to or coupled to another object, regardless of whether the one object is directly connected or coupled to another object, or whether there are one or more intervening objects between the one object and another object. In addition, it should be understood that reference to "an embodiment" or "embodiment" of the present disclosure is not intended to be interpreted as excluding the existence of additional embodiments that are also combined with the cited features.

[0110] In addition to any previously indicated modifications, numerous other variations and alternative arrangements may be devised by those skilled in the art without departing from the spirit and scope of the present specification, and the appended claims are intended to cover such modifications and arrangements. Therefore, although the information has been described in detail and in detail as described above in conjunction with what are currently considered to be the most practical and preferred aspects, it will be apparent to those of ordinary skill in the art that many modifications may be made, including but not limited to form, function, mode of operation, and use, without departing from the principles and concepts set forth herein. Likewise, as used herein, the examples and embodiments are intended to be illustrative in all respects only and should not be construed as limiting in any way.

Claims

1. A method, comprising: Obtaining (602) an input volume acquired using an ultrasound imaging system; identifying ( 606 ) a midsagittal plane (MSP) of the fetal head using the first track and the second track; visually displaying (608) the MSP on the fetal head; In response to determining that the MSP is not a plane of acquisition, warning (612) a user of poor facial rendering or the possibility of poor facial rendering, and prompting (614) the user to re-orient the ultrasound probe and reacquire the input volume; and The input volume is displayed (618) and / or saved in a memory.

2. The method according to claim 1, wherein: The ultrasound data used to generate the input volume is acquired using a fixed ultrasound probe.

3. The method according to claim 1, wherein: The fetal head is a segmented fetal head, and wherein the first track is a first segmented track and the second track is a second segmented track, and wherein identifying the MSP of the segmented fetal head comprises: entering (604) the input volume as input to a segmentation model, the segmentation model being trained to output a segmentation mask identifying the segmented fetal head, the first segmented track, and the second segmented track; and A plane perpendicular to a vector connecting the centers of mass of the first segmented track and the second segmented track is analytically determined (606).

4. The method according to claim 3, wherein: The plane is equidistant from the first segmented track and the second segmented track and is orthogonal to the vector connecting the centroids of the first segmented track and the second segmented track.

5. The method according to claim 4, wherein: The plane is determined analytically using a vector equation including points on the plane and the vector connecting the centroids of the first segmented track and the second segmented track.

6. The method according to claim 1, wherein: The collected plane is coplanar with the azimuth plane or is inclined at an angle relative to the azimuth plane, and the inclination angle is within an angle threshold.

7. The method according to claim 6, wherein: The angle threshold ranges from 0° to the maximum acceptable angle.

8. The method according to claim 3, wherein: The segmentation model comprises a deep learning model trained using a plurality of training pairs, each training pair comprising an annotated training input volume and a ground truth segmentation, and wherein a corresponding annotation of each annotated training input volume indicates a segmentation for a fetal head, a first track, and a second track in each annotated training input volume.

9. A system, comprising: one or more processors (204); and A memory (120) storing instructions executable by the one or more processors to perform the following operations: Obtaining (602) an input volume (302) acquired using an ultrasound imaging system (100); identifying (606) a midsagittal plane (MSP) (310) of the segmented fetal head (306); visually displaying (608) the MSP (1208, 1508, 310, 500, 1308, 1408) on the segmented fetal head (420, 306); In response to determining that the MSP is not a plane of acquisition, warning (612) a user of poor facial rendering or the possibility of poor facial rendering, and prompting (614) the user to re-orient the ultrasound probe (106, 236) and reacquire the input volume (302); and The input volume (302) is displayed and / or the input volume (302) is stored in a memory (120).

10. The system according to claim 9, wherein: The MSP (310) that identifies the segmented fetal head (306) is based on output from a segmentation model (304) that identifies the segmented fetal head (306), a first segmented track (802), and a second segmented track (804).

11. The system according to claim 9, wherein: Determining that the MSP (310) is not the acquired plane includes: determining a tilt angle (908, 1008, 1108) of the MSP (310) relative to one of three canonical planes; and A determination is made as to whether the tilt angle (908, 1008, 1108) of the MSP (310) is within an angle threshold by comparing the tilt angle to a maximum acceptable tilt angle.

12. The system according to claim 10, wherein: The training of the segmentation model (418, 304) includes: receiving (702) a plurality of annotated training input volumes (402) at various orientations, each annotated training input volume (302) annotated with a ground truth segmentation, the ground truth segmentation comprising a fetal head, a first track (502), and a second track (504); selecting (704) an annotated training input volume (302) from the plurality of annotated training input volumes (402); Inputting (706) the annotated training input volume (302) to the segmentation model (418, 304); receiving (708) the segmented fetal head (306, 420), at least the first segmented track (802), and at least the second segmented track (804) output from the segmentation model (304, 418); and The ground truth segmentation and the output segmented fetal head (306, 420), the first segmented track (802), and the second segmented track (804) are compared (710) and model parameters are adjusted via back-propagation.

13. The system according to claim 12, wherein: The training of the segmentation model (418, 304) further includes: In response to the segmentation model (418, 304) identifying more than two segmented tracks (308, 422), all segmented tracks (308, 422) located near a first area are combined to form the first segmented track (802), and all segmented tracks (308, 422) located near a second area are combined to form the second segmented track (804).

14. The system according to claim 12, wherein: The various orientations include the segmented fetal head (306, 420) in various positions relative to three canonical planes, each canonical plane (902, 1002, 1102) being orthogonal to the other canonical planes.

Images