Processing sequence of ultrasound images

Abstract

According to one aspect, there is provided a method of processing a sequence of ultrasound US images having periodically moving anatomical features. The method comprises: i) using a first machine learning ML model to mark a detection result of the anatomical feature in a respective image according to a view plane of the anatomical feature visible in each image in the sequence; ii) obtaining a first cluster of all successive images in the sequence corresponding to a first view plane based on the markers; iii) using the first cluster as a first clip of the first view plane of the anatomical feature; repeating steps i), ii) and iii) to obtain a plurality of clips of different view planes of the anatomical feature; and if the first clip includes a cluster of consecutive images, selecting the first clip from the plurality of clips as a preferred clip of the anatomical feature: the respective indicia for the cluster of consecutive images are statistically more significant than other indicia in the plurality of clips.

Description

Technical Field

[0001] The disclosure herein relates to ultrasound imaging. In particular, but not exclusively, the disclosure relates to processing a sequence of ultrasound images having periodically moving anatomical features. Background Art

[0002] Congenital heart disease (CHD) is one of the major defects that affect fetal health and pregnancy outcomes. It has been observed that CHD affects approximately 1% of births in the United States each year, see the article entitled "The incidence of congenital heart disease" by Hoffman JL and Kaplan S (Journal of the American College of Cardiology, 2002, Vol. 39, No. 12, pp. 1890-1900). It has been observed that CHD can be asymptomatic, mildly symptomatic, but can become fatal after birth. According to Oster ME et al., titled "Temporal trends in survival among infants with critical congenital heart defects" (Pediatrics, May 2013, Vol. 131, No. 5, e1502-8, doi: 10.1542 / peds, pp. 2012-3435, Epub 2013 Apr 22, PMID: 23610203, PMCID: PMC4471949), one in four infants with CHD require a surgical procedure within their first year of life. This emphasizes the need for early detection of CHD for better treatment options and patient outcomes.

[0003] As described in the article "Early Evaluation of the Fetal Heart" by Hernandez-Andrade E, Patwardhan M, Cruz-Lemini M, Luewan S. (Journal of Fetal Diagnosis and Therapy, 2017, Vol. 42, pp. 161-173, doi: 10.1159 / 000477564), there are several risk factors associated with the presence of CHD. These risk factors include noncardiac structural abnormalities, previous history of CHD, abnormal ductus venosus, increased nuchal translucency, monochorionic twins, anomalies of the right subclavian artery, consanguinity, and assisted reproductive technology.

[0004] Ultrasound (US) monitoring has been shown to be effective in the early detection of fetal cardiac abnormalities. However, in order to perform fetal cardiac examinations, a lot of manual labor is required.

[0005] WO2019 / 178404A1 discloses a computer vision pipeline for fully automatic interpretation of cardiac function, which includes preprocessing of echo studies, CNN processing for view recognition, segmentation of ventricles and delineation of cardiac boundaries, particle tracking for calculating longitudinal strain, and target disease detection. US2006 / 0064017A1 discloses a processor for identifying cardiac views of medical ultrasound images. Summary of the invention

[0006] As described above, in order to perform US examination of the fetal heart, a lot of manual processing of the US images is usually required. An extended US procedure is usually recommended to detect the presence of any malformation in the fetal heart. The extended US examination involves accurately detecting the fetal heart views recommended by the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) and / or the American Institute of Ultrasound in Medicine (AIUM) to diagnose CHD. The purpose of the embodiments herein is to improve efficiency and reduce the manual effort currently required to perform US examinations on a sequence of US images of anatomical features (e.g., the fetal heart) that move periodically.

[0007] According to a first aspect, a method for processing a sequence of ultrasound (US) images of an anatomical feature with periodic movement is provided. The method comprises: i) using a first machine learning ML model to mark the detection result of the anatomical feature in the corresponding image according to the view plane of the anatomical feature visible in each image in the sequence; ii) obtaining a first cluster of all consecutive images in the sequence corresponding to the first view plane based on the marking; iii) using the first cluster as a first clip of the first view plane of the anatomical feature; repeating steps i), ii) and iii) to obtain multiple clips of different view planes of the anatomical feature; and if the first clip includes a cluster of the following consecutive images, selecting the first clip from the multiple clips as a preferred clip of the anatomical feature: the corresponding label for the cluster of the consecutive images is statistically more significant than other labels in the multiple clips.

[0008] According to a second aspect, there is a device for processing a sequence of US images of an anatomical feature with periodic movement. The device includes: a memory including instruction data representing a set of instructions; and a processor configured to communicate with the memory and execute the set of instructions. When the set of instructions is executed by the processor, the processor performs the following operations: i) using a first machine learning ML model to mark the detection results of the anatomical feature in the corresponding image according to the view plane of the anatomical feature visible in each image in the sequence; ii) based on the marking, obtain a first cluster of all consecutive images in the sequence corresponding to the first view plane; iii) use the first cluster as a first clip of the first view plane of the anatomical feature; repeat steps i), ii) and iii) to obtain multiple clips of different view planes of the anatomical feature; and if the first clip includes a cluster of the following consecutive images, select the first clip from the multiple clips as the preferred clip of the anatomical feature: the corresponding label for the cluster of the consecutive images is statistically more significant than other labels in the multiple clips.

[0009] According to a third aspect, there is provided a computer program product comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured such that when run by a suitable computer or processor, the computer or processor is caused to perform the method of the first aspect.

[0010] In this way, machine learning can be used in a pipelined manner to separate clips corresponding to specific view planes (e.g., view planes required for fetal cardiac US examinations) from the full video stream of US images. This can significantly reduce the manual burden on clinical users to perform US examinations.

[0011] These and other aspects will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Example embodiments will now be described, by way of example only, with reference to the following drawings, in which:

[0013] Figure 1 is an example apparatus according to some embodiments herein;

[0014] Figure 2 is an example ultrasound system according to some embodiments herein;

[0015] Figure 3 is an example method according to some embodiments herein;

[0016] Figure 4is an example flow chart according to some embodiments herein;

[0017] Figure 5 is an example flow chart according to some embodiments herein;

[0018] Figure 6 is an example flow chart according to some embodiments herein;

[0019] Figure 7 Some example correlation plots are shown. DETAILED DESCRIPTION

[0020] The disclosure herein relates to US examinations of anatomical features with periodic movement (e.g., fetal heart examinations, adult heart examinations, and examinations of a patient's vascular structure). The methods herein can be used to extract short sequences (e.g., "clips") from a sequence of US images taken as part of a US feed. In some embodiments, the clips correspond to sequences in which a constant view plane can be seen, or to individual periodic cycles. Additional embodiments describe methods for automatically extracting a fetal cardiac cycle and detecting a fetal heart rate. In short, the disclosure herein proposes an AI-based workflow in which ultrasound scans are bookmarked according to the fetal heart cycle and the bookmarks are automatically saved, which can be referred for use or saved for documentation / later analysis. This can simplify the ultrasound workflow and make ultrasound physicians' examinations faster.

[0021] In more detail, we now turn to Figure 1 In some embodiments, there is an apparatus 100 for processing a sequence of US images of anatomical features with periodic movement according to some embodiments herein. Typically, the apparatus may form part of a computer apparatus or system (e.g., a laptop, desktop computer or other computing device). In some embodiments, the apparatus 100 may form part of a distributed computing arrangement or cloud.

[0022] The apparatus includes: a memory 104 including instruction data representing a set of instructions; and a processor 102 (e.g., a processing circuit or a logic unit) configured to communicate with the memory and execute the set of instructions. Generally, the set of instructions, when executed by the processor, can cause the processor to perform any embodiment of the method 300 described below.

[0023] Embodiments of the apparatus 100 may be used to process a sequence of US images of an anatomical feature that moves periodically. More specifically, the set of instructions, when executed by the processor, causes the processor to perform the following operations: i) using a first machine learning ML model to label the detection results of the anatomical feature in the corresponding image according to the view plane of the anatomical feature visible in each image in the sequence; ii) based on the labeling, obtain a first cluster of all consecutive images in the sequence corresponding to a first view plane; and iii) use the first cluster as a first clip of the first view plane of the anatomical feature.

[0024] Processor 102 can include one or more processors, processing units, multi-core processors or modules, which are configured or programmed to control device 100 in a manner described herein. In a specific embodiment, processor 102 can include multiple software modules and / or hardware modules, which are all configured to perform or are used to perform individual steps or multiple steps of the method described herein. Processor 102 can include one or more processors, processing units, multi-core processors and / or modules, which are configured or programmed to control device 100 in a manner described herein. In some embodiments, for example, processor 102 may include multiple (e.g., interoperable) processors, processing units, multi-core processors and / or modules configured for distributed processing. Those skilled in the art will appreciate that such processors, processing units, multi-core processors and / or modules can be located in different locations and can perform different steps of the method described herein and / or different parts of a single step.

[0025] The memory 104 is configured to store program code that can be executed by the processor 102 to perform the methods described herein. Alternatively or additionally, one or more memories 104 may be external to the device 100 (i.e., separate from or remote from the device 100). For example, one or more memories 104 may be part of another device. The memory 104 can be used to store a sequence of US images, a first machine learning model, a first clip, and / or any other information or data received, calculated, or determined by the processor 102 of the device 100 or from any interface, memory, or device external to the device 100. The processor 102 may be configured to control the memory 104 to store a sequence of US images, a first machine learning model, a first clip, and / or any other information or data.

[0026] In some embodiments, the memory 104 may include multiple sub-memories, each of which may store one instruction data. For example, at least one sub-memory may store instruction data representing at least one instruction in the set of instructions, and at least one other sub-memory may store instruction data representing at least one other instruction in the set of instructions.

[0027] will realize that Figure 1 Only the components required for illustrating this aspect of the present disclosure are shown, and in actual embodiments, the device 100 may include additional components for the components shown. For example, the device 100 may also include a display. The display may include, for example, a computer screen (e.g., a liquid crystal display (LCD)) and / or a screen on a mobile phone or tablet computer. The device may also include a user input device (e.g., a keyboard, a mouse, or other input devices that enable a user to interact with the device), such as to provide an input to be used in the method described herein. The device 100 may include a battery or other power source for powering the device 100 or a module for connecting the device 100 to a mains power supply.

[0028] In some embodiments, the apparatus is incorporated into a US imaging system.For example, a US imaging system may include the apparatus 100 and a display for displaying a sequence and / or a first clip of US images.

[0029] The US imaging system may also include other components (e.g., those associated with obtaining and processing US image data). Figure 2 An example US imaging system 200 is shown in FIG. The US system 200 includes an array transducer probe 4 having a transducer array 6 for transmitting ultrasound waves and receiving echo information. The transducer array 6 may include CMUT transducers; piezoelectric transducers (formed from materials such as PZT or PVDF); or any other suitable transducer technology. In this example, the transducer array 6 is a two-dimensional array of transducers 8 capable of scanning a 2D plane or a three-dimensional volume of a region of interest. In another example, the transducer array may be a 1D array.

[0030] The transducer array 6 may be coupled to a microbeamformer 12 that controls the reception of signals by the transducer elements. The microbeamformer is capable of at least partially beamforming signals received by sub-arrays (often referred to as "groups" or "patches") of transducers as described in U.S. Pat. Nos. 5,997,479 (Savord et al.), 6,013,032 (Savord), and 6,623,432 (Powers et al.).

[0031] In an alternative embodiment, instead of the microbeamformer 12, the transducer array may be provided by a main system beamformer ( Figure 2 ) directly operated.

[0032] The system 200 may also include a transmit / receive (T / R) switch 16 to which the microbeamformer 12 can be coupled and which switches the array between a transmit mode and a receive mode. Transmit operations to transmit ultrasound beams from the transducer array 6 are directed by a controller 18 coupled to the microbeamformer via the T / R switch 16 and a main transmit beamformer (not shown) that can receive input from user operation of a user interface or control panel 38. The controller 18 can include transmit circuitry arranged to drive the transducer elements of the array 6 (directly or via the microbeamformer) during the transmit mode.

[0033] Note that in an alternative embodiment, instead of the microbeamformer 12, the transducer array is operated directly by the main system beamformer, and the T / R switch 16 can protect the main beamformer 20 from high energy transmit signals.

[0034] In a typical line-by-line imaging sequence, the beamforming system within the probe may operate as follows. During transmit, the beamformer (which may be a microbeamformer or a main system beamformer, depending on the implementation) activates a transducer array or a subaperture of a transducer array. A subaperture may be a one-dimensional transducer row or a two-dimensional transducer patch within a larger array. In transmit mode, the focusing and steering of the ultrasound beam generated by the array or a subaperture of the array is controlled as described below.

[0035] When a backscattered echo signal is received from the object, the received signal is subjected to receive beamforming (described below) in order to align the received signal and, in the case of a subaperture, then shift the subaperture, for example, by one transducer element. The shifted subaperture is then activated, and the process is repeated until all transducer elements of the transducer array have been activated.

[0036] For each line (or sub-aperture), the total received signal for the associated line used to form the final ultrasound image will be the sum of the voltage signals measured by the transducer elements of the given sub-aperture during the receive period. After the beamforming process below, the resulting line signal is typically referred to as radio frequency (RF) data. Each line signal (RF data set) generated by the various sub-apertures is then subjected to additional processing to generate the lines of the final ultrasound image. The change in the amplitude of the line signal over time will contribute to the change in the brightness of the ultrasound image with depth, where high amplitude peaks will correspond to bright pixels (or sets of pixels) in the final image. Peaks that appear near the beginning of the line signal will represent echoes from shallow structures, while peaks that gradually appear later in the line signal represent echoes from structures at increasing depths within the object.

[0037] One of the functions controlled by the transducer controller 18 is the direction in which the beam is steered and focused. The beam can be steered straight ahead (orthogonal to the transducer array) from the transducer array, or at different angles for a wider field of view. The steering and focusing of the transmit beam can be controlled based on the transducer element actuation time.

[0038] In general ultrasound data acquisition, two methods are distinguished: plane wave imaging and "beam steering" imaging. The two methods are distinguished by the presence of beam forming in the transmit mode ("beam steering" imaging) and / or the presence of beam forming in the receive mode (plane wave imaging and "beam steering" imaging).

[0039] First, let's look at the focusing function. By activating all transducer elements simultaneously, the transducer array generates a plane wave that diverges as it travels through the object. In this case, the beam of ultrasound waves remains unfocused. By introducing a position-dependent time delay into the activation of the transducer, the wavefront of the beam can be made to converge at a desired point, known as the focal zone. The focal zone is defined as a point where the lateral beam width is less than half the transmit beam width. In this way, the lateral resolution of the final ultrasound image is improved.

[0040] For example, if the time delay causes the transducer elements to be activated in series, starting with the outermost elements and finishing at the center element(s) of the transducer array, then the focal zone will be formed at a given distance away from the probe in line with the center element(s). The distance of the focal zone from the probe will vary depending on the time delay between each subsequent round of transducer element activation. After the beam passes through the focal zone, the beam will begin to diverge, forming a far-field imaging region. It should be noted that for a focal zone positioned close to the transducer array, the ultrasound beam will diverge rapidly in the far field, resulting in beam width artifacts in the final image. Typically, the near field located between the transducer array and the focal zone shows little detail due to the large overlap in the ultrasound beam. Therefore, changing the positioning of the focal zone can significantly change the quality of the final image.

[0041] It should be noted that in transmit mode, unless the ultrasound image is divided into multiple focal zones (each of which may have a different transmit focus), only one focus may be defined.

[0042] Additionally, upon receiving echo signals from within the object, the inverse of the above process may be performed to perform receive focusing. In other words, the incoming signal may be received by the transducer elements and subjected to an electronic time delay before being passed into the system for signal processing. The simplest example of this is known as delay-and-sum beamforming. The receive focus of the transducer array may be adjusted dynamically based on time.

[0043] Turning now to the function of beam steering, by properly applying time delays to the transducer elements, a desired angle can be imposed on the ultrasound beam as it leaves the transducer array. For example, by activating a transducer on a first side of the transducer array, followed by activating the remaining transducers in a sequence ending at the opposite side of the array, the wavefront of the beam will be tilted toward the second side. The magnitude of the steering angle relative to the normal to the transducer array depends on the magnitude of the time delay between subsequent transducer element activations.

[0044] Additionally, the steered beam can be focused, where the total time delay applied to each transducer element is the sum of the focusing time delay and the steering time delay. In this case, the transducer array is called a phased array.

[0045] In the case of CMUT transducers (which require a DC bias voltage for their activation), the transducer controller 18 can be coupled to control a DC bias control 45 for the transducer array. The DC bias control 45 sets the DC bias voltage(s) to be applied to the CMUT transducer elements.

[0046] For each transducer element of the transducer array, an analog ultrasound signal (commonly referred to as channel data) enters the system through a receive channel. In the receive channel, a partially beamformed signal is generated by the microbeamformer 12 based on the channel data and then passed to the main receive beamformer 20, where the partially beamformed signals from the individual transducer tiles are combined into a fully beamformed signal (referred to as radio frequency (RF) data). The beamforming performed in each time phase can be performed as described above, or additional functions can be included. For example, the main beamformer 20 can have 128 channels, each of which receives a partially beamformed signal from a tile of dozens or hundreds of transducer elements. In this way, the signals received by thousands of transducers of the transducer array can effectively contribute to a single beamformed signal.

[0047] The beamformed received signals are coupled to a signal processor 22. The signal processor 22 can process the received echo signals in various ways, such as: bandpass filtering; decimation; separation of the I component from the Q component; and harmonic signal separation, which is used to separate linear signals from nonlinear signals so that nonlinear (higher harmonics of the fundamental frequency) echo signals returning from tissue and microbubbles can be identified. The signal processor can also perform additional signal enhancement, such as speckle reduction, signal compounding, and noise removal. The bandpass filter in the signal processor can be a tracking filter whose passband slides from a higher frequency band to a lower frequency band as the echo signal is received from an increasing depth, thereby rejecting noise at higher frequencies (which come from greater depths where there is generally no anatomical information).

[0048] The beamformer for transmission and the beamformer for reception are implemented in different hardware and can have different functions. Of course, the receiver beamformer is designed to take into account the characteristics of the transmit beamformer. Figure 2 In the illustration, for simplicity, only the receiver beamformer 12, the receiver beamformer 20 are shown. In a complete system, there would also be a transmit chain with a transmit microbeamformer and a main transmit beamformer.

[0049] The function of the microbeamformer 12 is to provide initial combining of the signals in order to reduce the number of analog signal paths. This is typically performed in the analog domain.

[0050] Final beamforming is done in the main beamformer 20 and is typically done after digitization.

[0051] The transmit and receive channels use the same transducer array 6 with fixed frequency bands. However, the bandwidth occupied by the transmit pulses can vary depending on the transmit beamforming used. The receive channel can capture the entire transducer bandwidth (this is the classical approach), or by using bandpass processing, the receive channel can extract only the bandwidth containing the desired information (e.g., harmonics of the main harmonic).

[0052] The RF signal can then be coupled to a B-mode (i.e., brightness mode or 2D imaging mode) processor 26 and a Doppler processor 28. The B-mode processor 26 performs amplitude detection on the received ultrasound signal to image structures in the body (e.g., organ tissue and blood vessels). In the case of line-by-line imaging, each line (beam) is represented by an associated RF signal, whose amplitude is used to generate a brightness value to be assigned to a pixel in the B-mode image. The exact positioning of the pixel within the image is determined by the positioning along the associated amplitude measurement of the RF signal and the number of lines (beams) of the RF signal. B-mode images of such structures can be formed in a harmonic image mode or a fundamental image mode or a combination of the two, as described in U.S. Pat. No. 6,283,919 (Roundhill et al.) and U.S. Pat. No. 6,458,083 (Jago et al.). The Doppler processor 28 processes the temporally different signals caused by tissue movement and blood flow to detect moving matter (e.g., the flow of blood cells in the image field). The Doppler processor 28 typically includes a wall filter having parameters set to pass or reject echoes returning from selected types of material within the body.

[0053] The structural and motion signals generated by the B-mode processor and the Doppler processor are coupled to the scan converter 32 and the multi-plane reformatter 44. The scan converter 32 arranges the echo signals in the spatial relationship from which they are received in the desired image format. In other words, the scan converter is used to convert the RF data from a cylindrical coordinate system to a Cartesian coordinate system suitable for displaying an ultrasound image on the image display 40. In the case of B-mode imaging, the brightness of the pixel at a given coordinate is proportional to the amplitude of the RF signal received from the location. For example, the scan converter can arrange the echo signals into a two-dimensional (2D) sector format or a pyramid three-dimensional (3D) image. The scan converter can overlay the B-mode structural image with a color corresponding to the motion at a point in the image field, where the Doppler estimated velocity is used to generate the given color. The combined B-mode structural image and color Doppler image can depict tissue motion and blood flow within the structural image field. The multi-plane reformatter converts the echoes received from points in a common plane in a volumetric region of the body into an ultrasound image of the plane, as described in U.S. Pat. No. 6,443,896 (Detmer). The volume renderer 42 converts the echo signals of the 3D data set into a projected 3D image as viewed from a given reference point, as described in US Pat. No. 6,530,885 (Entrekin et al.).

[0054] The 2D or 3D images are coupled from the scan converter 32, the multi-planar reformatter 44, and the volume renderer 42 to the image processor 30 for further enhancement, buffering, and temporary storage for display on the image display 40. The imaging processor may be adapted to remove certain imaging artifacts (e.g., acoustic shadowing (e.g., caused by strong attenuators or refraction), post-enhancement (e.g., caused by weak attenuators), reverberation artifacts (e.g., where multiple highly reflective tissue interfaces are located in close proximity), etc.) from the final ultrasound image. In addition, the image processor may be adapted to process certain speckle reduction functions to improve the contrast of the final ultrasound image.

[0055] In addition to being used for imaging, the blood flow values ​​produced by the Doppler processor 28 and the tissue structure information produced by the B-mode processor 26 are also coupled to a quantification processor 34. The quantification processor can be used to make measurements in the image. The quantification processor can receive input from a user control panel 38.

[0056] Output data from the quantization processor is coupled to a graphics processor 36 for reproducing measurement graphs and values ​​together with the image on the display 40, and for audio output from the display device 40. The graphics processor 36 can also generate graphic overlays for display with the ultrasound image. These graphic overlays can contain standard identification information (e.g., patient name), date and time of the image, imaging parameters, etc. For these purposes, the graphics processor receives input (e.g., patient name) from a user interface 38. The user interface is also coupled to the transmit controller 18 to control the generation operation of the ultrasound signal generated from the transducer array 6, and thus control the image produced by the transducer array and the ultrasound system. The transmit control function of the controller 18 is only one of the functions performed. The controller 18 also takes into account the operating mode (given by the user) in the receiver analog-to-digital converter and the corresponding required transmitter configuration and bandpass configuration. The controller 18 can be a state machine with fixed states.

[0057] The user interface is also coupled to the multi-planar reformatter 44 for selecting and controlling the planes of a plurality of multi-planar reformatted (MPR) images that may be used to perform quantitative measurements in the image fields of the MPR images.

[0058] will realize that Figure 2 The US image system illustrated in is merely an example, and the US image system may include components different from those described above.

[0059] Now go to Figure 3 , Figure 3 A computer-implemented method for processing a sequence of US images of anatomical features with periodic movement according to some embodiments herein is shown. The method 300 may be performed by an apparatus such as the apparatus 100 or the US system 200. In short, in a first step 302, the method comprises: in a first step i), using 302 a first machine learning ML model to mark the detection results of the anatomical feature in the corresponding image according to the view plane of the anatomical feature visible in each image in the sequence. In a second step, the method comprises: ii) obtaining 304 a first cluster of all consecutive images in the sequence corresponding to the first view plane based on the marking; and in a third step, the method comprises: iii) using 306 the first cluster as a first clip of the first view plane of the anatomical feature.

[0060] The sequence of ultrasound images may be part of an ultrasound feed (e.g., ultrasound video). For example, a live ultrasound feed collected in real time (or near real time) as part of an ultrasound examination. Each ultrasound image in the sequence may also be referred to as an ultrasound frame.

[0061] Ultrasound imaging systems (e.g., Figure 2 The system shown in the figure obtains images in a sequence of images. The images in the sequence of images can be two-dimensional (2D) images or three-dimensional (3D) images. The images can include image components. As used herein, "image components" refer to pixels in 2D or voxels in 3D. In an example where the image is a 2D image, the image can include pixels; in an example where the image is a 3D image, the medical image can include voxels.

[0062] The sequence of images may have been acquired as part of a US examination of an anatomical feature having periodic motion. The anatomical feature may be all or part of an organ. Examples of anatomical features having periodic motion include, but are not limited to, a fetal heart; a heart of a child, adult, or animal; or a portion of the vascular system of a fetus, child, adult, or animal. However, it will be appreciated that these are merely examples and that the methods herein are equally applicable to other anatomical features having periodicity (e.g., regularly repeating motion). In general, method 300 may be applied to any organ having pulsating motion in order to measure pulsatility. For example, the lungs are another example in which such pulsating motion can be observed.

[0063] In step 302, a first machine learning model is used to label detection results of anatomical features in each image in the sequence according to the view plane of the anatomical features visible in the corresponding image.

[0064] In other words, the first machine learning model takes as input a US image and provides as output one or more labels indicating the view plane of the anatomical features visible in the image. The view plane is a 2D plane passing through the anatomical features represented in the image.

[0065] For example, in an embodiment where the anatomical feature is a heart (e.g., a fetal heart, a child heart, an adult heart, or an animal heart), the first machine learning model can be configured to label the image according to a cardiac view plane. Example cardiac view planes include, but are not limited to: 4CH, LVOT, RVOT, 3VV, and 3VT view planes. Other cardiac view planes are described by the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) and / or the American Institute of Ultrasound in Medicine (AIUM).

[0066] In the example where the anatomical feature is another organ, the view plane may be a standard view plane associated with the organ. More generally, the view plane may be defined in any manner (eg, at an angle at which the view plane bisects the organ compared to a reference plane).

[0067] Technicians will be familiar with machine learning and machine learning processes for training machine learning (ML) models. ML is a method that allows programmers to implement programs by finding patterns in data samples. The program or model obtained by machine learning is called a machine learning model. The model can be used for various analysis tasks (e.g., classification or regression). The model is trained using training data including training examples, each training example including an example input and a corresponding "correct" real situation output. The model is trained on the training data using a machine learning process.

[0068] For example, a first machine learning model may have been trained using a training dataset comprising example input US images of anatomical features and corresponding ground-truth labels indicating view planes visible in the corresponding example input images.

[0069] The machine learning process includes processes that are run on training data to create or train a machine learning model. Thus, the machine learning process includes processes and / or instructions by which the training data is used to train the machine learning model. For example, if the machine learning model is a neural network, the model can be trained using processes such as back propagation and gradient descent.

[0070] Typically, the first machine learning model is configured to take the US image as input and provide an indication of a view plane visible in the image as output. In some examples, the first machine learning model can be a neural network model (e.g., a convolutional neural network model). The convolutional neural network model is suitable for processing image data.

[0071] Other deep learning (DL) architectures (e.g., U-net, HR-net) can also be used.

[0072] As another example, the first machine learning model can be a "You Only Look Once" (YOLO) network described in an article titled "You Only Look Once: Unified, Real-Time Object Detection" published by Redmon et al. in 2016. YOLO can be trained to label US images with labels indicating the view planes visible in each image in a sequence of US images. YOLO is suitable for processing a large number of images in an image feed (e.g., a video feed) and is therefore well suited for processing US image data in real time with efficient performance.

[0073] The first machine learning model can be set up and trained using open source libraries and the principles described therein (e.g., Scikit-Learn, “Scikit-learn: Machine Learning in Python”, Pedregosa et al., JMLR 12 (pp. 2825-2830, 2011)).

[0074] However, the skilled person will appreciate that these are merely examples and that the first machine learning model may be any type of ML model that has been trained to label US images according to view planes.

[0075] In a second step, the method comprises: ii) obtaining 304 a first cluster of consecutive images in the sequence all corresponding to a first view plane based on the marking. In other words, the first cluster of images all correspond to a specific (e.g. the same) view plane and are consecutive to each other in the sequence of images.

[0076] Therefore, frames or images belonging to the same view plane form a cluster. In this regard, the following terms can be defined:

[0077] - Intra-frame distance (L): Considering frame "i" predicted as LVOT view plane and the next frame predicted as LVOT is i+n, in which case the intra-frame distance is n.

[0078] - Thus, if the intra-frame distance of an incoming frame is less than L and when appending to a cluster, if its size, then the file is added to that cluster.

[0079] - If the intra-frame distance of an incoming frame is greater than L, it forms a new cluster and the previous clustering algorithm is discarded.

[0080] Turning now to step 306, in a third step, the method comprises: iii) using 306 the first cluster as a first montage of a first view plane of the anatomical feature. The first montage is a small video / image sequence, all images being imaged on the same view plane of the anatomical feature.

[0081] For example, in this step, method 300 may include displaying the first clip to a user on a display associated with the ultrasound system.

[0082] The method may further comprise: repeating steps i), ii) and iii) to obtain a plurality of clips of the first view plane. The plurality of clips may be displayed to the sonographer so that the sonographer can replay or view different clips (or segments) of the sequence of images of the first view plane.

[0083] The method may also include repeating steps i), ii) and iii) to obtain multiple clips of different view planes of the anatomical feature. These different clips may be displayed to the sonographer so that the sonographer can click on and thereby replay the portion of the examination corresponding to a particular view plane.

[0084] The method may further comprise: repeating steps i), ii) and iii) to obtain a plurality of clips of different view planes of the anatomical feature, and selecting a first clip from the plurality of clips as a preferred clip of the anatomical feature if the first clip comprises a cluster of consecutive images for which corresponding markings are statistically more significant than other markings in the plurality of clips. In this way, a "best" clip may be determined taking into account different view planes, the "best" clip showing the segment of the sequence of images that most clearly shows the anatomical feature. This may help sonographers to be able to replay and, for example, perform measurements in the most appropriate view plane of the acquired US image sequence.

[0085] For example, the frequency or period of anatomical motion (eg, heart rate in the case where the anatomical feature is a heart or a fetal heart) may be determined based on the preferred editing.

[0086] In the case where the anatomical feature is a fetal heart, identifying and recording the best "silhouette" / clip (a small video / image sequence) for each relevant cardiac plane for a fetal cardiac examination addresses the problem of identifying fetal cardiac planes for a fetal cardiac examination to capture the function of the fetal heart. The method 300 can be performed in real time during a fetal cardiac examination so that the best silhouette corresponding to the image sequence can be recorded in an automated manner while the user is examining the fetal heart so that these images can be used for reporting and diagnostic purposes.

[0087] exist Figure 4 This situation is illustrated in the figure. Figure 4 An embodiment of method 300 is shown in which the anatomical feature is a fetal heart. In this embodiment, a sequence of ultrasound images 402 is acquired using an ultrasound monitor 404. At 406, a second machine learning model classifies each image as an image that includes a fetal heart or an image that does not include a fetal heart. If a heart is detected, the image can be cropped to the fetal heart region. If the image includes an image of a fetal heart, the method proceeds to step 302 and the first ML model is used to mark the detection of the anatomical feature in the corresponding image according to the view plane of the anatomical feature visible in each image in the sequence (in other words, the first ML model detects the fetal heart view plane being acquired).

[0088] Figure 5It illustrates how a first cluster 504 of consecutive images corresponding to a first view plane in a sequence is obtained in step 304. In this embodiment, the start and end of a cluster are defined by images in the sequence where the view plane changes. The cluster is then saved as a first clip of the first view plane. The detected view planes are stacked to form a sequence of frames.

[0089] The coaching process may be displayed on a display associated with the ultrasound scanning system to provide the user with real-time feedback 502 regarding the plane of the fetal heart that the transducer probe is currently imaging. In this manner, real-time feedback is provided to the user.

[0090] In the case of multiple clusters and multiple cardiac cycles in each of the multiple clusters, the most statistically significant one cardiac cycle can be saved as a default. The user can also be provided with an option to select the number of cardiac cycles to be output. Feedback can be provided to the sonographer to guide the sonographer to keep the probe stable so that clusters of sufficient detection quality are obtained (e.g., until they have been obtained).

[0091] In some embodiments, cardiac cycle extraction and heart rate detection are performed if the length of the cluster is greater than a minimum threshold T. The cardiac cycle calculation depends on the length of the acquired frame sequence. The minimum threshold T can be set according to the frame rate of the cart and the heart rate of the fetus.

[0092] For example, if the fetal heart beats Bm times per minute (Bs = Bm / 60), and fps is F, then

[0093] T=N×F / Bs

[0094] Here, the factor N is the minimum number of cardiac cycles required to calculate a statistically significant result. The skilled person will appreciate that the value of N may vary depending on the circumstances, however, as an example, N may be in the range of 3-5.

[0095] Turning now to other embodiments, after steps 302, 304, and 306, a clip can be used to obtain a synchronized cardiac view silhouette (eg, a mini-clip comprising one periodically moving portion of the motion that starts and ends at a specific point in the cycle).

[0096] This can be performed in the following manner. Given a sequence of N ultrasound images from a first viewing plane (I = {I1, I2, ..., I N}), each image in the sequence is registered relative to a common frame (e.g., the first frame in the clip) using a set of planar transformations (e.g., translation, rotation, shear, affine, etc.).

[0097] Then, a minimum intensity image I is obtained (eg, created) from the first cluster of images in the first clip. min The intensity of each image component in the minimum intensity image is determined as the minimum intensity of the image component in the equivalent position in each image in the first cluster of images. For example, in the composite image, the value of each pixel / voxel is the minimum value of the equivalent pixel / voxel from the images in the first cluster of images. This operation is performed to obtain an image of persistent or stationary structure in the image.

[0098] In other words, the persistent structure image I min Captures non-moving structures in the image. This is calculated as the minimum of the intensity at each pixel in all images in the extracted frame sequence. This is the base image used to track the position of moving objects (e.g., valves).

[0099] The method may then include: by comparing each image in the clip with I min Compare and select the first image I in the first clip that represents the turning point of the periodic motion pivot As with I min The minimum or maximum intensity deviation image is used to determine I pivot A turning point is a frame where its difference with other frames is minimized or maximized.

[0100] The skilled person will be familiar with the term pivot frame, which refers to an anchor frame in a video sequence. "Global pivot frame" as used herein refers to a common starting point (or frame) relative to the entire movie clip and is not limited to only one period.

[0101] The method may then include determining a first subset of images in the first clip corresponding to one period of the periodic movement as images located between the first image and a second image representing the next successive turning point in the periodic motion. In other words, the first subset of images is selected as those images located between two successive turning points in the periodic motion (having a period corresponding to I min The maximum deviation of two consecutive images, or with I min The minimum deviation between two consecutive images.

[0102] Once a first subset of images corresponding to a single cycle of periodic motion has been obtained, a second subset of images including a single cycle of periodic motion can be obtained, the second cycle starting and ending at predefined points in the cardiac cycle, as described below. First, the image number of a third image that is at a predefined timing of the periodic motion in the clip is determined. Then, the relative positioning of the third image in the sequence relative to the first image and the second image is determined. Then, a second subset of images in the first clip is selected by selecting a second subset of images that is offset from the first subset of images by the relative positioning of the third image, the second subset of images starting and ending at the predefined timing of the motion in the clip.

[0103] In some embodiments where the anatomical feature is a heart, the above steps may be repeated for a plurality of different predefined phases of the periodic motion. This creates a clip corresponding to a cardiac cycle, wherein the clip starts and ends at different points in the cardiac cycle.

[0104] In other embodiments where the anatomical feature is a heart, the method further comprises: repeating steps i), ii) and iii) for a plurality of different view planes to obtain a plurality of single-cycle clips for display to the user, all of which are synchronized with a common cardiac phase (e.g., where the clips start and end at predefined times of cardiac motion). This ensures that each extracted cycle starts at a common predetermined cardiac phase (e.g., end-systolic phase).

[0105] In one embodiment, for a sequence of N images (I = {I1, I2, ..., I N})Execute the above steps as follows.

[0106] A global pivot frame of the cardiac cycle is determined for a given cardiac view plane. For example, at the end-systole in the 4CH view plane, the frame corresponding to the end-systole phase is selected as the global pivot frame I. pivot Among all the correlation vectors c(t), let the correlation vector with the smallest deviation be denoted as c′.

[0107] Reference c′, extract the frames capturing two consecutive cardiac cycles from the original image sequence It corresponds to an arbitrary pivot frame that is randomly seeded during the iteration process.

[0108] Based on this subsequence of frames capturing two cardiac cycles, the persistent structure image I min Calculated as all frames in the first cycle of the subsequence (I' I ={I'1,I'2,…,I' h}).

[0109] By making Imin with I' I The local reference frame I is calculated by minimizing the sum of squared errors between local , I local Represented as I local =I′ j , so that argmin j ((I min -I′ j ) 2 ),

[0110] Then, for I pivot The search space is restricted to the frame [I' j+(h / 2) ,I' j+h ], where the pivot frame is calculated as I pivot =I′ i , so that argmax i ((I min -I′ i ) 2 ), That is, by making I within a constrained time range in a cycle min with I' I I is calculated by maximizing the sum of squared errors between pivot , so that limiting similar maximum values ​​from other frames causes errors. Moreover, consider two consecutive cycles that maintain I local Continuity occurs near the end of a cycle. Then, in the original image sequence I, the pivot location is updated by shifting the peak location by the step size relative to sequence I', and the cardiac cycle is extracted using the updated pivot frame in the same manner as explained for an arbitrary pivot position. This ensures that each extracted cycle starts at the pivot location of a predetermined cardiac phase (e.g., the end-systolic phase).

[0111] In this manner, for a given cardiac view plane, the method 300 can be used to automatically extract cardiac cycles (e.g., the end-systolic phase of a 4CH view) starting from the pivot position. These contents can be used to display to the user a synchronized cardiac movie clip of one or more cardiac cycles starting from the end-systolic frame. Thus, the method herein provides a way to extract synchronized cardiac view silhouettes and display synchronized cardiac view silhouettes to a sonographer.

[0112] Turning now to other embodiments, cardiac cycle and heart rate detection may also be performed. The steps described in this section may be performed using clips obtained as described above with respect to method 300. However, it should be noted that the steps of method 300 are not essential to the subsequent steps, and the following references Figure 6The described steps may also be performed independently of the method 300. For example, the method 300 may further include converting each image in the first clip into a feature vector to obtain a sequence of feature vectors.

[0113] Each feature vector may include an encoding of a spatial pattern in the corresponding image. For example, a feature vector may include one or more features of: a histogram of oriented gradients in the corresponding image, a scale-invariant feature transform of the corresponding image, a local binary pattern of the corresponding image, or any other metric that depends on a spatial pattern in the image.

[0114] In further steps, method 300 may further include: determining a correlation between feature vectors in the sequence of feature vectors; and using the correlation to determine a third subset of images from the first clip corresponding to one period of the periodic movement.

[0115] The correlation between feature vectors may be obtained by selecting a first feature vector fp in the sequence of feature vectors, and correlating the first feature vector fp with each other feature vector in the sequence of feature vectors to obtain an N-dimensional correlation vector c, where N is the number of images in the first clip.

[0116] Peaks in the periodic motion can then be detected by finding zero crossings in each correlation vector in the 1-dimensional Laplace domain. In other words, method 300 may include detecting peaks in a periodic signal by determining zero crossings in the 1-dimensional Laplace domain of the correlation. The average number of images in a cycle of the periodic motion can then be determined based on the detected peaks. This value can be used to determine the period and / or frequency of the periodic motion, for example, if the anatomical feature is a fetal heart, it can be used to obtain the fetal heart rate in an automatic manner. Thus, in this way, the variation of the spatial pattern with respect to time can be exploited in order to detect the cardiac cycle (as the number of frames per cycle) in a sequence of US images.

[0117] Now turning to an example, assuming that there are multiple cardiac cycles in the cine cycle, the spatial variation of the cardiac image pattern occurs periodically, wherein each cycle includes a systolic phase and a diastolic phase. Given such a sequence of N ultrasound images from a static view plane (I = {I1, I2, ..., I N}), each frame in the sequence is registered to the first frame using a set of planar transformations (e.g., translation, rotation, shear, affine, etc.). Although the registered image sequence is aligned by consistent structures in the images, the image sequence still has considerable local inhomogeneities due to minor probe variations, noise, random minor movements of the fetus, etc. These anomalies are strong enough to interfere with the desired correlation between the images in the sequence.

[0118] To make the images robust to such spatial anomalies, local feature descriptors (e.g., histograms of oriented gradients, scale-invariant feature transforms, accelerated robust features, local binary patterns, etc.) are used to represent the images. This representation encodes the spatial patterns in the images with corresponding feature vectors, where each image produces one feature vector. Since all images are registered and consist of the same dimensions, all images are represented as feature vectors of the same length D. For N images in the sequence, there are N D-dimensional feature vectors consisting of {f1,f2,…,f N} means that, among them, f1 to f N These feature vectors are then correlated and temporally accumulated to detect the occurrence of repetitive patterns.

[0119] The correlation of the feature vectors is performed in a cumulative manner (which iterates T times). Cumulative correlation is a process in which several coherent correlation vectors are averaged (or summed) to form a single correlation vector, which is normalized to the scale considered. This average correlation vector represents the correlation pattern from multiple time scales, which is achieved by accumulation through averaging. The time scale is determined by the number of frames stacked to form the combined feature vector. When only one frame is considered, it is the finest scale (acquisition resolution), and by using a larger number of frames, the scale becomes coarser.

[0120] In the tth iteration (t=1,2,…,T), a seed position is randomly selected in the sequence as the pivot frame I p (where p∈{1,2,…,TK}). Then, the D-dimensional feature vector f of the pivot frame is p is associated with every D-dimensional feature vector of all other frames in the sequence, which at iteration c (t) 1 generates an N-dimensional correlation vector. (t) Each element of 1 corresponds to the pivot frame f p and any other frame in the sequence f i The relevance score of the feature vector.

[0121] In other words, any frame can be chosen as a "pivot frame". The correlation between the feature vector of the pivot frame and every other feature vector is then determined. Each correlation is a numerical value (a score), so this will produce a vector of correlation values ​​between the pivot frame and every other frame in the sequence.

[0122] Then, the second correlation vector c is calculated by correlating the D×2 dimensional feature matrix (t) 2, c (t) 2 by stacking the feature vectors f from the pivot frame and its next frame p and f p+1is formed as a two-tuple feature vector. By stacking, we construct a combined feature vector by appending the individual feature vectors of the pivot frame and its next consecutive frame. During correlation, similar combined feature vectors from every two consecutive frames are considered to determine the correlation value (score). Here, c (t) 2 denotes the two consecutive frames under consideration. Similarly, in the next step, c (t) k Denotes that k consecutive frames are considered to form a combined feature vector. This D×2-dimensional feature matrix is ​​associated with all other two-tuple feature vector combinations formed by the feature vectors of every two consecutive frames.

[0123] More generally, c (t) k represents the correlation vector of the k-th feature vector tuple in the t-th iteration. Similarly, the process is repeated K times by incrementing the tuple combination of feature vectors by one unit count each time. The K-th correlation vector c (t) K is to transform the K-tuple feature vector f p ,f p+1 ,…,f p+K is considered as a D×K dimensional feature matrix, which is correlated with all other K-tuple feature vector combinations formed by the feature vectors of each K consecutive frames in the sequence. The average of the correlation vectors of all K-tuple combinations in the t-th iteration is calculated as the final correlation vector c in the t-th iteration (t) , which is given by the formula Given.

[0124] Repeating this process for T iterations generates T correlation vectors (c (1) ,c (2) ,…,c (T) ), these T correlation vectors correspond to different randomly selected pivot frames. The iterative calculation of the random seeding of the pivot frames makes the system robust to temporal anomalies that may be caused by discontinuities in the temporal structure due to several differences (e.g., lost frames, damaged frames, motion blur, low resolution, etc.). In each of the T correlation vectors, the starting position of the frame tuple similar to the tuple formed by the pivot frame and its subsequent frames has a relatively high value, which periodically causes a corresponding peak in the value. The peaks are detected by finding the zero crossings in each correlation vector in the 1-dimensional Laplace domain. The difference between the peak locations essentially determines the number of frames between the beginning of two consecutive cardiac cycles (i.e., the number of frames in each cardiac cycle (or simply cardiac cycle)). Because there are multiple cardiac cycles in each image sequence, the average of the differences between each consecutive peak location in the t-th iteration gives the average number of frames in the cardiac cycle h (t), and the standard deviation of the differences between each successive peak position represents the quality of the calculated cardiac cycle s (t) Ideally, we would like (t) However, due to practical variability, it is often observed that s (t) The value of >0. In order to obtain a robust estimate of the cardiac period, a weighted average of the values ​​of the cardiac period obtained in each iteration is calculated. The weights are selected by a function that is inversely proportional to the standard deviation of the cardiac period measurements in each iteration, which gives a statistically consistent cardiac period estimate with respect to the number of frames per cycle. The estimated cardiac period is calculated as in, and (·) represents the product of the terms. The heart rate in beats per minute can be calculated as bpm, where F acq is the frequency of ultrasound image acquisition. Furthermore, the statistical confidence score is calculated as Among them, χ c is the confidence score, and Yes (t) The average value of c The value range of is between [0,1]. Ideally, for normal heart rate, χ c = 1. The threshold value χ is set based on experience c <τ s , χ c A lower value of indicates an abnormal heart rate. The score may also indicate an abnormality in the acquisition process (which affects the periodicity of the cardiac cycle).

[0125] exist Figure 6 The above steps are summarized in Figure 6 The steps of US image acquisition 602, image registration 604, and accumulation of temporal correlations between images in the sequence 606 are shown as described above. Figure 6 Also shown is a step 608 of weighted averaging the cardiac cycles obtained in each iteration (eg, each cluster) and a step 610 of determining the cardiac cycle and heart rate therefrom.

[0126] Figure 7 A schematic plot of the correlation vector is shown, along with a graphical depiction of the cardiac cycle and the correlation peak. This is provided as an example and for reference purposes only. Figure 7 The left part of shows a plot of the correlation vectors from one iteration. Also shown is a diagram of one cardiac cycle and the frames therein. The right diagram shows a plot of the correlation vectors from five iterations a)-e), which are used to estimate the final cardiac cycle by weighted averaging with the number of frames per cycle.

[0127] In this way, the heart / cardiac cycle can be extracted from the clip / silhouette: the operation of automatically extracting one or more heart / cardiac cycles from the detected silhouette corresponding to the view plane includes: identifying the end-diastolic frame in the cardiac cycle of all cardiac view planes (e.g., 4CH, LVOT, RVOT, 3VV and / or 3VT).

[0128] As described above, the cardiac cycles extracted from the cine loop / live scan can be used to calculate the fetal heart rate of the fetus.

[0129] Although the above examples mainly describe the application of the method 300 in fetal heart examination, those skilled in the art will appreciate that the method can also be used to solve any periodic detection problem of ultrasound.

[0130] The above-described embodiments provide, inter alia, the following: statistically robust determination of fetal heart rate using only an ultrasound image sequence; extraction of an optimal / required number of clustered ultrasound frames in an image sequence corresponding to individual cardiac cycles of the fetus; determination of heart rate anomalies that violate the periodicity of temporal variations in the spatial pattern of fetal heart ultrasound images; extraction of bookmarkable movie loops from a continuous sequence of fetal heart ultrasound images; extraction of multiple movie loops pivoted around a common cardiac phase (e.g., the end-systolic phase); and assistance in providing clues to frame indices that appear to have inconsistent cardiac cycles (which may be the result of probe / fetal movement or fetal heart anomalies).

[0131] In another embodiment, a computer program product is provided comprising a computer-readable medium having computer-readable code embodied therein, the computer-readable code being configured to cause the computer or processor to perform the method(s) described herein when executed by a suitable computer or processor.

[0132] Thus, it will be appreciated that the present disclosure also applies to computer programs, in particular computer programs on or in a carrier, suitable for putting the embodiments into practice. The program may be in the form of source code, object code, a code intermediate source and object code (e.g. in partially compiled form, or in any other form suitable for implementation of the method according to the embodiments described herein).

[0133] It will also be appreciated that such a program can have many different architectural designs. For example, the program code implementing the function of the method or system can be subdivided into one or more subroutines. Many different ways of allocating functions between these subroutines will be apparent to those skilled in the art. These subroutines can be stored together in an executable file to form a self-contained program. Such an executable file can include computer executable instructions, such as processor instructions and / or interpreter instructions (e.g., Java interpreter instructions). Alternatively, one or more or all of the subroutines in the subroutine can be stored in at least one external library file and, for example, statically or dynamically linked to the main program at runtime. The main program contains at least one call to at least one of the subroutines in the subroutine. The subroutines can also include function calls to each other.

[0134] The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a data storage device, such as a ROM (e.g., a CD ROM or a semiconductor ROM) or a magnetic recording medium (e.g., a hard disk). In addition, the carrier may be a transmissible carrier (e.g., an electrical or optical signal), which may be transmitted via an electrical or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being suitable for executing or for executing the relevant method.

[0135] Those skilled in the art can understand and implement variations of the disclosed embodiments when practicing the principles and techniques described herein by studying the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the words "one" or "an" do not exclude multiple. A single processor or other unit can implement the functions of several items recorded in the claims. Although certain measures are recorded in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used advantageously. The computer program may be stored or distributed on a suitable medium, for example, an optical storage medium or solid-state medium supplied together with other hardware or as part of other hardware, but may also be distributed in other forms (for example, distributed via the Internet or other wired or wireless telecommunications systems). Any figure mark in the claims should not be interpreted as limiting the scope.

Claims

1. A computer-implemented method of processing a sequence of ultrasound (US) images of anatomical features having periodic movement, the method comprising: Using (302) a first machine learning (ML) model, labeling detection results of the anatomical feature in each image in the sequence according to a view plane of the anatomical feature visible in the corresponding image; obtaining (304) a first cluster of all consecutive images in the sequence corresponding to a first viewing plane based on the markings; and using (306) the first cluster as a first clip of the first view plane of the anatomical feature; Characterized in that the method comprises: Repeating steps i), ii) and iii) to obtain a plurality of clips of different view planes of the anatomical feature; and The first clip is selected from the plurality of clips as a preferred clip for the anatomical feature if the first clip includes a cluster of consecutive images for which corresponding markers are statistically more significant than other markers in the plurality of clips.

2. The method according to claim 1, further comprising: The frequency of the periodic motion is determined from the preferred clips.

3. The method according to claim 1 or 2, further comprising: Determine a minimum intensity image I from the first cluster of images in the first clip min , wherein the intensity of each image component in the minimum intensity image is determined as the minimum intensity of image components in equal positions in each image in the first cluster of images.

4. The method according to claim 3, further comprising: By comparing each image in the clip with I min Compare and select a first image I in the first clip representing a turning point of the periodic motion pivot As with I min The minimum or maximum intensity deviation image is used to determine I pivot .

5. The method according to claim 4, further comprising: A first subset of images in the first clip corresponding to one period of the periodic movement is determined as images located between the first image and a second image representing a next successive turning point in the periodic motion.

6. The method according to claim 5, comprising: determining an image number of a third image occurring at a predefined time of the periodic motion in the clip; and determining a relative position of the third image in the sequence relative to the first image and the second image; and The second subset of images in the first clip is determined by selecting a second subset of images that is offset from the relative positioning of the third image with respect to the first subset of images, the second subset of images starting and ending at the predefined time of the motion in the clip.

7. The method according to claim 6, wherein: The anatomical feature is a heart, and the method further comprises: Repeating steps i), ii) and iii) for a plurality of different predefined phases of the periodic motion; and / or Steps i), ii) and iii) are repeated for a plurality of different view planes to obtain a plurality of single-cycle clips for display to a user, the plurality of single-cycle clips all being synchronized with a common cardiac phase.

8. The method according to any one of the preceding claims, further comprising: converting each image in the first clip into a feature vector to obtain a sequence of feature vectors; determining correlations between the feature vectors in the sequence of feature vectors; and The correlation is used to determine a third subset of images from the first clip that correspond to one period of the periodic movement.

9. The method according to claim 8, wherein: The feature vector includes: encoding of spatial patterns in the corresponding images; and / or One or more of the following characteristics: a histogram of oriented gradients in the corresponding image; a scale-invariant feature transform of the corresponding image; and The local binary pattern of the corresponding image.

10. The method according to claim 8, wherein: Step v) comprises: Select the first eigenvector f in the sequence of eigenvectors p ; The first eigenvector f p is correlated with each other feature vector in said sequence of feature vectors to obtain an N-dimensional correlation vector c, where N is the number of images in said first clip.

11. The method according to claim 8, wherein: In step vi), the method comprises: detecting peaks in the periodic signal by determining zero crossings in the one-dimensional Laplace domain of the correlation; and The average number of images in a period of the periodic motion is determined based on the detected peak values.

12. The method according to any one of claims 1 to 11, wherein: The anatomical feature is a fetal heart.

13. An apparatus for processing a sequence of ultrasound (US) images of anatomical features having periodic movement, the apparatus comprising: a memory including instruction data representing a set of instructions; as well as a processor configured to communicate with the memory and execute the set of instructions, wherein the set of instructions, when executed by the processor, causes the processor to perform the following operations: labeling, using a first machine learning (ML) model, detection results of the anatomical feature in each image in the sequence according to a view plane of the anatomical feature visible in the corresponding image; obtaining a first cluster of all consecutive images in the sequence corresponding to a first viewing plane based on the marking; and using the first cluster as a first clip of the first view plane of the anatomical feature; The set of instructions, when executed by the processor, causes the processor to perform the following operations: Repeating steps i), ii) and iii) to obtain a plurality of clips of different view planes of the anatomical feature; and The first clip is selected from the plurality of clips as a preferred clip for the anatomical feature if the first clip includes a cluster of consecutive images for which corresponding markers are statistically more significant than other markers in the plurality of clips.

14. An ultrasound imaging system, comprising: An ultrasonic probe (4), which is used to transmit ultrasonic waves and receive echo information; as well as An apparatus (100) for processing a sequence of ultrasound (US) images of anatomical features having periodic movement obtained based on received echo information.

15. A computer program product comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured such that when executed by a suitable computer or processor, the computer or processor is caused to perform the method according to any one of claims 1 to 12.

Images