Predicting the likelihood that an individual will have one or more diseases

JP2023552330A5Active Publication Date: 2024-12-04KONINKLIJKE PHILIPS NV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023532490
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-01-22
Filing Date
2021-11-29
Publication Date
2024-12-04
Estimated Expiration
2041-11-29

AI Technical Summary

Technical Problem

Existing ultrasound-based disease detection methods, particularly for cancers like breast and thyroid, suffer from low detection rates due to insufficiently trained sonographers and time constraints, and existing computer-implemented systems face challenges with computational complexity and accuracy.

Method used

A processing system that identifies regions of interest in multiple ultrasound images using machine learning, groups similar regions, and generates predictive indicators of disease presence by processing these groups, reducing the need for spatial alignment and computational load.

Benefits of technology

This approach enhances disease detection accuracy and reduces false positives by leveraging contextual information from multiple images, while minimizing computational effort and power requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

An apparatus for identifying the presence or absence of one or more diseases in a plurality of ultrasound images. The ultrasound images are processed to identify regions of interest within each image, each region of interest representing a portion of the ultrasound image suspected of exhibiting a disease. Similar regions of interest are grouped, and groups of similar regions are then processed using machine learning methods to predict whether they contain the disease of interest. Thus, a two-step process for identifying the disease of interest is performed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of ultrasonic imaging, and more particularly to the identification of diseases represented in ultrasonic images.

Background Art

[0002] Ultrasound is increasingly being used in the detection and diagnosis of various cancer types, such as breast cancer, thyroid cancer, and liver cancer. In particular, the early detection of cancer types using ultrasound is usually carried out by identifying diseases within ultrasound images. Unfortunately, the detection rate of early cancer types is relatively low due to several factors, such as a shortage of highly experienced sonographers and time constraints for performing ultrasound imaging screening.

[0003] Clinical Decision Support (CDS) systems that use computer-implemented ultrasound-based disease detection techniques are employed to assist clinicians when performing ultrasound image screening. Such systems can help clinicians by predicting the presence or absence of one or more diseases, thereby serving as double reading / confirmation to reduce the possibility of missed detections and increase the reliability of diagnoses.

[0004] Generally, there are two types of methods for ultrasound-based disease detection (automated or computer-implemented): traditional disease detection methods based on image processing steps and deep learning techniques. Traditional methods are generally not considered to be robust and / or flexible enough for the majority of the population as they rely on rules and specific assumptions. Deep learning techniques show excellent accuracy in object / disease detection as they do not rely as much on such strong assumptions, but are plagued by high computational complexity and processing times.

[0005] In the paper "3D tumor detection in automated breast ultrasound using deep convolutional neutral network" by LI YANFENG et al., published in Medical Physics, vol. 47, no. 11, a tumor detection method for automated breast ultrasound (ABUS) was proposed. ABUS is known as an automated ultrasound device for chest screening. ABUS uses 3D ultrasound technology to acquire operator-independent volume images of the chest, generates coronal view fragments from the acquired 3D volume, and displays such fragments. [Overview of the project] [Problems that the invention aims to solve]

[0006] Therefore, there is a continuing desire to improve the mechanisms for predicting the likelihood that an individual will have one or more diseases. [Means for solving the problem]

[0007] The present invention is defined by the claims.

[0008] According to an example of one aspect of the present invention, a processing system is provided for predicting the likelihood that an individual has one or more diseases, the processing system being configured to: acquire multiple ultrasound images of the individual; identify regions of interest in the multiple ultrasound images, each region of interest being a portion of one of the multiple ultrasound images representing an area of ​​the individual potentially having a disease; group the regions of interest from different ultrasound images of the multiple ultrasound images based on similarities between the regions of interest; and process each group of regions of interest using a machine learning method to generate a predictive indicator indicating the likelihood that the group of regions of interest contains the individual's disease. The multiple ultrasound images include time-series ultrasound images. In some embodiments, the time-series ultrasound images are a video of ultrasound images, e.g., a cineloop. Identifying regions of interest representing areas of potential disease includes identifying the presence or absence of one or more regions of interest. For some ultrasound images, there may be no regions of interest identified in each ultrasound image, and for some ultrasound images, there may be one or more regions of interest identified in each ultrasound image.

[0009] This disclosure proposes a method for identifying the presence or absence of one or more diseased areas in an individual, specifically for generating one or more predictive indicators indicating the likelihood of disease presence. An ultrasound image of a patient is processed to identify a region of interest (i.e., a portion of the ultrasound image) containing a potential disease—i.e., a candidate area predicted to exhibit a disease. Thus, a region of interest is a portion of the ultrasound image and is itself an image (but smaller in size than the ultrasound image). A region of interest contains a potential disease if it is predicted—i.e., likely—that the area of ​​the individual represented by the region of interest contains a potential disease. Mechanisms for identifying regions of interest containing potential diseases are well known to those skilled in the art and include, for example, machine learning methods, edge detection algorithms, and image segmentation techniques.

[0010] Regions of interest are then grouped or clustered based on their similarities to create groups of regions of interest. In this way, different portions of ultrasound images that correspond to each other (i.e., contain the same potential lesions) are grouped together. Therefore, regions of interest are considered similar to each other (and grouped accordingly) if they are expected to contain the same potential lesions of an individual. Furthermore, grouping or clustering is performed based on the similarity of regions of interest without requiring relative spatial relationships between them. In this way, grouping or clustering can be performed without spatially aligning multiple ultrasound images. This is particularly advantageous when multiple ultrasound images are not spatially aligned. For example, when multiple two-dimensional ultrasound images are acquired manually with a handheld ultrasound probe, the relative spatial relationships between these two-dimensional ultrasound images are unknown, unlike those acquired by ABUS (Automated Brachial Ultrasound). Conventionally, multiple ultrasound images can first be combined into a three-dimensional (3D) volume using various spatial alignment techniques (e.g., spatial registration, 3D reconstruction), and then further signal processing (e.g., disease detection) can be performed based on the spatially aligned images or fragments. However, spatial alignment or 3D can be difficult in some scenarios. In contrast, it is proposed here to group multiple regions of interest based on their similarities, without relying on prior spatial registration or alignment across multiple images, in other words, without relying on knowledge of the relative spatial relationships between multiple images or between regions of interest identified in multiple images.

[0011] The groups are then processed using machine learning methods to generate predictive indicators that indicate the likelihood of the group containing individual diseases. In this way, the machine learning methods effectively predict whether or not diseases are present in groups within a region of interest. Thus, machine learning methods, such as classifiers, generate predictive indicators by processing groups within a region of interest that contain the same potential diseases.

[0012] In relation to the present invention, a predictive indicator is any data that changes in response to a change in the predicted likelihood (determined by a machine learning method) that a group of diseases contains diseases. The predictive indicator includes binary, categorical, or numerical data. Binary data indicates a prediction about whether a group contains diseases or not (e.g., "0" indicates a predicted absence, "1" indicates a predicted presence, or vice versa). Categorical data indicates a probability category for whether a group contains diseases (e.g., "possible", "very likely", "unlikely"). Numerical data indicates a numerical probability that a group in a region of interest contains diseases, for example, on a scale of 0 to 1 or 0 to 100.

[0013] The proposed method reduces the false positive rate of identifying individual disease (e.g., from still images only) by using multiple ultrasound images, for example, ultrasound images taken over a period of time, thereby utilizing additional contextual information to identify individual disease. By first identifying the region of interest, the amount of processing performed to identify disease is reduced compared to, for example, running a machine learning process across multiple ultrasound images without further processing. Thus, the proposed method provides a mechanism for performing high-quality and accurate disease identification with less computational effort.

[0014] Preferably, each ultrasound image is a two-dimensional ultrasound image. Ultrasound systems for generating 2D ultrasound images have wide utility and adoption and are increasingly being used in resource-constrained environments, such as low power consumption and in environments with limited power (e.g., battery-powered or in areas with unreliable power). Therefore, the present invention is particularly advantageous when used to process 2D ultrasound images. Thus, reducing the computational load (minimizing power consumption requirements) while still obtaining an accurate indicator of potential disease for such systems is particularly beneficial.

[0015] In some embodiments, the processing system is configured to process each group of regions of interest by performing a process that includes generating a sequence of regions of interest using the regions of interest of the group of regions of interest, and using a machine learning method to process the sequence of regions and predict whether or not the sequence of regions contains individual diseases.

[0016] A sequence of regions of interest effectively forms a single data structure containing data from all regions of interest within the group.

[0017] The processing system is configured to generate a sequence of regions by performing steps that include stacking regions of interest. If the regions of interest are extracted from a two-dimensional image, for example, a two-dimensional ultrasound image, this process effectively forms a pseudo-3D volume.

[0018] The processing system is configured to identify regions of interest in ultrasound images by performing a process that includes processing each ultrasound image using a second machine learning method to identify the region of interest. Thus, at least two machine learning methods are used to predict the presence or absence of disease. This approach increases the likelihood of identifying potential disease and facilitates a fully automated mechanism for employing existing and well-developed mechanisms for identifying potential disease in ultrasound images, as well as for improving reliability.

[0019] In some cases, a third machine learning method is used to group regions of interest, for example, using the characteristics and / or other metadata (e.g., location, size, shape, etc.) of each region of interest. Of course, even if machine learning methods are not used to identify regions of interest, they can still be used to group them.

[0020] In some preferred examples, multiple ultrasound images comprise a sequence of ultrasound images. The sequence of ultrasound images is preferably chronological, such that later ultrasound images in the sequence are acquired by the imaging system later in time than earlier ultrasound images. This effectively results in time-dependent information being incorporated into a group of regions of interest. This disclosure recognizes that the use of time-based information increases the accuracy of identifying the presence or absence of one or more diseased areas.

[0021] Preferably, multiple ultrasound images are acquired, for example, by the same ultrasound imaging system using the same ultrasound imaging probe. More preferably, multiple images are acquired while the ultrasound imaging probe (used to acquire ultrasound images) remains stationary. This enhances the ease and accuracy of identifying groups of linked regions of interest.

[0022] In some embodiments, the order of ultrasound images in a sequence depends on the time each ultrasound image was captured. For example, a sequence of ultrasound images includes sequential frames of an ultrasound video. The context provided by a temporally ordered sequence of ultrasound images (e.g., an ultrasound video) allows time-based information to be considered for machine learning methods when predicting the presence or absence of one or more diseased organisms in a group of regions of interest. This disclosure identifies that this information is particularly advantageous in reducing the false-positive detection rate of one or more diseased organisms in ultrasound images.

[0023] In some examples, within each group of regions of interest, each region of interest is derived from ultrasound images that are sequentially adjacent to ultrasound images of other regions of interest within the same group of regions of interest.

[0024] Preferably, each region of interest is no more than 0.4 times the size of the ultrasound image, for example, no more than 0.25 times the size of the ultrasound image. The smaller the size of the regions of interest, the greater the effect of reducing the computational complexity of processing a group of regions of interest (rather than the entire ultrasound image).

[0025] In some examples, the processing system is configured to display a visually perceptible output on a display device in response to each prediction indicator. Thus, information in response to the prediction indicator can be output to a user (e.g., a clinician).

[0026] When at least one prediction indicator indicates that at least one group of regions of interest may contain a disease, the processing system is configured to: identify the location of the disease identified with respect to the displayed ultrasound image of the patient, and overlay a visually perceptible output on the displayed ultrasound image in response to the identified location of the disease.

[0027] The processing system is configured to group regions of interest by performing a process that includes: determining a similarity measure between different regions of interest from different ultrasound images, the similarity measure representing the size of the overlap between the relative area occupied by one region of interest within an ultrasound image and the relative area occupied by another region of interest of another ultrasound image, and grouping two different regions of interest into the same group in response to a similarity measure between the two different regions of interest that exceeds a predetermined threshold.

[0028] Thus, the similarity measurement is based on the size of the overlap, e.g., the IoU (Intersection over Union) measure between regions of interest from different ultrasound images. This facilitates a simple mechanism for identifying whether the same potential disease is being identified, e.g., when the same potential disease is likely to be in the same relative location in each ultrasound image. This approach is particularly advantageous when multiple ultrasound images include frames from an ultrasound video, as this embodiment increases the likelihood that the potential disease will be in the same location between different frames of the ultrasound video.

[0029] The processing system is configured to group regions of interest by performing a process that includes: identifying metadata of regions of interest that provides information about the size, location, reliability and / or occurrence of the regions of interest; and grouping regions of interest based on the region of interest metadata. The additional information is then used to determine similarities between different regions of interest. In some embodiments, machine learning methods are used to group regions of interest, for example, based on the region of interest metadata.

[0030] A computer-based method for predicting the likelihood of an individual having one or more diseases is also proposed.

[0031] The computer-based method includes: acquiring multiple ultrasound images of an individual; identifying regions of interest within the multiple ultrasound images, where each region of interest is a portion of one of the multiple ultrasound images representing an area with potential disease in the individual; grouping regions of interest from different ultrasound images based on similarities between them; and using machine learning methods to process each group of regions of interest and generate a predictive indicator that the group of regions of interest is likely to contain disease in the individual.

[0032] A processing system is also proposed, comprising memory and a processor connected to the memory and configured to perform such a computer implementation.

[0033] A computer program product is also proposed that, when executed on a computing device having a processing system, includes computer program code means that causes the processing system to execute all of the steps of any computer implementation described herein.

[0034] Those skilled in the art can easily adapt any processing system described herein to carry out any method described herein, and vice versa.

[0035] These and other aspects of the present invention will become apparent and be described with reference to embodiments described later.

[0036] For a better understanding of the present invention and to more clearly illustrate how the present invention is carried out, the accompanying drawings are provided herein by reference only as examples. [Brief explanation of the drawing]

[0037] [Figure 1] This figure shows an ultrasonic system for use in one embodiment. [Figure 2] This diagram shows a workflow for understanding the embodiment in context. [Figure 3] This figure shows a method according to one embodiment. [Figure 4] This figure shows a method according to one embodiment. [Figure 5] This diagram shows a method for grouping areas of interest. [Figure 6] This figure shows a processing system according to one embodiment. [Figure 7] This figure shows the effects of the embodiment. [Figure 8] This is a diagram showing the processing system. [Modes for carrying out the invention]

[0038] The present invention will be described with reference to the figures.

[0039] Please understand that the detailed descriptions and specific examples, while illustrating exemplary embodiments of the apparatus, systems, and methods, are intended solely for illustrative purposes and do not limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, systems, and methods of the invention will be better understood from the following description, the appended claims, and the appended drawings. Please understand that the figures are for illustrative purposes only and are not drawn to exact proportions. Please also understand that the same reference numerals are used to indicate the same or similar parts across multiple figures.

[0040] The present invention provides a mechanism for determining the probability / likelihood of the presence or absence of one or more diseases in an individual by processing multiple ultrasound images of that individual. The ultrasound images are processed to identify each region of interest within each image, and each region of interest represents a portion of the ultrasound image suspected to show a disease. Similar regions of interest are grouped together, and the groups of similar regions are then processed using machine learning methods to generate predictions as to whether they are likely to contain the disease in question. Thus, a two-step process is performed to identify the likelihood of disease.

[0041] Embodiments of the present invention are based on the recognition that important information is lost or absent when identifying potential disease using only a single ultrasound image, but that processing multiple ultrasound images simultaneously significantly increases the processing load of the image analysis process. Instead, a new method is proposed for assessing the likelihood of the presence or absence of potential disease, in which groups of regions of interest from different ultrasound images are formed and processed together to predict the likelihood of disease presence.

[0042] The embodiments are used in any suitable ultrasound analysis system, for example, in ultrasound analysis for cancer screening and diagnosis of surface organs. The proposed embodiments may be used in any suitable clinical setting, for example, during screening for cancer screening, or in the ultrasound department of a hospital for cancer screening and diagnosis. The present invention is applicable to all ultrasound imaging systems having advanced disease identification software for online or offline use, i.e., processing of data produced from memory or directly from the ultrasound system.

[0043] First, the general operation of an exemplary ultrasonic system will be described with reference to Figure 1. While the present invention utilizes ultrasonic images generated by such an ultrasonic system, other methods and systems for generating ultrasonic images will be apparent to those skilled in the art.

[0044] The system comprises an array transducer probe 4 having a transducer array 6 for transmitting ultrasound and receiving echo information. The transducer array 6 includes piezoelectric transducers formed from materials such as CMUT transducers, PZT, or PVDF, or any other suitable transducer technology. In this example, the transducer array 6 is a two-dimensional array of transducers 8 capable of scanning either a two-dimensional plane or a three-dimensional volume of the region of interest. In another example, the transducer array is a one-dimensional array.

[0045] The transducer array 6 is connected to a microbeamformer 12 that controls the reception of signals by the transducer elements. The microbeamformer has the capability to beamform at least a portion of the signals received by a subarray of transducers (commonly referred to as a “group” or “patch”), such as those described in U.S. Patent No. 5,997,479 (Savord et al.), No. 6,013,032 (Savord), and No. 6,623,432 (Powers et al.).

[0046] It should be noted that the microbeamformer is entirely optional. Furthermore, the system includes a transmit / receive (T / R) switch 16 to which the microbeamformer 12 may be connected and to switch the array between transmit and receive modes, protecting the main beamformer 20 from high-energy transmit signals when the microbeamformer is not used and the transducer array is operated directly by the main system beamformer. The transmission of the ultrasonic beam from the transducer array 6 is directed by a transducer controller 18 connected to the microbeamformer by the T / R switch 16 and a main transmit beamformer (not shown), which can receive input from user operation on the user interface or control panel 38. The controller 18 may include transmit circuits positioned to move the transducer elements of the array 6 (directly or via the microbeamformer) during transmit mode.

[0047] In a typical line-by-line imaging sequence, the beamforming system within the probe operates as follows: During transmission, the beamformer (which may be a microbeamformer or main system beamformer, depending on the implementation) activates the transducer array or sub-apertures of the transducer array. The sub-apertures are either one-dimensional lines of transducers or two-dimensional patches of transducers within a larger array. In transmission mode, the focusing and guiding of the ultrasonic beam generated by the array or sub-apertures of the array are controlled as described below.

[0048] When a backscattered echo signal is received from the target, the received signal undergoes received beamforming (as described below) to align the received signal, and if a sub-aperture is used, the sub-aperture is then shifted, for example, by one transducer element. The shifted sub-aperture is then activated, and this process is repeated until all transducer elements of the transducer array are activated.

[0049] For each line (or sub-aperture), the total received signal used to form the associated lines in the final ultrasound image is the sum of the voltage signals measured by the transducer element of a given sub-aperture during the reception period. The resulting line signals, following the beamforming process described below, are typically referred to as radio frequency (RF) data. Each line signal (RF data set) generated by the various sub-apertures then undergoes additional processing to generate the lines in the final ultrasound image. Changes in the amplitude of the line signals over time contribute to changes in the brightness of the ultrasound image with depth, where high amplitude peaks correspond to brighter pixels (or collections of pixels) in the final image. Peaks appearing near the beginning of the line signal represent echoes from shallow structures, while peaks appearing later in the line signal represent echoes from structures at increasing depths within the object.

[0050] One of the functions controlled by the transducer controller 18 is the direction in which the beam is guided and focused. The beam is guided straight ahead of the transducer array (perpendicular to the transducer array) or at different angles to a wider field of view. The guidance and focusing of the transmitted beam are controlled as a function of the transducer element operating time.

[0051] Two methods can be distinguished in general ultrasound data acquisition: plane-wave imaging and "beam-guided" imaging. The two methods are distinguished by the presence of beamforming in the transmission ("beam-guided" imaging) and / or reception modes (plane-wave imaging and "beam-guided" imaging).

[0052] First, looking at the focusing function, by activating all transducer elements simultaneously, the transducer array generates a plane wave that branches as it passes through the target. In this case, the ultrasound beam remains unfocused. By introducing a position-dependent time delay to the activation of the transducers, it is possible to converge the beam's wavefront at a desired point, called the focusing zone. The focusing zone is defined as the point where the transverse beamwidth is less than half the transmit beamwidth. In this way, the lateral resolution of the final ultrasound image is improved.

[0053] For example, if a time delay causes the transducer elements to activate sequentially, starting with the outermost element and ending with the central element of the transducer array, a focusing zone will be formed at a given distance from the probe, according to the central element. The distance of the focusing zone from the probe will vary depending on the time delay between each subsequent sequence of transducer element activations. After the beam passes through the focusing zone, it will begin to split, forming a far-field imaging region. Note that for focusing zones located close to the transducer array, the ultrasonic beam will split rapidly in the far field, resulting in beam width artifacts in the final image. Typically, the near-field, located between the transducer array and the focusing zone, shows little detail due to significant overlap in the ultrasonic beam. Therefore, changing the location of the focusing zone can lead to significant changes in the quality of the final image.

[0054] Please note that in transmission mode, only one focus is defined unless the ultrasound image is divided into multiple focus zones (each with a different transmission focus).

[0055] In addition, when an echo signal is received from within the target, it is possible to perform the reverse of the process described above in order to perform receive focusing. In other words, the input signal is received subject to an electron time delay before it is passed to the system for signal processing by the transducer elements. The simplest example of this is called delay and sum beamforming. It is possible to dynamically adjust the receive focusing of the transducer array as a function of time.

[0056] Looking at the beam guiding function here, it is possible to convey a desired angle relative to the ultrasonic beam as it exits the transducer array through the precise application of time delays to the transducer elements. For example, the wavefront of the beam will be bent toward the second side by the remaining transducers in a sequence ending on the opposite side of the array activating the transducers on the first side of the subsequent transducer array. The size of the beam guiding angle relative to the perpendicular of the transducer array depends on the size of the time delay between subsequent transducer element activations.

[0057] Furthermore, it is possible to focus the guided beam, so that the total time delay applied to each transducer element is the sum of the time delays for focusing and guiding. In this case, the transducer array is referred to as a stepped array.

[0058] For CMUT transducers that require a DC bias voltage for activation, the transducer controller 18 may be connected to control the DC bias control 45 of the transducer array. The DC bias control 45 sets the DC bias voltage applied to the CMUT transducer elements.

[0059] For each transducer element in the transducer array, an analog ultrasonic signal, typically referred to as channel data, enters the system via a receiving channel. In the receiving channel, a partially beamformed signal is fabricated from the channel data by a microbeamformer 12 and then passed to the main receiving beamformer 20, where the partially beamformed signals from individual patches of transducers are combined into a fully beamformed signal, referred to as radio frequency (RF) data. The beamforming performed at each stage is carried out as described above, or includes additional functions. For example, the main beamformer 20 has 128 channels, each receiving partially beamformed signals from patches of tens or hundreds of transducer elements. In this way, the signals received by thousands of transducers in the transducer array can efficiently contribute to a single beamformed signal.

[0060] The beamformed received signal is connected to the signal processor 22. The signal processor 22 can process the received echo signal in various ways, such as: band-pass filtering, decimation, I and Q component separation, and harmonic signal separation, which acts on separate linear and nonlinear signals to enable identification of nonlinear (higher harmonics of the fundamental frequency) echo signals returned from tissue and microbubbles. The signal processor also performs additional signal enhancements such as speckle reduction, signal synthesis, and denoising. The band-pass filter in the signal processor can be a tracking filter, having its passband, which shifts from higher frequency bands to lower frequency bands as the echo signal is received from increasing depth, thereby removing noise at higher frequencies from greater depths that typically do not contain anatomical information.

[0061] Transmitting and receiving beamformers can be implemented with different hardware and have different functions. Of course, the receiver beamformer is designed to take into account the characteristics of the transmitting beamformer. In Figure 1, for clarity, only the receiver beamformers 12 and 20 are shown. In a complete system, there would also be a transmitting chain with transmitting microbeamformers and a main transmitting beamformer.

[0062] The function of the microbeam former 12 is to provide an initial combination of signals to reduce the number of analog signal paths. This is typically done in the analog domain.

[0063] The final beamforming takes place in the main beamformer 20, usually after digitization.

[0064] The transmit and receive channels use the same transducer array 6 having a fixed frequency band. However, the bandwidth occupied by the transmit pulse can vary depending on the transmit beamforming used. The receive channel can capture the entire transducer bandwidth (this is the traditional method), or, by using band-pass processing, it can extract only the bandwidth containing the desired information (e.g., harmonics of the main harmonics).

[0065] The RF signal is then connected to a B-mode (i.e., luminance mode, or 2D imaging mode) processor 26 and a Doppler processor 28. The B-mode processor 26 performs amplitude detection on the received ultrasound signal for imaging of structures within the body, such as organ tissues and blood vessels. In the case of line-by-line imaging, each line (beam) is represented by an associated RF signal, and the amplitude of the associated RF signal is used to generate a luminance value that will be assigned to a pixel in the B-mode image. The precise location of a pixel in the image is determined by the location of the associated amplitude measurement along the RF signal and the number of lines (beams) of the RF signal. B-mode images of such structures are formed in harmonic or fundamental imaging modes, or in a combination of both as described in U.S. Patent No. 6,283,919 (Roundhill et al.) and U.S. Patent No. 6,458,083 (Jago et al.). The Doppler processor 28 processes temporally distinct signals resulting from tissue movement and blood flow for the detection of moving matter, such as the flow of blood cells in the image field. The Doppler processor 28 typically includes a wall filter with parameters set to pass or reject echoes returned from a selected type of material within the body.

[0066] The structural and motion signals generated by the B-mode and Doppler processors are connected to a scan converter 32 and a multi-section reformatter 44. The scan converter 32 arranges the echo signals in the spatial relationships in which they were received in the desired image format. In other words, the scan converter acts to convert the RF data from a cylindrical coordinate system to a Cartesian coordinate system suitable for displaying the ultrasound image on the image display 40. In the case of B-mode imaging, the brightness of a pixel at a given coordinate is proportional to the amplitude of the RF signal received from that location. For example, the scan converter arranges the echo signals in a two-dimensional (2D) sector format or a pyramidal three-dimensional (3D) image. The scan converter can superimpose the B-mode structural image with a color corresponding to the motion at a point in the image field, where the Doppler estimated velocity is supposed to produce a given color. The combined B-mode structural image and color Doppler image depict the motion and blood flow of tissue in the structural image field. The multi-plane reformatter converts echoes received from points in a common plane within the volume region of the body into ultrasound images of that plane, as described in U.S. Patent No. 6,443,896 (Detmer). The volume renderer 42 converts the echo signals of the 3D dataset into projected 3D images as viewed from a given reference point, as described in U.S. Patent No. 6,530,885 (Entrekin et al.).

[0067] 2D or 3D images are connected from the scan converter 32, multi-section reformatter 44, and volume renderer 42 to the image processor 30 for further enhancement, buffering, and temporary storage for display on the image display 40. The imaging processor is adapted to remove certain imaging artifacts from the final ultrasound image: for example, acoustic shadowing caused by strong attenuation or refraction, for example, posterior enhancement caused by weak attenuation, for example, when highly reflective tissue interfaces are located in close proximity, and reflection artifacts. In addition, the image processor is adapted to handle certain speckle reduction functions to improve the contrast of the final ultrasound image.

[0068] In addition to being used for imaging, the blood flow values ​​generated by the Doppler processor 28 and the tissue structure information generated by the B-mode processor 26 are connected to the quantification processor 34. The quantification processor generates measurement results for different flow conditions, such as blood flow volume and flow rate, in addition to structural measurement results such as organ size and gestational age. The quantification processor receives input from the user control panel 38, for example, points in the anatomical tissue of the image to be measured.

[0069] The output data from the quantification processor is connected to the graphics processor 36 for the reproduction of measurement graphics and values ​​in the image on the display 40, and for audio output from the display device 40. The graphics processor 36 can also generate an image overlay for display with the ultrasound image. These image overlays may include a standard for identifying information such as the patient name, the date and time of the image, and imaging parameters. For these purposes, the graphics processor receives input from the user interface 38, for example, the patient name. The user interface is also connected to the transmit controller 18 to control the generation of ultrasound signals from the transducer array 6, and thus the images produced by the transducer array and ultrasound system. The transmit control function of the controller 18 is only one of the functions it performs. The controller 18 also takes into account the mode of operation (given by the user) and the corresponding required transmitter and bandpass configurations in the receiver analog-to-digital converter. The controller 18 can be a state machine with fixed states.

[0070] The user interface is also connected to a multi-section reformatter 44 for plane selection and control of multiple MPR images used to perform quantified measurements in the image field of the multi-section reformatting (MPR) images.

[0071] This disclosure relates to a process for analyzing multiple ultrasound images, for example, those generated by the aforementioned ultrasound images. This process is performed by an ultrasound image processor, for example, a graphics processor 36, an image processor 30, and / or a separate / dedicated processing system (not shown).

[0072] Figure 2 schematically shows workflow 200 to understand the method employed by one embodiment of the present invention. A workflow represents a process performed by the processing system.

[0073] The workflow involves processing multiple ultrasound images 210, all of the same resolution. The multiple ultrasound images are individual ultrasound images, in particular ultrasound images of the same anatomical area of ​​the individual (and preferably acquired from the same or similar viewpoints with, for example, a largely stationary imaging probe). Preferably, each ultrasound image is a two-dimensional image, but this is not mandatory (for example, a 3D ultrasound image may be used).

[0074] An ultrasound image is any image obtained using an ultrasound imaging process, for example, using an ultrasound imaging system such as the one described above. In some examples, the "initial" ultrasound image undergoes additional processing (e.g., using one or more filters and / or one or more layers of a neural network) before being used as an ultrasound image for the purpose of the method / workflow. Thus, an ultrasound image is a feature space image or an image space ultrasound image.

[0075] In a particularly preferred example, the multiple ultrasound images include a sequence of ultrasound images, for example, a sequence of ultrasound images captured by the same ultrasound imaging probe at different times. Preferably, the order of the ultrasound images in the sequence depends on the time each ultrasound image was captured. For example, the sequence includes sequential frames of ultrasound video.

[0076] Each of the multiple ultrasound images is processed in process 220 to identify regions of interest within each ultrasound image. A region of interest is a portion or fragment of an ultrasound image (i.e., not the entire ultrasound image) that represents an area of ​​an individual that has / contains a potential disease.

[0077] A diseased area includes one or more of the following appropriate features resulting from an individual injury, disease, or illness: a tumor, a growth, an abscess, a nodule, swelling, a lump, an ulcer, or any other suitable feature. Mechanisms for identifying a region of interest in an ultrasound image are well known to those skilled in the art and include, for example, machine learning methods, edge detection algorithms, and image segmentation techniques.

[0078] The regions of interest 230 then undergo a grouping or clustering process 240, where similar regions of interest are grouped together. In other words, regions of interest are grouped (for example, into one or more groups of regions of interest) based on their similarity. Similarity of regions of interest is defined based on the similarity of the content of the regions of interest. In particular, regions of interest are grouped if they are predicted to identify the same potential disease in an individual.

[0079] In the example shown, each group 250 of the region of interest is then processed in process 260 using machine learning methods to generate a predictive indicator 270 that indicates the likelihood that the group of the region of interest contains individual disease.

[0080] The predictive indicator 270 includes binary, categorical, or numerical data representing the likelihood that a group contains a disease. For example, the predictive indicator is a binary indicator (e.g., "0" or "1") indicating whether a group in the area of ​​interest contains a disease. Another example is the predictive indicator being the probability (i.e., a numerical indicator) that a group in the area of ​​interest contains a disease. The numerical indicator may be on a scale of 0 to 1, 0 to 10, 1 to 10, 0 to 100, or 1 to 100 (although other examples may be used). Yet another example is the predictive indicator being a categorical indicator indicating the categorical likelihood that a group in the area of ​​interest contains a disease (e.g., "likely," "unlikely," "very likely," "neither likely nor unlikely").

[0081] Accordingly, the proposed method performs a two-step process to predict the likelihood of disease presence in an individual using multiple ultrasound images. Each ultrasound image is processed individually to identify a region of interest, which is an area / part of the ultrasound image that potentially depicts disease (i.e., a region that is a candidate containing disease). Similar regions of interest are then grouped together and processed using machine learning methods to generate predictive indicators in response to the likelihood that the group exhibits disease. Processing using groups effectively confirms or rejects the proposal that each region of interest within the group exhibits disease (a proposal suggested by processing each individual ultrasound image).

[0082] The proposed method eliminates the need to perform complex, high-intensity processing of all ultrasound images simultaneously by processing each individual ultrasound image individually before processing the group of ultrasound image portions. The inventors recognized that the proposed method can significantly reduce the false positive rate while maintaining a high accuracy / precision in predicting disease potential.

[0083] Now that an overview of the workflow has been provided, a more complete example of how to execute the illustrative workflow is given below.

[0084] Figure 3 shows method 300 according to one embodiment. Method 300 is carried out by a processing system according to one embodiment.

[0085] Method 300 includes step 310 of acquiring multiple ultrasound images of an individual. The multiple ultrasound images are as previously described with reference to Figure 2. For ease of explanation, the illustrated ultrasound images are image-spatial ultrasound images (i.e., each pixel directly represents a portion of the individual's anatomical tissue).

[0086] Specifically, the ultrasound images are preferably two-dimensional ultrasound images, but three-dimensional ultrasound images can also be used. Preferably, the multiple ultrasound images include a sequence of ultrasound images in which, for example, ultrasound images sequentially after or later than one ultrasound image are captured at a later point in time than the other ultrasound images. In some examples, the multiple ultrasound images include (sequential) frames from an ultrasound video.

[0087] The ultrasound image is then processed in step 320, which includes identifying one or more regions of interest in the ultrasound image, where each region of interest is a portion of the ultrasound image representing an area with potential disease in the individual. Thus, a region of interest is a portion of a larger ultrasound image and is itself an ultrasound image.

[0088] The process for identifying regions of interest in ultrasound images is readily apparent to those skilled in the art, and involves using machine learning methods (e.g., neural networks or Naive Bayes classifiers) to identify any portion of an ultrasound image that contains or represents a potential disease in an individual. Typically, a region of interest is defined by identifying a rectangle or volume (or its coordinates) that defines the outer boundary of the area or volume containing the individual's potential disease. Thus, each region of interest is a specific portion of the ultrasound image.

[0089] Preferably, each region of interest is no more than 0.4 times the size of the ultrasound image, for example, no more than 0.25 times the size of the ultrasound image. The smaller the region of interest, the greater the reduction in computational complexity (compared to the entire ultrasound image) for processing the group of regions of interest.

[0090] Regions of interest are processed to remove or delete any overlapping regions of interest (e.g., regions of interest in the same ultrasound image that identify the same potential disease). This is done, for example, by processing each ultrasound image to identify any regions of interest in that ultrasound image that overlap with each other more than a predetermined amount (e.g., have an IoU value greater than a predetermined value), and then deleting one or more of the overlapping regions of interest. The deleted regions of interest are those associated with the lowest confidence / probability of identifying the potential disease. Such confidence / probability values ​​may be generated when regions of interest are identified using machine learning techniques, but other techniques can also generate such confidence / probability measures.

[0091] Method 300 then performs step 330 of grouping or clustering the identified regions of interest to form groups of regions of interest. This grouping is performed based on the similarities between the identified regions of interest to one another. Each group is fitted / configured to include regions of interest that describe / have the same potential disease.

[0092] Step 330 is performed by linking or associating regions of interest from different ultrasound images. Linked regions of interest are regions of interest believed to contain the same potential disease. Step 330 then creates or forms groups of regions of interest by grouping or clustering the linked regions of interest into a single group, for example, if they meet certain requirements.

[0093] Preferably, each group contains only one region of interest from any given ultrasound image (i.e., each ultrasound image can contribute up to one region of interest to a group of ultrasound images).

[0094] In a particularly preferred example where multiple ultrasound images comprise a sequence of ultrasound images, each group of regions of interest contains only regions of interest from sequentially adjacent ultrasound images. In other words, each region of interest (within a particular group of regions of interest) originates from ultrasound images (in a sequence of ultrasound images) that sequentially adjaculate to ultrasound images containing other regions of interest (within a particular group of regions of interest).

[0095] This method means that groups of regions of interest can only be constructed by comparing regions of interest from sequentially adjacent (i.e., immediately preceding or succeeding) ultrasound images in order to effectively construct a sequence of regions of interest. This provides additional information useful for subsequent analysis of the groups of regions of interest, creating groups of regions of interest that are sequentially related to each other (e.g., temporally related).

[0096] This method has the advantages of providing a group of regions of interest that are sequential to one another, providing useful contextual information for subsequent analysis (for example, sequence information provides an important indicator of whether or not disease is present), and reducing the number of comparisons that need to be performed (only regions of interest in sequentially adjacent ultrasound images need to be compared to each other).

[0097] In other words, step 330 is performed by sequentially linking or associating adjacent ultrasound images before creating a group of regions of interest by grouping linked regions of interest into a single group.

[0098] In the first scenario, step 330 includes determining a similarity measure between different regions of interest based on the relative overlap between the areas occupied by each region of interest in their respective ultrasound images. If this similarity measure exceeds some predetermined threshold, the regions of interest are grouped (or linked / associated) with one another.

[0099] For example, the size of the overlap between an area occupied by one region of interest in one ultrasound image and an area occupied by another region in a different ultrasound image can be used as a similarity measure. The size of the overlap is determined using the IoU evaluation metric. This procedure can establish links between regions of interest across different ultrasound images.

[0100] As just one example, the region of interest in the first ultrasound image is in coordinates (X1, Y1, X2, Y X Consider a scenario where the region of interest of the second ultrasound image is a rectangle defined by coordinates (X3, Y3, X4, Y4). Since the coordinate information defines the scope or extent of each region of interest, this coordinate information can be used to identify the size of the relative overlap between the two regions of interest and to calculate the IoU measure.

[0101] In some cases, if the IoU measure is greater than some predetermined value, preferably 0.4 or greater, for example 0.5 or greater, for example 0.6 or greater, then the regions of interest are considered sufficiently similar (i.e., suitable to be placed in the same group).

[0102] In the second scenario, step 330 includes grouping the regions of interest based on metadata associated with each region of interest.

[0103] This metadata, for example, represents the features of the region of interest. Exemplary metadata for a region of interest may include the location of the region of interest (e.g., the location of the center of the region of interest), its geometry (e.g., its shape) and / or its size, its confidence value (e.g., a value representing the reliability or probability that the region of interest contains potential disease – when provided by a machine learning method) and / or its feature map.

[0104] A feature map of a region of interest is a map generated by applying one or more filters, such as a convolutional filter, to the region of interest. Specifically, a feature map is a map generated by applying one or more layers of a neural network to the region of interest. The meaning of the term "feature map" is well known in the field of machine learning.

[0105] In some examples, a classifier, such as a machine learning method, processes each domain of interest and classifies it into one of several classifications. A group of domains of interest might include only those domains that share the same classification.

[0106] Preferably, a combination of the methods described above is used. Specifically, for multiple images formed from multiple images in a single sequence, each group of regions of interest includes only regions of interest from ultrasound images that are sequentially adjacent to and overlap each other by a predetermined amount or more with one or two other ultrasound images that provide one or more other regions of interest to the group of regions of interest.

[0107] Method 300 includes step 340 of using a machine learning method to process a group of regions of interest and generate a predictive indicator that the group of regions of interest is likely to contain individual diseases.

[0108] In a particularly preferred example, the output of the machine learning method is data in the form of binary indicators, categorical indicators, and / or numerical indicators, which changes in response to the predicted likelihood that a group of regions contains a disease.

[0109] This process uses multiple, similar regions of interest to predict whether they contain or depict individual disease. This provides additional, contextually relevant information for generating predictive indicators, compared to simply processing a single ultrasound image, for example. Thus, this method can provide a more accurate mechanism for determining the likelihood of disease presence (in each group of regions of interest) because additional contextual information is provided.

[0110] By using groups of regions of interest, the proposed method also avoids the need to process groups of full-size ultrasound images, which would otherwise be computationally expensive and / or require significant additional memory space.

[0111] It should also be noted that the proposed method uses a two-step process to identify or predict the likelihood of disease presence, thereby improving the accuracy of identifying individual diseases.

[0112] Step 340 involves processing each group of regions of interest to generate a sequence of regions of interest. The sequence is time-series such that the regions of interest are sequentially ordered based on the time when the first ultrasound image (of each region of interest) was first captured. The sequence of regions is then processed using machine learning methods to generate predictive indicators.

[0113] If the sequence of regions of interest is obtained from a sequence of ultrasound images, the order of the regions of interest will correspond to the order of the sequence of ultrasound images from which they are obtained.

[0114] This sequence in the region of interest is called a "tube" or "tubule" and effectively represents a sequence (part of the image) depicting the same potential disease. For example, a "tube" represents a video of the potential disease (rather than a video of the entire area imaged by the ultrasound imaging probe) that is sized / shaped to fit around the potential disease.

[0115] In some examples, a sequence of regions of interest includes stacks of regions of interest within a group of regions of interest. For example, if each region of interest is a 2D image, the sequence of regions of interest includes a 3D volume representing a simple stacking of the 2D images on top of each other.

[0116] To put it another way, groups of regions of interest are effectively treated as "volumes" that can be classified or processed using machine learning methods. That is, a sequence of regions of interest is a single data structure containing stacked regions of interest, i.e., all combinations of regions of interest.

[0117] The input to a machine learning method is a group of regions themselves, a sequence of regions of interest, and / or other data derived from a group of regions of interest, such as one or more feature maps derived by processing a group of regions of interest (e.g., using one or more filters or layers of a neural network).

[0118] A machine learning algorithm is any self-training algorithm that processes input data to produce or predict output data. For step 340, the input data includes data derived from a group of regions of interest, and the output data includes predictive indicators that show the likelihood that the group of regions of interest contains individual diseases.

[0119] Suitable machine learning algorithms for use in the present invention will be apparent to those skilled in the art. Examples of suitable machine learning algorithms include decision tree algorithms and artificial neural networks (e.g., CNNs, RNNSs, and / or LSTMs). Other machine learning algorithms such as logistic regression, support vector machines, or naive Bayesian models are suitable alternatives.

[0120] The structure of an artificial neural network (or simply a neural network) is inspired by the human brain. A neural network consists of layers, each containing multiple neurons. Each neuron contains mathematical operations. Specifically, each neuron contains differently weighted combinations of a single type of deformation (for example, the same type of deformation, such as a sigmoid, but with different weights). In the process of processing input data, the mathematical operations of each neuron are performed on the input data to produce a numerical output, and the output of each layer in the neural network is sequentially fed to the next layer. The final layer provides the output.

[0121] Methods for training machine learning algorithms are well known. Typically, such methods involve obtaining a training dataset containing training input data entries and corresponding training output data entries (commonly labeled as "ground truth" data). An initialized machine learning algorithm is applied to each input data entry to generate predicted output data entries. The error between the predicted output data entries and the corresponding training output data entries is used to refine the machine learning algorithm. This process can be repeated until the errors converge, and the predicted output data entries are sufficiently similar to the training output data entries (e.g., ±1%). This is commonly known as supervised learning.

[0122] For example, when a machine learning algorithm is formed from a neural network, the mathematical operations (or their weights) of each neuron are corrected until the error converges. Known methods for correcting neural networks include gradient descent and backpropagation algorithms.

[0123] Training input data entries correspond to example data (e.g., the group itself) derived from a group of interest. Training output data entries correspond to example predictions (in the form of binary, categorical, or numerical data) about the likelihood of disease presence in the group.

[0124] Method 300 further includes step 350 of displaying a visually perceptible output on a display device in response to a predictive indicator.

[0125] Therefore, step 350 includes controlling the display device to provide a visually perceptible output in response to a predictive indicator (for each group of regions), i.e., to indicate the predicted probability that the disease is present in the individual. The indicator provides, for example, a predicted probability that the disease is present in the individual and / or a binary indicator (e.g., showing "disease present" or "disease absent").

[0126] As another example, if the predictive indicator is a numerical value indicating a predicted probability, and one of the predictive indicators has a value exceeding a predetermined value, then step 350 includes controlling the display device to indicate that a disease is present.

[0127] In some examples, a visually perceptible output indicates which of the group's regions of interest (if any) are predicted to contain or depict the disease. This can take the form of a probability and / or binary indicator, for example, for each group of regions or only for those regions that are considered likely to contain the disease (e.g., those that are likely to exceed some predetermined threshold).

[0128] In some examples, in response to determining that at least one of a group of regions of interest contains a disease, for example, if the predicted probability exceeds some threshold, step 350 includes identifying the location of the identified disease with respect to a displayed ultrasound image of the patient and superimposing a visually perceptible output (e.g., annotation, rectangle, etc.) onto the displayed ultrasound image in response to the identified location of the disease. The location of the identified disease is the location of the region of interest (of the group of regions of interest predicted to contain the disease) in the displayed ultrasound image. Specifically, the visually perceptible output covers the location of the disease in the ultrasound image, for example, at the location of the region of interest predicted to contain the disease. The ultrasound image is one of several ultrasound images containing the region of interest that formed part of the group of regions of interest in which the presence of the disease was identified.

[0129] In some examples, all regions of interest are identified (e.g., using their respective markers), and regions of interest within a group of regions of interest predicted to contain disease (e.g., having a probability greater than some predetermined value) are highlighted or otherwise emphasized, for example, by a specific color, pattern, etc. This facilitates the easier identification of potential areas where disease may be found (e.g., for diagnostic purposes), while also providing information on the automated prediction of the likelihood that an area contains disease.

[0130] In some examples, information about the predicted likelihood of an area containing the target disease is displayed, for example, using shading or transparency adjustment techniques.

[0131] In some examples, the display device is configured to sequentially display each ultrasound image from a group of ultrasound images (e.g., to play an ultrasound video). Step 350 includes controlling the display device to provide a visually perceptible output of the location of the region of interest (from the group of regions of interest) related to the currently displayed ultrasound image with respect to the currently displayed ultrasound image, in response to determining that at least one of the groups of regions of interest may contain a disease (e.g., by processing a predictive indicator). In this way, the relative locations of diseases related to the group of regions of interest can be sequentially tracked or displayed.

[0132] Figures 2 and 3 are used to illustrate a relatively simple embodiment in which each of the multiple ultrasound images includes an image-spatial ultrasound image.

[0133] However, as mentioned above, in some examples, each of the multiple ultrasound images includes a feature space ultrasound image, i.e., a feature space image. The feature space image is a “conventional” ultrasound image (“image space image”) that has undergone further processing, for example, generating a feature space image using one or more filters and / or layers of a neural network.

[0134] In some cases, these feature spatial images are processed in the same manner as the ultrasound images described above. However, in other cases, some additional processing is performed to identify regions of interest (or groups thereof).

[0135] Figure 4 shows a process in which multiple ultrasound images include multiple feature space images 410, each of which is derived from its respective image space (ultrasound) image 420 (for example, as a result of a feature extraction process 430).

[0136] Therefore, the step of acquiring multiple ultrasound images includes acquiring multiple feature spatial ultrasound images. This is done by processing the multiple image spatial ultrasound images in the feature extraction process 430 to generate a feature spatial ultrasound image.

[0137] The process of identifying a region of interest in an ultrasound image includes independent processing of the feature space ultrasound image to identify the region of interest in the feature space ultrasound image.

[0138] In an alternative embodiment, the process includes a process 435 for identifying a region of interest in the image-spatial ultrasound image 420. This is performed using any of the aforementioned mechanisms, for example, by machine learning methods. The region of interest in the feature-spatial ultrasound image is then determined based on the region of interest in the image-spatial ultrasound image, as schematically shown using dotted lines. Specifically, the spatial relationships between different areas of the feature-spatial ultrasound image and the image-spatial ultrasound image are known and can therefore be used to identify the relevant region of interest in the feature-spatial ultrasound image.

[0139] Therefore, the step of identifying a region of interest in an ultrasound image includes identifying the region of interest by first identifying the region of interest in the initial ultrasound image (used to derive the multiple ultrasound images), before identifying the region of interest in multiple ultrasound images based on the identified region of interest in the initial ultrasound image.

[0140] Multiple regions of interest in (feature space) ultrasound images are then grouped in process 440 to generate one or more groups 460 of regions of interest.

[0141] Each group 450 of the region of interest is processed using a machine learning method to generate a predictive indicator 470 that indicates the likelihood of the group of the region of interest containing disease. This is done using any of the aforementioned methods.

[0142] Figure 5 illustrates an exemplary process for grouping areas of interest.

[0143] Specifically, Figure 5 illustrates the process for grouping regions of interest from a sequence of ultrasound images (e.g., sequential frames of an ultrasound video).

[0144] Figure 5 shows three ultrasound images, all two-dimensional and having the same resolution. Specifically, there are the first ultrasound image 510, the second ultrasound image 520, and the third ultrasound image 530. The ultrasound images form a (temporal) sequence of ultrasound images, for example, representing frames of an ultrasound video.

[0145] Each ultrasound image has been processed, for example, using machine learning methods, to identify regions of interest. These regions of interest are schematically shown in Figure 5 using rectangles.

[0146] Regions of interest are then grouped. This is done by linking or associating regions of interest in adjacent frames if the measure of similarity (in adjacent frames) of the regions of interest exceeds some predetermined threshold. The measure of similarity is the IoU value, calculated by dividing the size of the overlap between regions (i.e., the size of the overlap if the regions were located in the same ultrasound image) by the size of the union of the regions.

[0147] A sequence of regions of interest is then acquired by selecting linked regions of interest from different ultrasound images. Specifically, an acquired sequence of regions of interest is a sequence containing up to one region of interest from each ultrasound image, where each region of interest in the sequence is linked or associated with another region of interest in the sequence. The order of the regions of interest in a sequence corresponds to the order of the ultrasound images in the sequence from which they were acquired.

[0148] The regions placed in the sequence of regions of interest are then deleted (the sequence of regions of interest is preserved), and the process is repeated. This process is schematically illustrated using the arrows in Figure 5.

[0149] Figure 6 shows a processing system 600 according to one embodiment. The processing system is described in relation to the overall ultrasound imaging system 60.

[0150] The processing system 600 is configured to acquire multiple ultrasound images of an individual. These multiple ultrasound images are acquired, for example, from a memory 610 and / or an ultrasound scanner 620 configured to generate ultrasound images.

[0151] The processing system 600 is also configured to identify regions of interest in the ultrasound image, each region of interest being a portion of the ultrasound image representing an area in an individual that has potential disease.

[0152] The processing system 600 is also configured to group regions of interest from different ultrasound images based on similarities between regions of interest, and to process each group of regions of interest using machine learning methods to predict whether or not the group of regions of interest contains an individual's disease.

[0153] The processing system 600 is also configured to display a visually perceptible output on a display device in response to each predictive indicator.

[0154] As stated above, the processing system 600 is appropriately adapted to carry out any of the methods described herein with necessary modifications, and a person skilled in the art can easily adapt the processing system 600 appropriately.

[0155] Figure 7 shows the effectiveness of the proposed method for predicting whether an individual has one or more diseases.

[0156] The first graph, 710, shows the recall accuracy curve (recall on the x-axis, accuracy on the y-axis) for accurately identifying disease in a two-dimensional ultrasound image using a known two-dimensional neural network analysis technique. This graph was generated using benchmark data. Specifically, the first graph, 710, shows the recall accuracy curve resulting from the use of a disease detection algorithm using a fast RCNN.

[0157] The second graph 720 shows a recall accuracy curve for accurately identifying diseased tissue in multiple ultrasound images by performing the technique disclosed herein. The second graph 720 was generated using the same benchmark data used to produce the first graph 710.

[0158] The first and second graphs 710 and 720 have the same scale ratio (for example, 0.0 to 1.0 with respect to the x-axis and 0.0 to 1.0 with respect to the y-axis).

[0159] The proposed method allows for higher accuracy values ​​to be obtained for a higher level of reproducibility, and vice versa. In other words, an improvement on our invention demonstrates that the use of additional context provided in multiple ultrasound images in the disclosed manner (e.g., using ultrasound video), where context is not present in simple still images, can reduce false positive and / or missing detections of disease.

[0160] Figure 8 shows a schematic diagram of a processing system 600 according to an embodiment of the present disclosure. As shown, the processing system 600 includes a (data) processor 860, a memory 864, and a communication module 868. These elements communicate with each other directly or indirectly, for example, via one or more buses.

[0161] The processor 860 includes a central processing unit (CPU), a digital signal processor (DSP), an ASIC, a controller, an FPGA, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 860 may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In some embodiments, the processor is a distributed processing system, for example, in the form of a set of distributed processors.

[0162] Memory 864 includes cache memory (e.g., the cache memory of processor 860), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory devices, hard disk drives, other forms of volatile and non-volatile memory, or combinations of different types of memory. In one embodiment, memory 864 includes a non-temporary computer-readable medium that stores instructions. For example, memory 864, or a non-temporary computer-readable medium, has program code recorded therein, the program code includes instructions for causing a processing system 600, or one or more components of the processing system 600, in particular processor 860, to perform the operations described herein. For example, the processing system 600 can perform the operation of method 700. Instructions 866 are also referred to as code or program code. The terms “instruction” and “code” should be interpreted broadly to include any type of computer-readable statement. For example, the terms “instruction” and “code” refer to one or more programs, routines, subroutines, functions, procedures, etc. “Instruction” and “code” include a single computer-readable statement or a number of computer-readable statements. Memory 864, in which code is recorded, is referred to as a computer program product.

[0163] The communication module 868 may include any electronic and / or logic circuits to facilitate direct or indirect data communication between the processing system 600, the penetration device, and / or the user interface (or other further devices). In this sense, the communication module 868 can be an input / output (I / O) device. In some cases, the communication module 868 facilitates direct or indirect communication between the processing circuit 600 and / or various elements of the system (Figure 6).

[0164] It will be understood that the disclosed methods are preferably computer implementations. As such, the concept of a computer program is also proposed, which includes computer program code for implementing any described method when the computer program is executed on a processing system, for example, a computer or a set of distributed processors.

[0165] Different parts, lines, or blocks of code in a computer program according to one embodiment are executed by a processing system or computer to perform any method described herein. In some alternative implementations, the functions shown in a block diagram or flowchart occur in an order other than that shown in the diagram. For example, two blocks shown consecutively are actually executed substantially simultaneously, or they are sometimes executed in reverse order depending on the associated functionality.

[0166] This disclosure proposes a computer program (product) that includes instructions causing a computer or processing system to perform any of the methods (steps) described herein when the program is executed by the computer or processing system. The computer program (product) may be stored on a non-temporary computer-readable medium.

[0167] Similarly, a computer-readable (storage) medium is also proposed that includes instructions causing a computer or processing system to perform any of the methods (steps) described herein when executed by the computer or processing system. A computer-readable data carrier in which the aforementioned computer program (product) is stored is also proposed. A data carrier signal that performs the aforementioned computer program (product) is also proposed.

[0168] Modifications to the disclosed embodiments may be understood and brought about by those skilled in the art in carrying out the claimed invention, based on a study of the drawings, this disclosure, and the appended claims. In the claims, the word “including” does not exclude other elements or steps, and singular elements do not exclude plural elements. The mere fact that certain measures are stated in different dependent claims does not imply that combinations of these measures cannot be used to one's advantage.

[0169] The methods disclosed are preferably computer-based methods and are carried out by a suitable processing system. Just as any method described herein is adapted to perform a process carried out by any processing system described herein, any processing system described herein is adapted to appropriately carry out any method described herein.

[0170] Computer program products are also proposed that include computer program code means that cause a processing system to perform all the steps of any method described herein when executed on a computing device having a processing system.

[0171] A single processor or other unit may perform some of the functions described in the claims. Computer programs may be stored / distributed in a suitable medium, for example, an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, for example, via the Internet or other wired or wireless telecommunications systems.

[0172] The mere fact that certain measures are stated in different dependent claims does not indicate that a combination of these measures cannot be used to one's advantage. Where the term “adapted” is used in the claims or description, it is intended to be equivalent to the term “configured.” Any reference numerals in the claims should not be construed as limiting the scope.

Claims

1. 1. A processing system for predicting the likelihood that an individual has one or more diseases, comprising: acquiring a plurality of ultrasound images of the individual, the plurality of ultrasound images comprising a time series of ultrasound images; identifying regions of interest within the plurality of ultrasound images, each region of interest being a portion of one of the plurality of ultrasound images that represents an area of ​​the individual having a potential disease; grouping regions of interest from different ones of the ultrasound images based on similarities of the regions of interest; processing each group of areas of interest using machine learning techniques to generate a predictive indicator of the likelihood that said group of areas of interest contains a disease condition for said individual; A processing system.

2. The processing system of claim 1 , wherein each ultrasound image is a two-dimensional ultrasound image.

3. The processing system, for each group of regions of interest, generating a sequence of regions of interest using the regions of interest of the group of regions of interest; processing the sequence of the region using machine learning techniques to generate a predictive indicator of the likelihood that the sequence of the region contains a disease in the individual; 3. The processing system of claim 1, further comprising: a processing system for processing each group of regions of interest by performing a process comprising:

4. The processing system of claim 3 , wherein the processing system generates a sequence of regions by performing a step including stacking the regions of interest.

5. 5. The processing system of claim 1 , wherein the processing system identifies regions of interest in the ultrasound images by performing a process including processing each ultrasound image using a second machine learning method to identify regions of interest.

6. The processing system of claim 1 , wherein the plurality of ultrasound images comprises a video of ultrasound images.

7. The processing system of claim 6 , wherein the order of the ultrasound images in the sequence is dependent on the time each ultrasound image was captured.

8. 8. The processing system of claim 6 or 7, wherein in each group of regions of interest, each region of interest is derived from an ultrasound image that is sequentially adjacent to an ultrasound image of another region of interest in the same group of regions of interest.

9. The processing system of claim 1 , wherein each region of interest is no greater than 0.25 times the size of the ultrasound image.

10. The processing system of claim 1 , wherein the processing system is configured to display a visually perceptible output on a display device in response to each predictive indicator.

11. If the processing system detects that at least one predictive indicator indicates that at least one group of regions of interest is likely to contain disease, Identifying a location of the identified lesion with respect to a displayed ultrasound image of the patient; and superimposing a visually perceptible output on the displayed ultrasound image in response to the identified location of the disease. The processing system of claim 10.

12. the processing system comprising: determining a similarity measure between different regions of interest from different ultrasound images, said similarity measure representing a size of overlap between a relative area occupied by one region of interest in an ultrasound image and a relative area occupied by another region of interest in another ultrasound image; grouping the two different regions of interest into the same group in response to the similarity measure between the two different regions of interest exceeding a predetermined threshold; The processing system of claim 1 , further comprising:

13. An ultrasound scanner for generating a plurality of ultrasound images of an individual, including a time series of ultrasound images; 12. A processing system according to claim 1 for predicting the likelihood that said individual has one or more diseases based on said plurality of ultrasound images.

1. An ultrasound imaging system comprising:

14. 1. A computer-implemented method for predicting the likelihood that an individual has one or more diseases, the computer-implemented method comprising: acquiring a plurality of ultrasound images of the individual, the plurality of ultrasound images comprising a time series of ultrasound images; identifying regions of interest within the plurality of ultrasound images, each region of interest being a portion of one of the plurality of ultrasound images that represents an area of ​​the individual having a potential disease; grouping regions of interest from different ones of the ultrasound images based on similarities of the regions of interest; processing each group of areas of interest using machine learning techniques to generate a predictive indicator of the likelihood that said group of areas of interest contains a disease in said individual; 4. A computer-implemented method comprising:

15. A computer program comprising computer program code means which, when executed on a computing device having a processing system, causes said processing system to perform all of the steps of the computer implemented method according to claim 14.