Defining Label Locations in Medical Images

Abstract

A mechanism for defining the location of arbitrary labels in an ultrasound image. One or more label zones are identified by processing the ultrasound image. Each label zone is a contiguous region whose pixels have intensity values ​​that vary by less than a predetermined amount. Each label is positioned according to the location of the identified label zone.

Description

[Technical Field]

[0001] The present invention relates to the field of medical imaging. [Background technology]

[0002] Labeling any structures and / or features visible in a medical image is a common requirement in medical imaging procedures for the purpose of reducing errors in the analysis and facilitating the evaluation of any structure / feature within the medical image.

[0003] There has been increasing development and use of automated detection techniques for detecting or segmenting anatomical features and / or components within medical images. Such techniques typically generate segmentation information that identifies the boundaries of one or more regions within the medical image, as well as label information that provides a label or identifier for each identified region. Summary of the Invention [Problem to be solved by the invention]

[0004] An ongoing problem is how to properly define label locations automatically, i.e., without user input. Improper placement of label locations can lead to problems when analyzing medical images (e.g., if the label may be misattributed to the wrong region), or to occlusion of clinically relevant information in the medical image (e.g., if the label obfuscates clinically relevant features represented in the medical image).

[0005] One existing approach is to simply place the label at the centroid or center point of the relevant region, which reduces the chance of confusing the label of another region.

[0006] An improved approach for defining the location of labels for segmented regions within medical images is desired. [Means for solving the problem]

[0007] The invention is defined by the independent claims, the dependent claims defining advantageous embodiments.

[0008] According to an example according to one aspect of the present invention, there is provided a computer-implemented method for defining locations of one or more labels relative to one or more first regions in a medical image of a subject, the method comprising: receiving an ultrasound image, the medical image defining an intensity value for each of a plurality of pixels; obtaining segmentation information identifying boundaries of the one or more first regions by receiving the segmentation information or by generating the segmentation information from the medical image; for each of one or more first regions in the medical image, obtaining label information identifying a label for the region by receiving the label information or by generating the label information from the medical image; if present, identifying one or more label zones in the medical image, each label zone representing a contiguous portion of the medical image whose pixels have intensity values ​​that vary by less than a predetermined amount; defining a location for each label of one or more first regions identified in the label information based on the identified one or more label zones; It has.

[0009] This disclosure proposes techniques for placing labels for segmented regions of a medical image, where for each region, a label is positioned responsive to the location of one or more label zones identified within the medical image.

[0010] In particular, a label zone can represent a region having pixels with intensity values ​​that vary by less than a predetermined amount (i.e., have substantially uniform intensity values). It is recognized that relevant anatomical structures or elements typically have pixel intensity values ​​that vary by a significant amount. Thus, each label zone represents a zone or region that is less likely than other regions to block or obfuscate clinically relevant anatomical features or structures for the desired clinical test. Alternatively, a label zone can represent a region / zone not relevant to the desired clinical test. This avoids or reduces the likelihood that a label will be located within the relevant region. For example, each label zone can represent a contiguous portion of a medical image that is not included in either one or more first regions or one or more predetermined second regions not relevant to the desired clinical test.

[0011] The proposed approach thereby reduces the likelihood that labels will be placed in locations that obfuscate clinically relevant anatomical features or structures. The proposed approach thereby assists clinicians in confidently evaluating and / or analyzing medical images (e.g., to make a clinician's decision) by reducing the likelihood that potentially clinically relevant material will be blocked or obscured.

[0012] The medical image may be, for example, an ultrasound image of a patient or other subject.

[0013] The one or more first regions of the subject may be, for example, one or more anatomical regions of the subject. The one or more anatomical regions may be one or more predetermined regions, such as one or more regions associated with a desired clinical test.

[0014] The generation of segmentation information may be implemented, for example, using any existing or future developed segmentation technique, or a combination thereof.

[0015] In the context of the present invention, the location of a label is considered to be the location at which the label is located. The location of a label may be defined at a specific location relative to the label itself, for example the center of the label or a specific corner of the label.

[0016] A label is a textual or symbolic (e.g., numerical) annotation for a particular region. A label facilitates semantic identification of the region and / or identification of a property of the region (e.g., a measurement of a particular parameter of the region, such as its size).

[0017] Preferably, for each label zone, the pixels in the label zone have intensity values ​​that vary by less than a predetermined amount. It has been noted previously that regions or areas of uniform intensity are unlikely to contain clinically significant regions. This approach thereby provides a mechanism for easily identifying regions or adjacent regions where a label can be placed.

[0018] In some examples, in each label zone, the pixels of the label zone have intensity values ​​that are below a global threshold intensity value.

[0019] In some examples, if present, processing the medical image to identify one or more label zones includes, for each region of the one or more first regions in the medical image, processing the region to identify as a label zone any subregions of the region each representing a contiguous portion of the region whose pixels have intensity values ​​that vary by less than a predetermined amount.

[0020] In some examples, defining the position of each label of the one or more first regions includes, for each region of the one or more first regions, defining the position of the label of the region in response to the positions of any identified sub-regions of the region.

[0021] Optionally, for each region, each pixel in any identified subregion has an intensity value below a region-specific threshold intensity value derived from the minimum intensity value of any pixel in the region. This approach positions the label relative to the darkest regions / portions of the region. Dark regions are less likely to contain elements or features of clinical interest, thereby better positioning the label to reduce / avoid obscuring any potentially relevant anatomical features / elements.

[0022] More specifically, this approach results in the labels being positioned relative to anechoic regions or zones of the region when the medical image is an ultrasound image. Anechoic regions are regions that do not contain anatomical elements or features that reflect ultrasound.

[0023] In at least one example, each sub-region represents a portion of the region having a size equal to or greater than a predetermined percentage of the size of the region.

[0024] The predetermined percentage may be 10% or more, preferably 20% or more.

[0025] The step of defining the position of each label of the one or more first regions may include, for each region of the one or more first regions, identifying any label zones that overlap the region as overlapping label zones, and defining the positions of the labels of the region in response to the positions of any overlapping label zones.

[0026] The failure to define the location of each label for the one or more first regions may include a failure to define the location of the label for the region in response to an inability to identify, for each region of the one or more first regions, any overlapping label zones, if any, in response to the locations of a predetermined number of closest label zones to the region. The predetermined number may be one.

[0027] Defining a position of each label of the one or more first regions may include, for each region of the one or more first regions, defining a position of the label of the region that is further responsive to a centroid of the region.

[0028] The step of defining a position of each label of the one or more first regions may include defining, for each region of the one or more first regions, a position of the label of the region that is further responsive to the centroid of any other regions, if present.

[0029] In some examples, each label zone is configured such that when the ellipse fits within the boundary of the label zone, the eccentricity of the ellipse is less than a predetermined eccentricity, which may be, for example, a value of 0.995 or less, such as 0.99 or less, such as 0.95 or less.

[0030] This approach aims to avoid identifying subregions that are in the form of very thin and / or elongated shapes, which are not suitable for label positioning, for example, since the label is likely to turn out to be larger than the width and / or height of such a subregion.

[0031] Also provided is a computer program product or non-transitory storage medium comprising computer program code or instructions which, when executed by a processor, cause the processor to perform any of the methods disclosed herein.

[0032] A processor for defining the location of one or more labels in a medical image of a subject for a desired clinical examination is also provided, the processor being configured to perform the methods disclosed herein.

[0033] A medical imaging system is provided that includes an imaging system for generating medical images and a processor as disclosed herein coupled to the imaging system.

[0034] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0035] For a better understanding of the present invention and to show more clearly how it may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which: [Brief explanation of the drawings]

[0036] [Figure 1] 1 is a flowchart illustrating a method according to one embodiment. [Figure 2] 10 is a flowchart illustrating a method according to a further embodiment. [Figure 3] An example of a medical image is shown. [Figure 4] An example of a medical image is shown. [Figure 5] An example of a medical image is shown. [Figure 6] An example of a medical image is shown. [Figure 7] An example of a medical image is shown. [Figure 8] An example of a medical image is shown. [Figure 9] 1 illustrates a processor according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0037] The present invention will now be described with reference to the drawings.

[0038] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the present invention will become better understood from the following description, appended claims, and accompanying drawings. It should be understood that the figures are schematic only and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the figures to indicate the same or similar parts.

[0039] The present invention provides a mechanism for defining the location of any label in a medical image. One or more label zones are identified by processing the medical image. Each label zone is a region of the medical image that is not associated with a desired clinical examination and / or whose pixels have intensity values ​​that vary by less than a predetermined amount. Each label is positioned according to the location of the identified label zone.

[0040] The embodiments are based on the recognition that improper positioning of labels in a medical image can significantly affect the readability or interpretability of the medical image. Therefore, it would be advantageous to position the labels in such a way that interpretability is not affected. The proposed approach overcomes this by proper positioning of the labels.

[0041] The disclosed approach can be used to control the position of labels for any form of medical image, particularly ultrasound images.

[0042] 1 shows a method 100 for defining the location of one or more labels for a medical image. The method 100 may be performed by a processor, for example, by the processor 910 shown in FIG.

[0043] The method 100 can be conceptually divided into an acquisition process 110 (sometimes called an initialization process), a label zone identification process 120, and a label positioning process .

[0044] The acquisition process 110 includes acquiring 111 a medical image. The medical image defines an intensity value for each of a number of pixels. The intensity value typically represents the brightness of the image and can be in the range [0, 255] or [0, 511].

[0045] A medical image may be a 2D image or a 3D image. The term "pixel" refers to the smallest unit of space represented by a medical image and is considered interchangeable with the term "voxel" for 3D images.

[0046] Suitable examples of medical images are well known in the art and may include ultrasound images, magnetic resonance images, CT images, X-ray images, etc. It will be understood that a medical image is an image produced using medical imaging techniques that provides a visual representation of at least the interior of a subject.

[0047] Step 111 may involve, for example, retrieving or receiving medical images from a database or memory system, or from a medical imaging system.

[0048] The acquisition process 110 also includes acquiring 112 segmentation information that identifies the boundaries of one or more regions within the medical image.

[0049] Each region of / in a medical image may be a region of the medical image that is predicted or determined to contain a particular structure or feature, such as an anatomical feature / structure (e.g., an organ or part of an organ) or an interventional device (e.g., a catheter or pacemaker). In a preferred example, the boundary of a region may be a predicted boundary of the structure or feature predicted to be contained within the region, e.g., a shape predicted to bound the relevant structure or feature. Alternatively, the boundary of a region may be a bounding box or shape predicted to contain the relevant structure or feature.

[0050] Segmentation information is data initially generated by performing one or more segmentation techniques on a medical image. Exemplary segmentation techniques for identifying boundaries of one or more regions (e.g., each representing a different anatomical structure or element) are well established in the art. A segmentation technique identifies a particular region or boundaries of a particular region in a medical image.

[0051] Many tools and techniques for performing image segmentation are identified by Pham, Dzung L., Chenyang Xu, and Jerry L. Prince. “A survey of current methods in medical image segmentation.” Annual review of biomedical engineering 2.3 (2000): 315-337; Masood, Saleha, et al. “A survey on medical image segmentation.” Current Medical Imaging 11.1 (2015): 3-14; and / or Hesamian, Mohammad Hesam, et al. “Deep learning techniques for medical image segmentation: achievements and challenges.” Journal of digital imaging 32.4 (2019): 582-596.

[0052] The segmentation information may be obtained by retrieving or receiving the segmentation information from a database / memory system or a medical imaging system. Alternatively, step 112 may generate or include generating one or more segmentation techniques on the medical image. Examples of suitable segmentation techniques have been previously described, and other examples will be apparent to those skilled in the art.

[0053] The acquisition process 110 also includes a step 113 of acquiring, for each region in the medical image, label information that identifies the label of said region.

[0054] The label information may be obtained from a database or memory system, or by retrieving or receiving the label information from a medical imaging system. Alternatively, step 112 may include performing one or more labeling or classification techniques on the medical image to generate or generate the label information.

[0055] Each label may be a label that identifies a relevant region of the medical image, for example, identifying a structure / element that is predicted to be contained within (or bounded by) the relevant region of the medical image. In another example, the label may be a label that provides additional information about the structure / element (e.g., an anatomical structure / element) represented by the relevant region of the medical image. For example, if a region is predicted to contain a representation of a heart, the label may be a heart beat. As another example, the markings may be marks that provide an estimated size of the region. Other suitable examples will be readily apparent to those skilled in the art.

[0056] A label is thus a textual or symbolic (e.g., numerical) annotation for a particular region. A label facilitates semantic identification of the region and / or identification of a property of the region (e.g., a measurement of a particular parameter of the region, such as its size).

[0057] The label information may have been originally generated together with the segmentation information. For example, a segmentation technique may be designed or configured to segment (i.e., identify the boundaries of) one or more predetermined (e.g., anatomical) structures in a medical image, and thus the identity of the structures is known in advance. This information can be used to generate identifying labels for the regions.

[0058] It will be appreciated that the known identity of the structure represented by a region can also be used to provide additional or other information about the structure represented by the region, such as another characteristic of the region. For example, if a region bounds the heart, the heart rate of the imaged subject can be retrieved (e.g., from a database or heart rate monitoring) and used as a label for the region. As another example, if a region bounds a chamber of the heart, the blood flow velocity in / out of that chamber can be retrieved (e.g., from a database or from an appropriate monitoring device) and used as a label for the region. Other exemplary approaches will be apparent to those skilled in the art.

[0059] It will also be appreciated that the label information may have been generated by determining one or more measurements of the corresponding / associated region, e.g., by determining its size and / or by monitoring the movement of features / elements within the region over time. As another example, the label information may be generated by classifying the region, e.g., using classification techniques.

[0060] However, it will be appreciated that the exact mechanism for generating or creating the label information is not important to the basic approach proposed herein.

[0061] The label zone identification process 120 involves processing the medical image to identify one or more label zones, if any, where each label zone represents a contiguous portion of the medical image that is not related to the desired clinical examination and / or whose pixels have intensity values ​​that vary by less than a predetermined amount.

[0062] A contiguous portion is a portion of a medical image that includes multiple pixels, each of which is adjacent (e.g., directly touching or directly adjacent to) at least one other pixel in the contiguous portion, and preferably at least two other pixels in the contiguous portion.

[0063] Procedure 120 may include performing segmentation techniques on the medical image to identify as label zones any regions not related to the desired clinical exam, it being understood that the exact nature of such regions will depend on the type of clinical exam desired.

[0064] For example, if the clinical exam is a lung examination of the subject, procedure 120 may include identifying any region of the heart or any region outside the subject as a labeled zone. Similarly, if the clinical exam is a cardiac examination of the subject, procedure 120 may include identifying any region of the lung or any region outside the subject as a labeled zone. As another example, if the desired clinical exam is a renal examination, procedure 120 may include the intestines, spine, and / or any region outside the subject.

[0065] Procedure 120 can include identifying any zones that are not included in any of the one or more regions identified by the segmentation information, recognizing that the identified regions are likely to be clinically relevant to the clinical test, and consequently, other regions are less likely to be relevant to the clinical test.

[0066] The above-described approach provides a mechanism for identifying one or more labeled zones that are not relevant to the desired clinical test (ie, one or more "clinically irrelevant labeled zones").

[0067] In some examples, procedure 120 includes identifying as label zones one or more contiguous regions or zones whose pixels have intensity values ​​that vary by less than a predetermined amount, i.e., have substantially uniform intensity values. These label zones are sometimes referred to as "homogeneous label zones."

[0068] A contiguous region or zone is a region or zone that includes multiple pixels, each of which is adjacent (e.g., directly touching or directly adjacent to) at least one other pixel in the contiguous region / zone, and preferably at least two other pixels in the contiguous region / zone.

[0069] The predetermined amount may be, for example, a predetermined range of intensity values ​​(e.g., ±15 or ±10 in a scenario where possible intensity values ​​are in the range [0, 255]). Thus, in this embodiment, intensity values ​​of pixels within the same uniform label zone may differ by no more than ±X from the median intensity value within that uniform label zone. The value of X may be, for example, 15 or 10.

[0070] As another example, the predetermined amount may be a predetermined percentage of the median intensity value in the uniformly labeled zone. Thus, in this embodiment, the intensity values ​​of all pixels within the same uniformly labeled zone may differ by no more than 1±Y times the median intensity value within that uniformly labeled zone. The value of Y may be, for example, 0.15 or 0.10.

[0071] Identifying a uniform label zone can be performed by using region-growing techniques. For example, a uniform label zone can be generated by first defining a particular pixel as a potential label zone and then comparing the intensity value of that potential label zone (initially a single pixel) with any neighboring pixels (i.e., any pixels neighboring any pixel in the potential label zone). Any neighboring pixels that are sufficiently similar (i.e., differ by less than a predetermined amount) are added to the potential label zone. This process is repeated until the neighboring pixels are no longer sufficiently similar. If the potential label zone (as a result of the process) is sufficiently large, it is a label zone; otherwise, it is discarded. A potential label zone may be sufficiently large if it contains more than a predetermined number of pixels, e.g., more than 20 pixels, or if it represents more than a predetermined percentage of the medical image.

[0072] Another approach to identifying homogeneously labeled zones is to process the medical image using a clustering algorithm, such as a fuzzy clustering algorithm, to identify labeled zones. Clustering algorithms identify zones of similarity, i.e., zones that do not differ by more than a predetermined amount.

[0073] Yet another approach is to perform clinical thresholding techniques (e.g., multi-level or single-level thresholding techniques) to identify any homogeneously labeled zones. Thresholding techniques are established in the art to identify contiguous portions of an image that have intensity values ​​within a particular range.

[0074] Other approaches for performing procedure 120 will be apparent to those skilled in the art.

[0075] In some examples, procedure 120 includes additional constraints / restrictions or additional criteria for the label zone, such as minimum and / or maximum intensity values, specific locations, minimum sizes and / or minimum / maximum eccentricities.

[0076] This may be done, for example, by identifying multiple possible label zones and discarding those that do not meet certain criteria, and / or by only identifying label zones that meet certain criteria.

[0077] In one example, the intensity value of each pixel within any label zone is less than a first global threshold intensity value. Effectively, this controls procedure 120 so that only "dark" portions of the medical image are identified label zones. Dark portions typically represent areas of little or no clinical interest and are therefore preferred for determining label placement. This approach is particularly advantageous when the medical image is an ultrasound image, since dark portions of the region represent anechoic portions of the region, thereby representing zones predicted to contain air, which is generally not considered clinically interesting. Label zones of this form can be labeled "dark homogeneous label zones."

[0078] In some examples, the first global threshold intensity value is derived from the minimum intensity value of any / all pixels in the medical image. For example, the first global threshold intensity value may be the minimum intensity value of the medical image plus a bias value and / or a bias percentage of the minimum intensity value. In one example, the first global threshold intensity value is the minimum intensity value + 30 or the minimum intensity value + 20 (for a scenario where possible intensity values ​​are in the range [0, 255]).

[0079] As another example, the first global threshold intensity value may be predetermined, eg, preset.

[0080] In another example, the intensity value of each pixel within any label zone exceeds a second global threshold intensity value. Effectively, this controls procedure 121 so that only "bright" portions of the medical image can be identified as label zones. These can be labeled as "bright homogeneous label zones."

[0081] Of course, a combination of these approaches can also be used. For example, in some examples, for each label zone, the intensity value of each pixel within that label zone is either less than a first global threshold intensity value or greater than a second (different, greater) global threshold intensity value. This approach effectively facilitates identification of the brightest or darkest regions of a medical image.

[0082] As an example, procedure 120 may be configured such that each label zone is located entirely within a region identified in the segmentation information, and thus may be a subregion of a region identified in the segmentation information.

[0083] For example, in some embodiments, procedure 120 includes processing the regions to identify (for each region) as label zones any subregions of the region that each represent a contiguous portion of the region whose pixels have intensity values ​​that vary by less than a predetermined amount.

[0084] Any of the above-described approaches for identifying labeled zones can be easily adapted to process only a region of a medical image and identify any subregion of that region as a labeled zone. The term "medical image" can be replaced with the term "region of a medical image" where appropriate.

[0085] In some preferred examples, the intensity value of each pixel within any subregion (i.e., label zone) within the region is less than the region's first region-specific threshold intensity value. Effectively, this controls procedure 120 so that only "dark" portions of the region are identified as subregions or label zones. Dark regions are preferred for determining label placement because they typically represent regions of little or no clinical interest. This approach is particularly advantageous when the medical image is an ultrasound image, since dark portions of the region represent anechoic portions of the region, thereby representing zones expected to contain air, which is generally considered to be of no clinical interest.

[0086] In some examples, a first region-specific threshold intensity value for a region is derived from the minimum intensity value of any pixel in the region. For example, the first region-specific threshold intensity value may be the minimum intensity value (for that region) plus a bias value and / or a bias percentage of the minimum intensity value. In one example, the first region-specific threshold intensity value is the minimum intensity value + 30 or the minimum intensity value + 20 (for a scenario where possible intensity values ​​are in the range [0, 255]).

[0087] In another example, the intensity value of each pixel in any label zone exceeds a second region-specific threshold intensity value for the region. Effectively, this controls procedure 121 so that only the "bright" parts of the region are identified as sub-regions or label zones. This approach may be preferable, for example, when the region identifies other bright objects in a medical image, such as bone or an interventional device.

[0088] Of course, combinations of these approaches can also be used. For example, in some examples, for each subregion or label zone within a region, the intensity value of each pixel within that subregion is either less than a first region-specific threshold intensity value for that region or greater than a second (different and greater) region-specific threshold intensity value for that region. This approach effectively facilitates identification of the lightest or darkest subregions within a region.

[0089] In some examples, each subregion (label zone) represents a portion of a region having a size equal to or greater than a predetermined percentage of the region's size. Thus, each label zone represented by a subregion of a region identified in the segmentation information may have a minimum size (e.g., equal to or greater than a predetermined percentage of the region's size).

[0090] In some examples, only the largest sub-regions (which can represent label zones) are identified or selected, so that at most one sub-region can be identified as a label zone for each region.

[0091] In some examples, each identified label zone may have a minimum size, e.g., a predetermined percentage or fraction of the medical image. Thus, each label zone may have a size that is a predetermined fraction (e.g., 10% or 20%) or greater of the size of the medical image, where the size may correspond to the total area or volume of the medical image.

[0092] In some examples, the minimum size for each identified label zone may depend on the size of the label to be placed within the label zone. For example, the minimum size may be equal to or greater than the maximum size of the label of the closest X region (associated with the desired clinical test) closest to the identified label zone, where X is any integer value. An alternative to the closest X region is any region that falls within a predetermined distance of the identified label zone.

[0093] Thus, procedure 121 may include a step 124 of discarding any label zones having a size less than a predetermined size, for example a predetermined percentage of the medical image (the "PD size").

[0094] In some examples, each identified label zone is configured such that when an ellipse fits within the boundary of the label zone, the eccentricity of the ellipse is less than a predetermined eccentricity. This approach effectively checks the extent of the label zone to determine or establish whether the region is appropriately shaped for a label.

[0095] Approaches for fitting an ellipse to a known shape or region are well established in the art. Some exemplary approaches are disclosed by Mulchrone, Kieran F., and Kingshuk Roy Choudhury. "Fitting an ellipse to an arbitrary shape: implications for strain analysis." Journal of structural Geology 26.1 (2004): 143-153 or the fitEllipse function defined in OpenCV Python®.

[0096] Approaches for determining the eccentricity of an ellipse are well known in the field of mathematics and will not be described for the sake of brevity.

[0097] Therefore, procedure 121 can include step 125 of discarding any identified label zones that have an eccentricity above a predetermined eccentricity ("PD eccentricity") This approach aims to avoid cases where the label zones are very thin and / or elongated, which is not optimal for label positioning.

[0098] The predetermined eccentricity may be, for example, a value of 0.995 or less, such as 0.99 or less, for example 0.95 or less. For example, the predetermined eccentricity may be 0.99.

[0099] Process 130 includes processing any identified label zones to identify the location of the label for each region. As previously mentioned, the label for each region is identified in the label information.

[0100] The position of the label can be defined by the position of the center of the label. Thus, the position of the label may be the position of the center of the label. In another example, the position of the label may be a predetermined corner of the label, for example, the bottom left corner of the label. The exact definition of the position of the label may depend on the technology or programming language used to define the position of an object, such as a label, relative to an image.

[0101] The process 130 may include, for each region, identifying 131 any associated label zones, which are label zones that meet one or more predetermined criteria that can be associated with the region.

[0102] In some instances, different regions have different non-overlapping sets of one or more label zones, while in other instances, label zones may be shared between different regions.

[0103] In one example, the associated label zone is a label zone that entirely overlaps with the region, i.e., is a subregion of the region. In approaches where procedure 120 includes identifying subregions of the region, step 131 can be effectively integrated into procedure 120.

[0104] In another example, an associated label zone is a label zone that partially or wholly overlaps (i.e., overlaps) with a region, e.g., is a sub-region of the region or extends within the region. In this example embodiment, a label zone that partially overlaps with a region may be the only associated zone for that region if there is no label zone that is a sub-region of that region (i.e., completely overlaps).

[0105] In yet another example, an associated label zone is a label zone that partially or wholly overlaps a region, or is within a predetermined distance of a region. In this example embodiment, a label zone that is within a predetermined distance of a region (but does not overlap with said region) may be a sub-region of that region, or may be an associated zone of that region only if there are no label zones that partially overlap with that region. Similarly, in some embodiments, a label zone that partially overlaps with a region may only be an associated zone of that region if there are no label zones that are sub-regions of that region.

[0106] In yet another example, the associated label zones are label zones that partially or wholly overlap with the region, or (e.g., if there are no overlapping label zones) are a predetermined number of closest label zones to the region.

[0107] In some scenarios, a label zone includes one or more label zones that are not relevant to the desired clinical test (i.e., one or more "clinically irrelevant label zones") and one or more label zones whose pixels have intensity values ​​that vary by less than a predetermined amount (i.e., one or more "homogeneous label zones"). In such examples, a clinically irrelevant label zone may be a relevant label zone for a region only if there are no label zones that are homogeneous label zones in a subregion of the region that overlap the region, or that are within a predetermined distance of the region, or that are one of a predetermined number of label zones closest to the region. Similarly, a homogeneous label zone within a predetermined distance of a region may be a subregion of the region and only be a relevant label zone for the region if there are no label zones that overlap the region.

[0108] In approaches where no associated label zone is identified for a region, the region itself can be treated or act as an associated label zone for purposes of subsequent processing.

[0109] Thus, in order of priority, the one or more preferred associated label zones are: a label zone contained entirely within the region (if any), a label zone overlapping the region (if any), a homogeneous label zone adjacent to the region (if any), and a clinically irrelevant label zone adjacent to the region (if any). There may be an upper limit to the number of associated label zones used. If no label zones meet these criteria, the region itself can serve as the associated label zone.

[0110] However, this priority is not required and different embodiments or use case scenarios may utilize different orders or possible options for the associated label zones.

[0111] In a preferred example, process 130 includes, for each region, defining the location of the label such that the location is within one of the region's associated label zones (if any). As noted above, if no associated label zone is identified, the region itself can serve as the associated label zone.

[0112] In a preferred example, if there are multiple associated label zones, process 130 defines the location of the label so that its location is within the largest of the identified associated label zones. In another example, if there are multiple associated label zones, process 130 defines the location of the label so that it is located within the associated label zone closest to the centroid of the region to be labeled.

[0113] In some examples, defining the location of the label for the region includes defining the location of the label in response to a location of the centroid of at least one of the associated label zones. This approach aims to distance the label from the boundary of the associated label zone, thereby reducing the likelihood that the label will block or obscure potentially clinically relevant information in the medical image.

[0114] For example, the label may be positioned so that it is located at or near the centroid of one of the associated label zones, eg, the largest associated label zone.

[0115] In a more complex example, multiple possible positions are defined for one of the associated label zones (e.g., the largest associated label zone), and the label is positioned to be located at one of these possible positions.

[0116] A complete example approach for defining a location for a region's label using multiple locations for associated label zones is provided below.

[0117] For purposes of this example, it is assumed that only a single associated label zone has been identified, which may be, for example, the largest of multiple associated label zones (if multiple associated label zones are identified) or the only identified associated label zone (if only one associated label zone is identified).

[0118] As noted above, if no associated label zone has been identified for a region, the region itself can serve as the associated label zone. For simplicity, the term "associated label zone" will be used hereafter to refer to this single associated label zone or, where appropriate, the region itself.

[0119] Process 130 may include process 132, which identifies a plurality of possible locations for the label within the associated label zone. In this way, each associated label zone (and therefore each region) has an associated or corresponding plurality of possible locations for the label (of said region).

[0120] In one example, process 132 may simply involve defining any position within the relevant label zone as a possible position, with all positions within the relevant label zone acting as multiple possible positions.

[0121] In the illustrated example, process 132 executes by identifying multiple possible labels in response to the location of the centroid of the associated label zone.

[0122] Step 132 may include, for example, sub-step 132A of identifying the centroid of the associated label zone using well-established centroid identification techniques, and sub-step 132B of identifying multiple locations based on the identified location.

[0123] As an example, sub-step 132B can be performed using a topological skeleton.

[0124] Approaches for determining or discovering topological skeletons are well established in the art, such as the approach disclosed by Golland, Polina, W. Eric, and L. Grimson. "Fixed topology skeletons." Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662). Vol. 1. IEEE, 2000, or the "skeletonize" function provided by the scikit-image package of algorithms for image processing in Python.

[0125] Sub-step 132B can correspondingly include identifying multiple potential locations for the label using the topological skeleton. Each potential location is located on the topological skeleton and represents one of K locations closest to the centroid of the associated label zone. Thus, step 132B includes identifying K nearest neighbors of the location along the topological skeleton to the centroid of the associated label zone. The value of K can be predetermined or can depend on the number of points in the topological skeleton, for example, 10% of the total number of points in the topological skeleton.

[0126] Therefore, the process 132 may further include a sub-step 132C of determining or finding a topological skeleton (alternatively labeled a skeleton or skeletal structure) of the associated labeled zone.

[0127] An alternative approach to performing step 132B is to simply identify multiple possible locations at a predetermined distance and / or direction from the centroid of the associated label zone. For example, the multiple possible locations could include locations and distances of 0, 5, 10, and 20 pixels from the centroid in multiple directions (e.g., at least four evenly distributed directions from the centroid). This would generate XN + 1 possible locations, where X is the number of directions and N is the number of distances (a value of "1" represents a possible location at the centroid of the associated label zone).

[0128] Any other suitable technique for defining multiple locations using the centroids of associated label zones will be apparent to those skilled in the art.

[0129] Of course, other approaches for defining the multiple possible locations can also be used, for example, some approaches may identify as multiple possible locations only those locations that are more than a predetermined distance from the boundary of the associated label zone.

[0130] The process 130 may then select one of a number of possible locations to act as the identified location in step 133 .

[0131] Step 133 may include identifying the potential location that has the smallest distance from the centroid of the region (associated with the plurality of potential locations).

[0132] Step 133 may include identifying the locations that are likely to have the greatest average distance from the centroids of any other regions (or associated label zones of any other regions) identified in the segmentation information. This effectively identifies the locations that are furthest (on average) from other regions, i.e., furthest from the rest of the other centroids.

[0133] In another example, step 133 may include identifying possible locations further away from the boundary of the region associated with the associated label zone, which is particularly advantageous when the associated label zone is a sub-region of the region.

[0134] In some examples, process 130 further includes step 134 of determining or establishing whether the identified location (from step 133) is less than a predetermined distance from the boundary of the region.

[0135] The predetermined distance may be, for example, a distance selected to reduce or minimize the amount of overlap between the label and the boundary of the anatomical region, which preferably avoids overlap between the visual representation of the label and the boundary because the boundary contains important clinical information.

[0136] In response to a negative determination at step 134, the identified location is output as the location for process 130.

[0137] In response to a positive determination in step 134, the identified location is modified in step 135 to increase the distance of the identified location from the boundary of the region.

[0138] As an example, step 135 may include identifying the pixel in the associated label zone that is furthest from the boundary of the region as the furthest pixel, and modifying the potential location to be at the location of the furthest pixel.

[0139] Thus, step 135 may effectively comprise computing a distance transform for each and every pixel in the associated label zone and identifying the location of the pixel with the largest distance transform value as the location of the label, such that this position effectively indicates the point in the associated label zone that is farthest from the boundary.

[0140] Additionally, the label font size may be adjusted to fit the label position.

[0141] FIG. 2 illustrates a computer-implemented method 200 that uses any of the processes described above.

[0142] Thus, the method 200 includes performing the method 100 described above.

[0143] Method 200 also includes controlling 220 an output user interface (i.e., a display) to provide a visual representation of one or more labels contained in the medical image and the label information. The position of each label relative to the medical image is determined in method 100 and is used to control the position of each label, e.g., to align or match the identified position of the label.

[0144] In particular, for each label, the visual representation of the label can be controlled to overlay a visual representation of the medical image at a defined location of the label.

[0145] It will be appreciated that step 220 may further include controlling an output user interface to provide a visual representation of an identifying link (eg, an arrow or curved line) between the label and the region associated with the label.

[0146] Step 220 may include controlling the intensity or color of the label depending on the intensity of the label zone. For example, if the label zone has a relatively low intensity, the label may have a relatively high intensity, and vice versa.

[0147] In some preferred examples, step 220 includes controlling an output user interface to further provide a visual representation of the boundary of each region in the medical image identified in the segmentation information.

[0148] The effect of step 220 is to provide a display of the medical image and one or more labels for each region, optionally showing the boundaries of each region.

[0149] FIG. 3 shows an example medical image 300 and accompanying labels, the locations of which have been identified by performing the techniques disclosed herein.

[0150] More specifically, each label is located in a label zone 310, which represents a contiguous portion of the medical image 300 (here, the portion represents the lungs) that is not relevant to the desired clinical examination (here, a cardiac examination), i.e., a clinically irrelevant label zone.

[0151] FIG. 4 shows an example medical image 400 and accompanying labels, the locations of which have been identified by performing the techniques disclosed herein.

[0152] More specifically, each label is located in a different label zone 411, 412, where pixels have intensity values ​​that vary by less than a predetermined amount, representing successive portions of the medical image 400. Each label is located at the center of its respective label zone.

[0153] The first labeled zone 411 is an example of a uniformly labeled zone within a labeled region. In this example, the first labeled zone is a bright uniformly labeled zone. The label for the first labeled zone may be provided in a color or brightness that contrasts with the bright uniformly labeled zone (e.g., having a low intensity, e.g., black).

[0154] The second label zone 412 is an example of a uniform label zone outside the labeled region. In this example, the second label zone is a dark uniform label zone. Labels placed in the second label zone may be displayed using a contrasting brightness or color relative to the label zone, for example, a bright intensity (e.g., white) rather than a dark intensity.

[0155] FIG. 5 shows another example medical image 500 and an attached label whose location has been identified by performing the techniques disclosed herein.

[0156] More specifically, each label is again placed in a different label zone 511, 512, both of which represent contiguous portions of the medical image 500 where the pixels have intensity values ​​that vary by less than a predetermined amount. Each label is now placed at the center of its respective label zone.

[0157] In these examples, both the third labeled zone 511 and the fourth labeled zone 512 are examples of uniform labeled zones that are either within a labeled region or are labeled regions themselves.

[0158] FIG. 6 shows another example medical image 600 and an attached label whose location has been identified by performing the techniques disclosed herein.

[0159] More specifically, each label is again placed in a label zone 611, 612, both of which represent contiguous portions of the medical image 600 where pixels have intensity values ​​that vary by less than a predetermined amount. Each label is now placed at the center of its respective label zone.

[0160] In these examples, both the fifth label zone 611 and the sixth label zone 616 are examples of uniform label zones. This example also shows how a single label zone can be used to carry multiple labels, for example, for different regions of the medical image 600.

[0161] FIG. 7 shows yet another example of a medical image 700 and attached labels where the label locations have been identified by performing the techniques disclosed herein.

[0162] More specifically, Figure 7 shows a four-chamber (4CH) view of the heart. Potentially relevant areas of the heart for specific clinical examinations include the anatomical substructures of the heart. The following list provides examples of appropriate anatomical substructures for clinical examinations, with associated labels in parentheses: left atrium (LA); right atrium (RA); left ventricle (LV); right ventricle (RV); right ventricle (RV); mitral valve (MV); tricuspid valve (TV); IV septum (IV); interatrial septum (AT); AV septum (AV); spinal triangle (Sp), and descending aorta (Dar).

[0163] One or more of these regions can be identified, and the labels for said regions are then positioned to fall within the label zones identified using the approaches described above.

[0164] As a specific example, Figure 7 shows a label 715 for the right atrium RA. The label 715 is placed within a label zone 710 (represented by an outline), which is a uniformly labeled zone within the region to be labeled (e.g., within the right atrium). The label 715 is placed at a location 711 (represented by a circle) that is calculated or determined using the techniques described above. In particular, the location 711 is determined using the techniques described above that utilize a topological skeleton 712. The topological skeleton 712 is included for clarity of explanation and may not be present in, for example, a labeled medical image output by the present invention.

[0165] Other labels for other anatomical structures visible in the 4CH view of the heart are located using similar techniques. For clarity, the contours representing each label zone used for the corresponding label, the location of the labels (represented by their respective circles), and the topological skeleton used to determine the label locations are also shown. These may be omitted from the labeled medical image in practice.

[0166] FIG. 8 shows yet another example of a medical image 800 and an attached label where the label's location has been identified by performing the techniques disclosed herein.

[0167] More specifically, Figure 8 shows a three vessel and trachea (3VT) view. The following list, along with the associated labels in parentheses, provides examples of appropriate regions (e.g., anatomical substructures) for clinical examinations utilizing such views: Ductal Arch (Duc); Aortic Arch (Aortic Arch) (Aor); SVC (Svc); Trachea (Tra); and Spine Triangle (Sp).

[0168] One or more of these regions can be identified, and the labels for said regions are then positioned to fall within the label zones identified using the approaches described above.

[0169] As a specific example, Figure 8 shows a label 815 for an arch DC. The label 815 is placed within a label zone 810 (represented by an outline), which is a uniform label zone within the region to be labeled (e.g., within the arch). The label 815 is placed at a location 811 (represented by a circle) that is calculated or determined using the techniques described above. In particular, the location 811 is determined using the techniques described above that utilize a topological skeleton 812. The topological skeleton 812 is included for clarity of explanation and may not be present in, for example, a labeled medical image output by the present invention.

[0170] Other labels for other anatomical structures visible in the 3VT view are located using similar techniques. For clarity, the contours representing each label zone used for the corresponding label, the label locations (represented by their respective circles), and the topology skeleton used to determine the label locations are also shown. These may be omitted from the labeled medical image in practice.

[0171] 7 and 8 show examples in which the label zones are dark, uniform label zones. Thus, the labels may be displayed using a contrasting brightness or color to the label zone(s), for example, displayed in white rather than black.

[0172] FIG. 9 illustrates a system 900 according to one embodiment.

[0173] System 900 comprises a processor 910 and an output user interface 920. System 900 may optionally include an imaging system 930 and / or a memory or storage system 940. In some embodiments, imaging system 930 comprises an ultrasound transducer system capable of generating ultrasound images of an object using ultrasound imaging processing. Techniques for operating ultrasound transducer systems and generating ultrasound images are well established in the art and will not be described herein for the sake of brevity. The ultrasound transducer may be in immediate contact with the object when acquiring the ultrasound image.

[0174] The processor 910 is configured to define the location of one or more labels for the medical image.

[0175] The processor 910 comprises an input interface 911 , a data processor 912 , and optionally an output interface 913 .

[0176] The processor 910 is configured to obtain medical images, segmentation information, and label information, at least some of which may be obtained via an input interface 911 of the processor 910.

[0177] The medical image defines an intensity value for each of a plurality of pixels, the segmentation information identifies boundaries of one or more regions within the medical image, and the label information identifies, for each region within the medical image, a label for the region.

[0178] Medical images may be received / retrieved from the imaging system 930 and / or the storage system 940 via the input interface 911 .

[0179] The segmentation information may be received / retrieved from the storage system 940 via the input interface 911. Alternatively, the segmentation information may be generated by performing one or more segmentation techniques on the acquired medical images.

[0180] The label information may be received / retrieved from the storage system 940 via the input interface 911. Alternatively, the label information may be generated by a data processor that performs one or more labeling techniques on the acquired medical images. One or more of the labeling techniques may be at least partially integrated into the segmentation technique.

[0181] The processor 910 is further configured to, for each region of the medical image for which a boundary is identified in the segmentation information, use the data processor 912 to process the medical image to identify, if present, one or more label zones, each label zone representing a contiguous portion of the medical image that is not relevant to the desired clinical examination and / or whose pixels have intensity values ​​that vary by less than a predetermined amount, and, if one or more label zones are present, to define, in response to the positions of the one or more label zones, the position of each label of the one or more regions as identified in the label information.

[0182] In this manner, the processor 910 can define the position of each label within the label information.

[0183] The processor 910 may further be configured to, for example, control an output user interface 920 via an output interface 913 to provide a visual representation of one or more labels included in the medical image and the label information, whereby the position of each label relative to the medical image is determined (100) and used to control the position of each label, for example, to align or match with the identified position for the label.

[0184] The processor 910 may be adapted or configured to perform any of the methods described herein, and a person skilled in the art would readily be able to make any necessary modifications to the processor to perform such methods.

[0185] Processor 910 can be implemented in numerous ways using software and / or hardware to perform the various functions required. Processor 910 may comprise one or more microprocessors that can be programmed using software (e.g., microcode) to perform the necessary functions. However, processor 910 may be implemented with or without such microprocessors, and may also be implemented as a combination of dedicated hardware to perform some functions and processors (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions.

[0186] Examples of processor components that may be used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0187] In various implementations, the processor 910 may be associated with one or more storage media, such as volatile and non-volatile computer memory 915, including RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processors (e.g., data processor 812), perform the necessary functions. The various storage media may be fixed within the processor or may be transportable, such that the program or programs stored thereon can be loaded into the processor. The storage media may also be located external to the processor 910, such as in the cloud.

[0188] It will be understood that the disclosed methods are preferably computer-implemented methods. Therefore, the concept of a computer program comprising code (e.g., instructions for a computer / processor) for implementing any of the described methods when said program is run on a processor such as a computer is also proposed. Thus, different parts, lines or blocks of code of a computer program according to an embodiment may be executed by a processor or computer to perform the methods described herein.

[0189] A local or remote storage medium is also proposed that stores or carries a computer program or computer code that, when executed by a processor, causes the processor to perform the methods described herein.

[0190] In some alternative implementations, the functions noted in the block diagrams or flowcharts may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

[0191] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. Means recited in mutually different dependent claims may be advantageously combined. Where a computer program is described above, it may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium, provided together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. It should be noted that where the term "adapted for" is used in the claims or description, the term "adapted for" is intended to be equivalent to the term "configured for." Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. 1. A computer-implemented method for defining locations of one or more labels relative to one or more first regions in a medical image of a subject, the method comprising: receiving the medical image, the medical image defining an intensity value for each of a plurality of pixels; obtaining segmentation information identifying boundaries of the one or more first regions by receiving the segmentation information or by generating the segmentation information from the medical image; for each of one or more first regions in the medical image, obtaining label information identifying a label for the region by receiving the label information or by generating the label information from the medical image; if present, identifying one or more label zones in the medical image, each label zone representing a contiguous portion of the medical image whose pixels have intensity values ​​that vary by less than a predetermined amount; defining a location for each label of one or more first regions identified in the label information based on the identified one or more label zones; A method comprising:

2. The method of claim 1 , wherein the medical image comprises an ultrasound image.

3. The method of claim 1 , wherein for each label zone, pixels in the label zone have intensity values ​​below a global threshold intensity value.

4. The step of identifying one or more label zones in the medical image may include, for each region of the one or more first regions, if present: identifying any subregions of the region that each represent a contiguous portion of the region whose pixels have intensity values ​​that vary by less than a predetermined amount as label zones; 4. The method according to claim 1, further comprising:

5. 5. The method of claim 4, wherein defining a position for each label of the one or more first regions comprises, for each region of the one or more first regions, defining a position of a label relative to the region according to a position of any identified subregions of the region.

6. 6. The method of claim 4 or 5, wherein for each region of the one or more first regions, each pixel in any identified subregion has an intensity value below a region-specific threshold intensity value derived from the minimum intensity value of any pixel within the region.

7. 7. A method according to any one of claims 4 to 6, wherein each sub-region represents a portion of the region having a size equal to or greater than a predetermined percentage of the size of the region.

8. 8. The method of claim 7, wherein the predetermined percentage is greater than or equal to 10%, preferably greater than or equal to 20%.

9. The step of defining a position for each label of the one or more first regions may include, for each region of the one or more first regions: identifying any label zones that overlap the region as overlapping label zones; defining the location of the labels in said regions according to the location of any overlapping label zones; 9. The method according to claim 1, comprising:

10. The step of defining a position for each label of the one or more first regions may include, for each region of the one or more first regions: In response to the inability to identify any overlapping label zones, if any, defining a position of said region relative to the label in response to the positions of a predetermined number of closest label zones to said region.

10. The method of claim 9, comprising:

11. 11. The method of claim 1, wherein defining a position for each label of the one or more first regions comprises, for each of the one or more first regions, defining a position of a label relative to the region further depending on a centroid of the region.

12. 12. The method of claim 1, wherein defining a position for each label of the one or more first regions comprises defining, for each region of the one or more first regions, a position of the label relative to the region further depending on the centroid of any other regions, if any.

13. A computer program product comprising instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1 to 12.

14. A processor for defining the location of one or more labels relative to a medical image of a subject, the processor being configured to perform the method of any one of claims 1 to 12.

15. 1. A medical imaging system comprising: an imaging system for generating medical images; a processor according to claim 14, coupled to the imaging system for defining the location of one or more labels relative to an ultrasound image; A medical imaging system comprising:

Images