Defining location for marker in medical image

Abstract

A mechanism for defining the position of any marker of an ultrasound image. One or more tagged regions are identified by processing the ultrasound image. Each mark region is a continuous region whose pixels have intensity values that vary by less than a predetermined amount. Each marker is positioned in response to the identified location of the marker region.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging. Background Art

[0002] A common requirement in medical imaging procedures is to label any structures and / or features visible in the medical image. This is to reduce errors in analysis and to facilitate the evaluation of any structures / features within the medical image.

[0003] Automated detection techniques are increasingly developed and used to detect or segment anatomical features and / or elements within medical images. Such techniques typically produce segmentation information that identifies the boundaries of one or more regions within the medical image, as well as labeling information that provides a label or identifier for each identified region.

[0004] An ongoing problem is how to automatically (i.e., without user input) appropriately define the location for labeling. When analyzing a medical image, inappropriately placed label locations may lead to confusion and / or errors (e.g., if a label may be incorrectly attributed to an incorrect region), or occlusion of clinically relevant information within the medical image, e.g., if a label obscures a clinically relevant feature represented within the medical image.

[0005] One existing method is to simply place the label at the centroid or center point of the relevant region. This method reduces the chance of the label being confused with the label of another region.

[0006] An improved method for defining the location of a label for a segmented region within a medical image is desired. Summary of the Invention

[0007] The underlying invention is defined by the independent claims. The dependent claims define advantageous embodiments.

[0008] According to one aspect of the present invention, by way of example, there is provided a computer-implemented method for defining the positions of one or more markers for one or more first regions in a medical image of an object. The method includes: receiving an ultrasound image, wherein the medical image defines an intensity value for each of a plurality of pixels; obtaining the segmentation information by receiving the segmentation information or by generating the segmentation information based on the medical image, the segmentation information identifying the boundaries of the one or more first regions; obtaining the marker information by receiving the marker information or by generating the marker information based on the medical image, the marker information identifying, for each of the one or more first regions in the medical image, a marker for the region; if there are one or more marker regions in the medical image, identifying the one or more marker regions in the medical image, wherein each marker region is a continuous portion of the medical image, and the pixels of the continuous portion have intensity values that vary by less than a predetermined amount; and, based on the identified one or more marker regions, defining the positions of each of the markers for the one or more first regions identified in the marker information.

[0009] The present disclosure presents a technique for placing markers for segmented regions of a medical image. For each region, the marker is positioned in response to the positions of one or more marker regions identified within the medical image.

[0010] In particular, a marker region may represent a region of pixels having intensity values that vary by less than a predetermined amount (i.e., having substantially uniform intensity values). It has been recognized that relevant anatomical structures or elements typically have pixel intensity values assigned in significant amounts. Thus, each marker region represents a region or area that is less likely to obstruct or obscure clinically relevant anatomical features or structures for a desired clinical examination than other regions. Alternatively, a marker region may represent a region / area that is not relevant to the desired clinical examination. This avoids or reduces the likelihood that a marker will be located within a relevant region. For example, each marker region may represent a continuous portion of the medical image that is not included in any of one or more first regions or one or more predetermined second regions that are not relevant to the desired clinical examination.

[0011] The proposed method thus reduces the likelihood that a marker will be positioned at a location that obscures clinically relevant anatomical features or structures. The proposed method thus reliably assists a clinician in evaluating and / or analyzing a medical image (e.g., to make a clinician decision) by reducing the likelihood that potentially clinically relevant material is obstructed or obscured.

[0012] For example, the medical image may be an ultrasound image of a patient or other object.

[0013] For example, one or more first regions of an object can be one or more anatomical regions of the object. The one or more anatomical regions can be one or more predetermined regions, such as one or more regions associated with a desired clinical examination.

[0014] For example, the generation of the segmentation information can be implemented using any existing segmentation techniques or segmentation techniques developed in the future or a combination thereof.

[0015] In the context of the present invention, the position of a marker is considered to be the position where the marker is located. The position of the marker can be defined at a specific position relative to the marker itself, such as the center of the marker or a specific corner of the marker.

[0016] A marker is a textual or symbolic (e.g., numerical) annotation for a specific region. The marker facilitates the semantic recognition of the region and / or the recognition of the characteristics of the region (e.g., the measurement of specific parameters of the region, such as its dimensions, etc.).

[0017] Preferably, for each marker region, the pixels of the marker region have intensity values that vary by less than a predetermined amount. It has been previously mentioned how an area or region of uniform intensity is unlikely to contain regions of clinical importance. Thus, this method provides a mechanism for easily identifying the regions where a marker can be located or near which it can be located.

[0018] In some examples, in each marker region, the intensity values of the pixels of the marker region are lower than a global threshold intensity value.

[0019] In some examples, the step of processing a medical image to identify one or more marker regions (if any) includes, for each of one or more first regions in the medical image: processing the region to identify any sub-region within the region that represents a continuous portion of the region, the pixels of the continuous portion having intensity values that vary by less than a predetermined amount, as a marker region.

[0020] In some examples, the step of defining the position of a marker for each of one or more first regions includes: for each of one or more first regions, defining the position of the marker for the region in response to the position of any identified sub-region of the region.

[0021] Optionally, for each region, each pixel in any identified sub-region has an intensity value lower than a region-specific threshold intensity value, which is derived from the minimum intensity value of any pixel within the region. This method locates the marker relative to the darkest area / portion of the region. Dark regions are less likely to contain elements or features of clinical interest, thus more appropriately locating the marker to reduce / avoid obscuring any potentially relevant anatomical features / elements.

[0022] More specifically, if the medical image is an ultrasound image, the method will result in localizing the markers relative to the anechoic area or regions of the area. An anechoic region is a region that does not contain any anatomical elements or features that reflect ultrasonic waves.

[0023] In at least one example, each sub-region represents a portion of the region having a size that is not less than a predetermined percentage of the size of the region.

[0024] The predetermined percentage may be not less than 10%, and preferably not less than 20%.

[0025] The step of defining the position of each marker for one or more first regions may include, for each region among the one or more first regions: identifying any marker regions that overlap with the region as overlapping marker regions; and, in response to the position of any overlapping marker regions, defining the position of the marker for the region.

[0026] The step of defining the position of each marker for one or more first regions may include, for each region among the one or more first regions: in response to failing to identify any overlapping marker regions, then in response to the position of a predetermined number of marker regions closest to the region (if any) to define the position of the marker for the region. The predetermined number may be 1.

[0027] The step of defining the position of each marker for one or more first regions may include, for each region among the one or more first regions, further defining the position of the marker for the region in response to the centroid of the region.

[0028] The step of defining the position of each marker for one or more first regions may include, for each region among the one or more first regions, further defining the position of the marker for the region in response to the centroid of any other region (if any).

[0029] In some examples, each marker region is configured such that if an ellipse is fitted to the boundary of the marker region, the eccentricity of the ellipse is less than a predetermined eccentricity. The predetermined eccentricity may be, for example, a value not exceeding 0.995, such as not exceeding 0.99, such as not exceeding 0.95.

[0030] The method aims to avoid identifying sub-regions in the form of very thin and / or elongated shapes. Such sub-regions are not suitable for the localization of markers, for example, because the markers may prove to be larger than the width and / or height of such sub-regions.

[0031] There is also provided a computer program product or non-transitory storage medium including computer program code or instructions, which when run by a processor cause the processor to perform any of the methods disclosed herein.

[0032] There is also provided a processor for defining the location of one or more markers of a medical image of an object for a desired clinical examination, wherein the processor is configured to perform the method as disclosed herein.

[0033] There is also provided a medical imaging system, comprising: an imaging system for generating a medical image; and a processor as disclosed herein, which is coupled to the imaging system.

[0034] These and other aspects of the present invention will be apparent with reference to the embodiments described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] For a better understanding of the present invention, and for a clearer illustration of how to implement the present invention, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0036] Figure 1 is a flowchart showing a method according to an embodiment;

[0037] Figure 2 is a flowchart showing a method according to another embodiment;

[0038] Figures 3 to 8 shows an example of a medical image; and

[0039] Figure 9 shows a processor according to an embodiment. DETAILED DESCRIPTION

[0040] The present invention will be described with reference to the accompanying drawings.

[0041] It should be understood that the detailed description and specific examples, although indicating exemplary embodiments of the present invention, are intended for illustrative purposes only and are not intended to limit the scope of the present invention. These and other features, aspects and advantages of the present invention will be better understood from the following description, the appended claims and the drawings. It should be understood that the drawings are merely schematic and not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to indicate the same or similar components.

[0042] The present invention provides a mechanism for defining the location of any marker for a medical image. One or more marker regions are identified by processing the medical image. Each marker region is a region in the medical image that is irrelevant to the desired clinical examination and / or has intensity values with a variation less than a predetermined amount. Each marker is located in response to the location of the identified marker region.

[0043] The embodiments are based on the recognition that improper positioning of markers in medical images can significantly affect the readability or interpretability of medical images. Therefore, it would be advantageous to position the markers such that the interpretability is not affected. The proposed method overcomes this by proper positioning of the markers.

[0044] The disclosed method can be used to control the position of markers in any form of medical image (in particular, ultrasound images).

[0045] Figure 1 A method 100 for defining the position of one or more markers for a medical image is shown. The method 100 can be executed by a processor (such as Figure 9 the processor 910 shown).

[0046] The method 100 can conceptually be divided into an acquisition process 110 (sometimes referred to as an initialization process), a marker region identification process 120, and a marker positioning process 130.

[0047] The acquisition process 110 includes a step 111 of acquiring a medical image. The medical image defines an intensity value for each of a plurality of pixels. The intensity value typically represents the brightness of the image and can be in the range [0, 255] or [0, 511].

[0048] The medical image can be a 2D image or a 3D image. The term "pixel" refers to the smallest spatial unit represented by the medical image and is considered interchangeable with the term "voxel" for 3D images.

[0049] Suitable examples of medical images are well known in the art and can include ultrasound images, magnetic resonance images, CT images, X-ray images, etc. It should be understood that a medical image is an image generated using a medical imaging technique that provides a visual representation of at least the interior of an object.

[0050] For example, step 111 can include obtaining or receiving the medical image from a database or memory system or from a medical imaging system.

[0051] The acquisition process 110 further includes a step 112 of obtaining segmentation information that identifies the boundaries of one or more regions in the medical image.

[0052] Each region of the medical image / each region in the medical image can be a region in the medical image that is predicted or determined to contain a specific structure or feature (such as an anatomical feature / structure (e.g., an organ or a part of an organ)) or an intervention device (e.g., a catheter or a pacemaker). In a preferred example, the boundary of the region can be a predicted boundary of the structure or feature predicted to be contained within the region, e.g., a predicted boundary that defines the shape of the relevant structure or feature. Alternatively, the boundary of the region can be a bounding box or shape predicted to contain the relevant structure or feature.

[0053] Segmentation information is data that is initially generated by performing one or more segmentation techniques on a medical image. Exemplary segmentation techniques for identifying the boundaries of one or more regions (e.g., each representing a different anatomical structure or element) are well-known in the art. The segmentation technique identifies the boundaries of one or more specific regions of the medical image.

[0054] Multiple tools and techniques for performing image segmentation have been identified by Pham, Dzung L., Chenyang Xu, and Jerry L. Prince. "A survey of current methods in medical image segmentation." Annual review of biomedical engineering 2.3 (2000): 315-337; Masood, Saleha et al. "A survey on medical image segmentation." Current Medical Imaging 11.1 (2015): 3-14; and / or Hesamian, Mohammad Hesam et al. "Deep learning techniques for medical image segmentation: achievements and challenges." Journal of digital imaging 32.4 (2019): 582-596.

[0055] Segmentation information can be obtained by retrieving or receiving the segmentation information from a database / memory system or from a medical imaging system. Alternatively, step 112 can include performing one or more segmentation techniques on the medical image to generate or produce the segmentation information. Examples of suitable segmentation techniques have been previously described, and other examples will be apparent to those skilled in the art.

[0056] The obtaining process 110 further includes a step 113 of obtaining labeling information that identifies a label for each region in the medical image.

[0057] Labeling information can be obtained by retrieving or receiving the labeling information from a database or memory system or from a medical imaging system. Alternatively, step 112 can include performing one or more labeling or classification techniques on the medical image to generate or produce the labeling information.

[0058] Each label can be a label identifying an associated region of a medical image. For example, it identifies a structure / element predicted to be contained within (or bounded by) the associated region of the medical image. In other examples, the label can be a label providing additional information about the structure / element (e.g., anatomical structure / element) represented by the associated region of the medical image. For example, if the region is predicted to contain a representation of the heart, the label can be the heart rate of the heart. As another example, the label can be a label providing a predicted size of the region. Other suitable examples will be apparent to those skilled in the art.

[0059] Thus, a label is a textual or symbolic (e.g., numerical) annotation for a specific region. The label facilitates semantic recognition of the region and / or recognition of the characteristics of the region (e.g., measurement of specific parameters of the region, such as its size, etc.).

[0060] The label information may be initially generated together with the segmentation information. For example, segmentation techniques can be designed or configured to segment one or more predetermined (e.g., anatomical) structures in a medical image (i.e., identify their boundaries), so the identity of the structure is pre-known. This information can be used to generate an identification label for the region.

[0061] It should be understood how the known identity of the structure represented by the region can also be used to provide additional or other information about the structure represented by the region, e.g., another attribute of the region. For example, if the region bounds the heart, the heart rate of the imaged subject can be obtained (e.g., from a database or a heart rate monitor) and used as a label for the region. As another example, if the region bounds a chamber of the heart, the blood flow rate into / leaving the chamber can be obtained (e.g., from a database or from a suitable monitoring device) and used as a label for the region. Other example methods will be apparent to those skilled in the art.

[0062] It should also be understood that label information can be generated by determining one or more measurements of the corresponding / associated region (e.g., by determining its size and / or monitoring the movement of features / elements within the region over time). As another example, label information can be generated by, for example, classifying the region using classification techniques.

[0063] However, it should be understood that the exact mechanism for generating or producing label information is immaterial to the underlying method presented herein.

[0064] The label region identification process 120 includes processing a medical image to identify one or more label regions, if any. Each label region represents a continuous portion of the medical image: the continuous portion is unrelated to a desired clinical examination and / or its pixels have intensity values that vary by less than a predetermined amount.

[0065] A continuous portion is a portion of a medical image that includes a plurality of pixels, each pixel being adjacent (e.g., in direct contact or immediately adjacent) to at least one other pixel in the continuous portion, and preferably adjacent to at least two other pixels in the continuous portion.

[0066] Process 120 may include performing a segmentation technique on the medical image to identify any regions that are not relevant to the desired clinical examination as flagged regions. It should be understood that the exact nature of such regions depends on the type of the desired clinical examination.

[0067] For example, if the clinical examination is a study of the lungs of an object, process 120 may include identifying any region of the heart or any region outside the object as a flagged region. Similarly, if the clinical examination is an investigation of the heart of an object, process 120 may include identifying any region of the lungs or any region outside the object as a flagged region. As another example, if the desired clinical examination is an investigation of the kidneys, process 120 may include regions of the intestines, spine, and / or any region outside the object.

[0068] Process 120 may include identifying any region in any region that is not included in one or more regions identified by the segmentation information. The method recognizes that the identified regions may have clinical relevance to the clinical examination, such that other regions are less likely to be relevant to the clinical examination.

[0069] The above method provides a mechanism for identifying one or more flagged regions (i.e., one or more “clinically irrelevant flagged regions”) that are not relevant to the desired clinical examination.

[0070] In some examples, process 120 includes identifying one or more continuous regions or areas whose pixels have intensity values that vary by less than a predetermined amount (i.e., have substantially uniform intensity values) as flagged regions. These flagged regions may be referred to as “uniform flagged regions”.

[0071] A continuous region or area is a region or area that includes a plurality of pixels, each pixel being adjacent (e.g., in direct contact or immediately adjacent) to at least one other pixel in the continuous region / area, and preferably adjacent to at least two other pixels in the continuous region / area.

[0072] For example, the predetermined amount may be a predetermined range for the intensity values (e.g., ±15 or ±10 for a scenario where the possible intensity values are in the range [0, 255]). Thus, in this embodiment, the intensity values of the pixels in the same uniform flagged region may differ from the median intensity value in the uniform flagged region by no more than ±X. For example, the value of X may be 15 or 10.

[0073] As another example, the predetermined amount can be a predetermined percentage of the median of the intensity values in the unified marker region. Thus, in this embodiment, the intensity values of all pixels in the same unified marker region can be different and can be no more than 1±Y times the median of the intensity values in the unified marker region. For example, the value of Y can be 0.15 or 0.10.

[0074] The identification of the unified marker region can be performed by using region growing techniques. For example, the unified marker region can be generated by first defining a specific pixel as the expected marker region and comparing the intensity value of the expected marker region (initially: a single pixel) with any adjacent pixels (i.e., any pixel adjacent to any pixel in the expected marker region). Any adjacent pixels that are similar enough (i.e., the difference is less than the predetermined amount) are added to the expected marker region. Then the process is repeated until no adjacent pixels are similar enough. If the expected marker region (at the result of the process) is large enough, it is the marker region, otherwise it is discarded. The expected marker region can be large enough when it contains more than a predetermined number of pixels (e.g., more than 20 pixels) or represents more than a predetermined percentage of the medical image.

[0075] Another method for identifying the unified marker region is to use a clustering algorithm (such as a fuzzy clustering algorithm) to process the medical image for identifying the marker region. The clustering algorithm will identify similar regions, i.e., regions with differences not exceeding the predetermined amount.

[0076] Yet another method is to perform thresholding techniques (e.g., multi-level or single-level thresholding techniques) on the medical image to identify any unified marker regions. Thresholding techniques for identifying continuous portions of an image with intensity values within a specific range are well established in the art.

[0077] Other methods for performing process 120 will be apparent to those skilled in the art.

[0078] In some examples, process 120 includes further constraints / limitations or further criteria for the marker region, for example, minimum and / or maximum intensity values; certain positions; minimum size and / or minimum / maximum eccentricity.

[0079] For example, this can be performed by identifying multiple potential marker regions and discarding any marker regions that do not meet specific criteria; and / or only identifying marker regions that meet specific criteria.

[0080] In one example, the intensity value of each pixel in any marked region is below a first global threshold intensity value. Effectively, this controls process 120 such that only the (one or more) "dark" portions of the medical image are identified as the (one or more) marked regions. The dark portions are preferred for determining the placement of the markers because they typically represent regions of little clinical interest. If the medical image is an ultrasound image, this method is particularly advantageous because the dark portions of the region will represent the anechoic portions of the region, thus representing regions predicted to contain air, which are generally considered clinically uninteresting. This form of marked region can be labeled as a "dark uniform marked region".

[0081] In some examples, the first global threshold intensity value is derived from the minimum intensity value of any / all pixels within the medical image. For example, the first global threshold intensity value can be the minimum intensity value of the medical image plus a bias value and / or plus a bias percentage of the minimum intensity value. In one example, the first global threshold intensity value is the minimum intensity value + 30 or the minimum intensity value + 20 (for a scenario where the possible intensity values are in the range [0, 255]).

[0082] As another example, the first global threshold intensity value can be predefined, e.g., pre-set.

[0083] In other examples, the intensity value of each pixel in any marked region is above a second global threshold intensity value. Effectively, this controls process 121 such that only the (one or more) "bright" portions of the medical image can be identified as the (one or more) marked regions. These can be labeled as "bright uniform marked regions".

[0084] Of course, combinations of these methods can be used. For example, in some examples, for each marked region, the intensity value of each pixel in the marked region is below the first global threshold intensity value or above a second (different and larger) global threshold intensity value. This method effectively facilitates the identification of the brightest or darkest areas of the medical image.

[0085] As an example, process 120 can be configured such that each marked region is completely within the region identified in the segmentation information. Thus, each marked region can be a sub-region of the region identified in the segmentation information.

[0086] For example, in some embodiments, process 120 includes (for each region) processing the region to identify any sub-region within the region that represents a continuous portion of the region, where the pixels of the continuous portion have intensity values that vary by less than a predetermined amount as a marked region.

[0087] Any of the above methods for identifying a marked region can be readily adapted to process only a region of a medical image to identify any sub-region of that region as a marked region. Where appropriate, the term "medical image" may be replaced by the term "region of a medical image".

[0088] In some preferred examples, the intensity value of each pixel in any sub-region within the region (i.e., the marked region) is lower than a first region-specific threshold intensity value for the region. Effectively, this controls process 120 such that only the (one or more) "dark" portions of the region are identified as the (one or more) sub-regions or marked regions. Dark regions are preferred for determining the placement of a marker as they typically represent regions of little clinical interest. This method is particularly advantageous if the medical image is an ultrasound image, as the dark portions of the region will represent the anechoic portions of the region, thus indicating regions predicted to contain air, which are generally considered to be clinically uninteresting.

[0089] In some examples, the first region-specific threshold intensity value for the region is derived from the minimum intensity value of any pixel within the region. For example, the first region-specific threshold intensity value can be the minimum intensity value plus a bias value and / or plus a bias percentage of the minimum intensity value. In one example, the first region-specific threshold intensity value is the minimum intensity value + 30 or the minimum intensity value + 20 (for a scenario where the possible intensity values are in the range [0, 255]).

[0090] In other examples, the intensity value of each pixel in any marked region is higher than a second region-specific threshold intensity value for the region. Effectively, this controls process 121 such that only the (one or more) "bright" portions of the region are identified as the (one or more) sub-regions or marked regions. For example, this method may be preferred if the region identifies a bone or other bright object (such as an interventional device) within the medical image.

[0091] Of course, combinations of these methods can be used. For example, in some examples, for each sub-region or marked region within a region, the intensity value of each pixel in that sub-region is lower than a first region-specific threshold intensity value for the region or higher than a second (different and larger) region-specific threshold intensity value for the region. This method effectively facilitates the identification of the brightest or darkest sub-regions within the region.

[0092] In some examples, each sub-region (which is a marked region) represents a portion of the region having a size that is not less than a predetermined percentage of the size of the region. Thus, each marked region represented by a sub-region of the region identified from the segmentation information can have a minimum size (e.g., not less than a predetermined percentage of the size of the region).

[0093] In some examples, only the largest sub-region (which may represent a marker region) is identified or selected. Thus, up to one sub-region can be identified as the marker region for each region.

[0094] In some examples, each identified marker region can have a minimum size, e.g., not less than a predetermined percentage or proportion of the medical image. Thus, each marker region can have a size that is not less than a predetermined percentage (e.g., 10% or 20%) of the size of the medical image. Here, the size can correspond to the total area or volume of the medical image.

[0095] In some examples, the minimum size for each identified marker region can depend on the size of the marker(s) to be located in the marker region(s). For example, the minimum size can be not less than the size of the largest marker for the closest X regions (related to the desired clinical examination) to the identified marker region. X is any integer value. An alternative to the closest X regions is any region that falls within a predetermined distance of the identified marker region.

[0096] Thus, process 121 can include step 124 of discarding any marker region having a size below a predetermined size (e.g., a predetermined percentage (“PD size”) of the medical image).

[0097] In some examples, each identified marker region is configured such that if an ellipse is fitted to the boundary of the marker region, the eccentricity of the ellipse is less than a predetermined eccentricity. This method effectively checks the spread of the marker region to determine or establish whether the region is properly shaped for the marker.

[0098] Methods for fitting an ellipse to a known shape or region are well established in the art. Examples of methods are disclosed by Mulchrone, Kieran F. and Kingshuk Roy Choudhury in "Fitting an ellipse to an arbitrary shape: implications for strain analysis." Journal of structural Geology 26.1 (2004): 143 - 153 or by the fitEllipse function defined in OpenCV defines some example methods.

[0099] Methods for determining the eccentricity of an ellipse are well known in the mathematical art and are not described for the sake of brevity.

[0100] Accordingly, process 121 may include step 125 of discarding any identified marked area having an eccentricity higher than a predetermined eccentricity (“PD eccentricity”). The method is intended to avoid cases where the (one or more) marked areas are very thin and / or elongated, which is not optimal for mark positioning.

[0101] For example, the predetermined eccentricity may be a value not exceeding 0.995, such as not exceeding 0.99, such as not exceeding 0.95. For example, the predetermined eccentricity may be 0.99.

[0102] Process 130 includes processing any identified marked area to identify the position of the mark for each area. As previously mentioned, the marks for each area are identified in the mark information.

[0103] The position of the mark may be defined by the position of the center of the mark. Thus, the position of the mark may be the position of the center of the mark. In other examples, the position of the mark may be a predefined corner of the mark, such as the lower left corner of the mark. The precise definition of the position of the mark may depend on the technique or programming language used to define the position of an object (such as a mark) relative to an image.

[0104] Process 130 may include step 131 of identifying any relevant marked areas for each area. A relevant marked area is a marked area that meets one or more predetermined criteria that can be associated with the area.

[0105] In some examples, different areas have different non - overlapping sets of one or more marked areas. In other examples, marked areas may be shared between different areas.

[0106] In one example, a relevant marked area is a marked area that completely overlaps the area, i.e., is a sub - area of the area. In a method where process 120 includes identifying sub - areas of an area, step 131 can be effectively integrated into process 120.

[0107] In another example, a relevant marked area is a marked area that partially or completely overlaps (i.e., intersects) the area, e.g., is a sub - area of the area or extends into the area. In an embodiment of this example, if there is no marked area that is a sub - area (i.e., completely overlaps) of the area, then the marked area that partially intersects the area may be the only relevant area for that area.

[0108] In yet another example, the relevant marker region is a marker region that: partially or fully overlaps with the region; or is within a predetermined distance of the region. In an embodiment of this example, if there is no marker region that is a sub-region of the region or that partially overlaps with the region, then a marker region within a predetermined distance of the region (but not overlapping with the region) can be the relevant region for that region only. Similarly, in some embodiments, if there is no marker region that is a sub-region of the region, then a marker region that partially overlaps with the region can be the relevant region for that region only.

[0109] In yet another example, the relevant marker region is a marker region that: partially or fully overlaps with the region; or (for example, if there is no overlapping marker region) is one of a predetermined number of marker regions closest to the region.

[0110] In some scenarios, the marker regions include one or more marker regions that are not relevant to the desired clinical examination (i.e., one or more "clinically irrelevant marker regions") and one or more marker regions whose pixel intensity values vary by less than a predetermined amount (i.e., one or more "uniform marker regions"). In such an example, if there is no marker region that is a sub-region of the region, that partially overlaps with the region, or that is a uniform marker region, then a clinically irrelevant marker region can be the relevant marker region for that region only, and the uniform marker region is within a predetermined distance of the region or is one of a predetermined number of marker regions closest to the region. Similarly, if there is no marker region that is a sub-region of the region or that partially overlaps with the region, then a uniform marker region within a predetermined distance of the region can be the relevant marker region for that region only.

[0111] In a method where no relevant marker region for a region is identified, the region itself can be considered or act as the relevant marker region for subsequent processing purposes.

[0112] Thus, in order of priority, one or more preferred relevant marker regions are: marker regions fully contained within the region (if any); marker regions that overlap with the region (if any); uniform marker regions close to the region (if any); and clinically irrelevant marker regions close to the region (if any). There may be an upper limit on the number of relevant marker regions to be used. If no marker region meets these criteria, then the region itself can act as the relevant marker region.

[0113] However, this order of priority is not required, and different embodiments or use case scenarios can utilize different orders or possible options for relevant marker regions.

[0114] In a preferred example, process 130 includes defining the position of the marker for each region such that the position lies within one of the relevant marker regions (if any) for that region. As previously mentioned, if no relevant marker region is identified, the region itself can serve as the relevant marker region.

[0115] In a preferred example, if there are multiple relevant marker regions, process 130 defines the position of the marker such that the position lies within the largest relevant marker region among the identified relevant marker regions. In other examples, if there are multiple relevant marker regions, process 130 defines the position of the marker such that the position lies within the relevant marker region closest to the centroid of the region to be marked.

[0116] In some examples, the step of defining the position of the marker for a region includes defining the position of the marker in response to the position of the centroid of at least one relevant marker region. The method aims to separate the marker from the boundary of the relevant marker region, thereby reducing the likelihood that the marker will block or obscure potentially clinically relevant information in the medical image.

[0117] For example, the marker can be positioned at or near the centroid of one of the relevant marker regions (e.g., the largest relevant marker region).

[0118] In a more complex example, multiple potential positions are defined for one of the relevant marker regions (e.g., the largest relevant marker region), and the marker is positioned at one of these potential positions.

[0119] A complete working example method for using multiple positions for a relevant marker region to define the position of the marker for a region is provided below.

[0120] For the purposes of this example, it is assumed that only a single relevant marker region is identified. For example, this relevant marker region can be the largest relevant marker region among multiple relevant marker regions (if multiple relevant marker regions are identified) or the only identified relevant marker region (if only one relevant marker region is identified).

[0121] As previously mentioned, in the case where no relevant marker region is identified for a region, the region itself can serve as the relevant marker region. For the sake of brevity, the term "relevant marker region" is used hereinafter to refer to this single relevant marker region or (where appropriate) the region itself.

[0122] Process 130 can include process 132 of identifying multiple potential positions for the marker within the relevant marker region. In this way, each relevant marker region (and thus each region) has an associated or corresponding multiple potential positions for the marker (of the region).

[0123] In one example, process 132 can simply include defining any position within the associated marker region as a potential position such that all positions within the associated marker region serve as a plurality of potential positions.

[0124] In the illustrated example, process 132 is performed by identifying a plurality of potential markers in response to the position of the centroid of the associated marker region.

[0125] Step 132 can include, for example, a sub-step 132A of identifying the centroid of the associated marker region using a validated centroid identification technique, and a sub-step 132B of identifying a plurality of positions based on the identified position.

[0126] As an example, a topological skeleton can be used to perform sub-step 132B.

[0127] Methods for determining or finding a topological skeleton are well known in the art, such as the method disclosed by Golland, Polina, W. Eric, and L. Grimson in "Fixed topology skeletons." Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662). Vol. 1. IEEE, 2000 or the "skeletonize" function provided by the "scikit-image" package of algorithms for image processing in the

[0128] Sub-step 132B can accordingly include using the topological skeleton to identify a plurality of potential positions for the marker. Each potential position lies on the topological skeleton and represents one of the K closest positions to the centroid of the associated marker region. Thus, step 132B includes identifying the K closest neighborhoods along the topological skeleton to the position of the centroid of the associated marker region. The value of K can be predetermined or responsive to the number of points of the topological skeleton, such as 10% of the total number of points of the topological skeleton.

[0129] Thus, process 132 can further include a sub-step 132C: determining or finding the topological skeleton (alternatively labeled as skeleton or skeleton structure) of the associated marker region.

[0130] An alternative method of performing step 132B is to simply identify a plurality of potential locations at a predetermined distance and / or direction from the centroid of the associated marker region. For example, the plurality of potential locations can include locations and distances of 0, 5, 10, and 20 pixels from the centroid in a plurality of directions (e.g., in at least 4 evenly distributed directions from the centroid). This will result in X.N + 1 potential locations, where X is the number of directions and N is the number of distances (where the value "1" represents the potential location at the centroid of the associated marker region).

[0131] Other suitable techniques for using the centroid of the associated marker region to define a plurality of locations will be apparent to those skilled in the art.

[0132] Of course, other methods for defining a plurality of potential locations can be used. For example, some methods can identify only locations that are greater than a predetermined distance from the boundary of the associated marker region as the plurality of potential locations.

[0133] Then, process 130 can select one of the plurality of potential locations in step 133 to act as the identified location.

[0134] Step 133 can include identifying the potential location having the minimum distance from the centroid of the region (associated with the plurality of potential locations).

[0135] Step 133 can include identifying the potential location having the maximum average distance from the centroid of any other region (or the associated marker region of any other region) identified in the segmentation information. This effectively identifies the location that is farthest (on average) from the other regions, i.e., farthest from the rest of the other centroids.

[0136] In another example, step 133 can include identifying the potential location that is farther from the boundary of the region associated with the associated marker region. This is particularly advantageous when the associated marker region is a sub-region of the region.

[0137] In some examples, process 130 further includes step 134 of determining or establishing whether the identified location (from step 133) is less than a predetermined distance from the boundary of the region.

[0138] For example, the predetermined distance can be a distance selected to reduce or minimize the amount of overlap between the marker and the boundary of the anatomical region. The boundary of the anatomical region contains important clinical information, so it is preferably to avoid the overlap of the visual representation of the marker and the boundary.

[0139] In response to a negative determination in step 134, output the identified location as the location for process 130.

[0140] In response to an affirmative determination in step 134, modify the identified location in step 135 to increase the distance between the identified location and the boundary of the region.

[0141] As an example, step 135 may include identifying the pixel in the relevant marker region that is farthest from the boundary of the region as the farthest pixel; and modifying the potential position to be at the position of the farthest pixel.

[0142] Thus, step 135 may effectively include calculating the distance transform for each pixel in the relevant marker region and identifying the position of the pixel with the maximum distance transform value as the position of the marker. This position thus effectively represents the point in the relevant marker region that is farthest from the boundary.

[0143] In addition, the marker font size may also be adjusted to fit the marker position.

[0144] Figure 2 A computer-implemented method 200 that employs any of the previously described processes is shown.

[0145] Thus, method 200 includes performing the previously described method 100.

[0146] Method 200 further includes step 220 of controlling an output user interface (i.e., a display) to provide a visual representation of a medical image and one or more markers included in marker information. The position of each marker relative to the medical image is determined in method 100 and is used to control the position of each marker, for example, to align or match with the identified position for the marker.

[0147] Specifically, for each marker, the visual representation of the marker may be controlled to overlap the visual representation of the medical image at the defined position for the marker.

[0148] It should be understood that step 220 may also include controlling the output user interface to provide a visual representation of an identification link (e.g., an arrow or a curve) between the marker and the region associated with the marker.

[0149] Step 220 may include controlling the intensity or color of the marker in response to the intensity of the marker region. For example, if the marker region has a relatively low intensity, the marker may have a relatively high intensity, and vice versa.

[0150] In some preferred examples, step 220 includes controlling the output user interface to further provide a visual representation of the boundary of each region in the medical image identified in the segmentation information.

[0151] The effect of step 220 is to provide a display of the medical image and one or more markers for each region, where the display optionally indicates the boundary of each region.

[0152] Figure 3An example medical image 300 and accompanying markers are shown, and the positions of the markers have been identified by performing the methods disclosed herein.

[0153] More specifically, each marker is located in a marker region 310, which here represents a continuous portion of the medical image 300 (here: a portion representing the lungs), and this continuous portion is not relevant to the desired clinical examination (here: the study of the heart), i.e., it is a clinically irrelevant marker region.

[0154] Figure 4 An example medical image 400 and accompanying markers are shown, and the positions of the markers have been identified by performing the methods disclosed herein.

[0155] More specifically, each marker is located in different marker regions 411, 412, which here represent continuous portions of the medical image 400, and the pixels in these continuous portions have intensity values that vary by less than a predetermined amount. Each marker is here located at the center of the corresponding marker region.

[0156] The first marker region 411 is an example of a uniform marker region located within the marker region. In this example, the first marker region is a bright uniform marker region. The markers for the first marker region can be provided in a contrasting color or brightness to the bright uniform marker region (e.g., having a low intensity, such as black).

[0157] The second marker region 412 is an example of a uniform marker region located outside the marker region. In this example, the second marker region is a dark uniform marker region. The markers located in the second marker region can be displayed using a brightness or color that contrasts with the marker region, e.g., displayed in a bright intensity (e.g., white) rather than a dark intensity.

[0158] Figure 5 Another example medical image 500 and accompanying markers are shown, and the positions of the markers have been identified by performing the methods disclosed herein.

[0159] More specifically, each marker is also located in different marker regions 511, 512, which represent continuous portions of the medical image 500, and the pixels in these continuous portions have intensity values that vary by less than a predetermined amount. Each marker is here located at the center of the corresponding marker region.

[0160] In these examples, both the third marker region 511 and the fourth marker region 512 are examples of uniform marker regions located within or being the marker region itself.

[0161] Figure 6 Another example medical image 600 and accompanying markers are shown, and the positions of the markers have been identified by performing the methods disclosed herein.

[0162] More specifically, each marker is also located in marker regions 611, 612, both of which represent contiguous portions of the medical image 600, the pixels of the contiguous portions having intensity values that vary by less than a predetermined amount. Each marker is located here at the center of the corresponding marker region.

[0163] In these examples, the fifth marker region 611 and the sixth marker region 616 are both examples of uniform marker regions. This example also shows how a single marker region can be used to carry multiple markers, for example, for different regions of the medical image 600.

[0164] Figure 7 Another example of a medical image 700 and accompanying markers is shown, where the positions of the markers have been identified by performing the method disclosed herein.

[0165] More specifically, Figure 7 A four-chamber (4CH) view of the heart is shown. Potential regions associated with a particular clinical examination of the heart include the anatomical substructures of the heart. The following list provides examples of suitable anatomical substructures for the clinical examination, along with the associated markers in parentheses: left atrium (LA); right atrium (RA); left ventricle (LV); right ventricle (RV); mitral valve (MV); tricuspid valve (TV); IV septum (IV); atrial septum (AT); AV septum (AV); spine triangle region (Sp) and descending aorta (Dar).

[0166] One or more of these regions can be identified. Then the markers for the regions are positioned to be located within the marker regions identified using the previously described method.

[0167] As a specific example, Figure 7 A marker 715 for the right atrium RA is shown. The marker 715 is positioned within a marker region 710 (represented by the outline), which is a uniform marker region within the region to be marked (e.g., within the right atrium). The marker 715 is positioned at location 711 (represented by the circular shape), which has been calculated or determined using the previously described method. Specifically, the technique using the topological skeleton 712 described previously is used to determine the location 711. The topological skeleton 712 has been included for purposes of illustrative clarity and may, for example, not be present in the marked medical image output by the present invention.

[0168] Other markers for other anatomical structures visible in the 4CH view of the heart have been positioned using similar techniques. For purposes of illustrative clarity, the outlines representing each marker region for the corresponding marker, the positions of the markers (represented by the respective circular shapes), and the topological skeletons used to determine the positions of the markers are also shown. In practice, these can be omitted from the marked medical image.

[0169] Figure 8Shows yet another example of a medical image 800 and an accompanying marker, where the location of the marker has been identified by performing the methods disclosed herein.

[0170] More specifically, Figure 8 Shows three-vessel and trachea (3VT) views. The following list provides examples of suitable regions (e.g., anatomical sub-structures) for clinical examination that utilize such views and the associated markers in parentheses: ductus arteriosus arch (Duc); aortic arch (Aor); SVC (Svc); trachea (Tra); and spinal triangle (Sp).

[0171] One or more of these regions can be identified. Then the marker for the region is positioned to be within the marker zone identified using the previously described method.

[0172] As a specific example, Figure 8 Shows a marker 815 for the ductus arteriosus arch DC. The marker 815 is located within a marker zone 810 (represented by the contour), where the marker zone 810 is a unified marker zone within the area to be marked (e.g., within the ductus arteriosus arch). The marker 815 is positioned at location 811 (represented by the circular shape), which has been calculated or determined using the previously described method. Specifically, the technique using the topological skeleton 812 described previously is used to determine the location 811. The topological skeleton 812 has been included for illustrative clarity purposes and may, for example, not be present in the marked medical image output by the present invention.

[0173] Other markers for other anatomical structures visible in the 3VT view have been positioned using similar techniques. For illustrative clarity, the contours representing each marker zone for the corresponding marker, the location of the marker (represented by the respective circular shape), and the topological skeleton used to determine the location of the marker are also shown. In practice, these can be omitted from the marked medical image.

[0174] It should also be noted that, Figure 7 and Figure 8 Shows an example where the marker zone is a dark unified marker zone. Thus, the marker can be displayed using a brightness or color contrast with the (one or more) marker zones, for example, displayed in white instead of black.

[0175] Figure 9 Shows a system 900 according to an embodiment.

[0176] System 900 includes a processor 910 and an output user interface 920. System 900 may optionally include an imaging system 930 and / or a memory or storage system 940. In some embodiments, the imaging system 930 includes an ultrasound transducer system capable of generating an ultrasound image of an object using an ultrasound imaging process. Techniques for operating the ultrasound transducer system and generating an ultrasound image are well known in the art and are not described herein for the sake of brevity. When acquiring an ultrasound image, the ultrasound transducer may be in direct contact with the object.

[0177] The processor 910 is configured to define the positions of one or more markers for a medical image.

[0178] The processor 910 includes an input interface 911, a data processor 912, and optionally an output interface 913.

[0179] The processor 910 is configured to obtain a medical image, segmentation information, and marker information. At least some of this information may be obtained via the input interface 911 of the processor 910.

[0180] The medical image defines an intensity value for each of a plurality of pixels. The segmentation information identifies the boundaries of one or more regions in the medical image. For each region in the medical image, the marker information identifies a marker for the region.

[0181] The medical image may be received / obtained via the input interface 911 from the imaging system 930 and / or the storage system 940.

[0182] The segmentation information may be received / obtained via the input interface 911 from the storage system 940. Alternatively, the segmentation information may be generated by performing one or more segmentation techniques on the obtained medical image.

[0183] The marker information may be received / obtained via the input interface 911 from the storage system 940. Alternatively, the marker information may be generated by performing one or more marker techniques on the obtained medical image by the data processor. One or more of the marker techniques may be at least partially integrated into the (one or more) segmentation techniques.

[0184] The processor 910 is further configured to, for each region in the medical image whose boundaries are identified in the segmentation information, use the data processor 912: process the medical image to identify one or more marker regions (if any), where each marker region represents a continuous portion of the medical image: which is not relevant to a desired clinical examination and / or whose pixels have intensity values that vary by less than a predetermined amount; and, in response to the positions of the one or more marker regions (if any), define the positions of each marker for the one or more regions identified in the marker information.

[0185] In this way, the processor 910 is capable of determining the position of each marker in the marker information.

[0186] The processor 910 may also be configured to control the output user interface 920 via the output interface 913, for example, to provide a medical image and a visual representation of one or more markers included in the marker information. Thus, the position of each marker relative to the medical image is determined 100 and used to control the position of each marker, for example, to align or match with the identified position for the marker.

[0187] The processor 910 may be adapted or configured to perform or implement any of the methods described herein. Those skilled in the art will be able to readily make any necessary modifications to the processor to perform such methods.

[0188] The processor 910 may be implemented in a variety of ways using software and / or hardware to perform the various functions required. The processor 910 may include one or more microprocessors that can be programmed using software (e.g., microcode) to perform the required functions. However, the processor 910 may be implemented with or without such microprocessors and may also be implemented as a combination of dedicated hardware for performing some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) for performing other functions.

[0189] Examples of processor components that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0190] In various embodiments, the processor 910 may be associated with one or more storage media, such as volatile and non-volatile computer memories 915, such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processors (e.g., data processor 812), perform the required functions. The various storage media may be fixed within the processor or may be transportable, such that the one or more programs stored thereon can be loaded into the processor. The storage media may also be located external to the processor 910, such as in the cloud.

[0191] It should be understood that the disclosed methods are preferably computer-implemented methods. Thus, the concept of a computer program is also proposed, which includes code (e.g., instructions for a computer / processor) for implementing any of the described methods when the program runs on a processor such as a computer. Thus, different parts, lines, or blocks of the code of a computer program according to an embodiment may be executed by a processor or computer to perform any of the methods described herein.

[0192] There is also provided a local or remote storage medium storing or carrying a computer program or computer code which, when executed by a processor, causes the processor to perform any of the methods described herein.

[0193] In some alternative embodiments, the functions noted in the block diagrams or flowcharts may not occur in the order noted in the figures. For example, depending on the functions involved, two consecutive blocks shown may actually be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order.

[0194] By studying the drawings, the disclosure and the appended claims, those skilled in the art can understand and realize variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may implement the functions of several items recited in the claims. The measures recited in mutually different dependent claims may be advantageously combined. If a computer program is discussed above, it may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium provided together with or as part of other hardware, but it may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. If the term "adapted to" is used in the claims or the specification, it should be noted that the term "adapted to" is intended to be equivalent to the term "configured to". Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A computer-implemented method (100) for defining the positions of one or more markers for one or more first regions in a medical image of an object, the method comprising: Receiving (111) the medical image, wherein the medical image defines an intensity value for each of a plurality of pixels; Obtaining (112) the segmentation information by receiving the segmentation information or by generating the segmentation information from the medical image, the segmentation information identifying the boundaries of the one or more first regions; Obtaining the marker information by receiving the marker information or by generating the marker information from the medical image, the marker information identifying (113) a marker for each of the one or more first regions in the medical image; Identifying (120) one or more marker regions in the medical image if there are one or more marker regions in the medical image, wherein each marker region represents a continuous portion of the medical image, the pixels of the continuous portion having intensity values that vary by less than a predetermined amount; and Defining (130) the positions of the markers for each of the one or more first regions identified in the marker information based on the identified one or more marker regions.

2. The method according to claim 1, wherein The medical image comprises an ultrasound image.

3. The method according to claim 1, wherein, For each marker region, the intensity values of the pixels of the marker region are lower than a global threshold intensity value.

4. The method according to any one of claims 1 to 3, wherein If there are one or more marker regions in the medical image, the step (120) of identifying the one or more marker regions in the medical image comprises, for each of the one or more first regions: Identifying any sub-region of the region that represents a continuous portion of the region, the pixels of the continuous portion having intensity values that vary by less than a predetermined amount, as a marker region.

5. The method according to claim 4, wherein The step of defining the positions of the markers for each of the one or more first regions comprises, for each of the one or more first regions, defining the position of the marker for the region in response to the position of any identified sub-region of the region.

6. The method according to any one of claims 4 or 5, wherein For each of the one or more first regions, the intensity value of each pixel in any identified sub-region is lower than a region-specific threshold intensity value, the region-specific threshold intensity value being derived from the minimum intensity value of any pixel within the region.

7. The method according to any one of claims 4 to 6, wherein Each sub-region represents a portion of the region having a size that is not less than a predetermined percentage of the size of the region.

8. The method according to claim 7, wherein, The predetermined percentage is not less than 10%, and preferably not less than 20%.

9. The method according to any one of claims 1 to 8, wherein, The step of defining the positions of the markers for each of the one or more first regions comprises, for each of the one or more first regions: Identifying any marker region that overlaps the region as an overlapping marker region; and Defining the position of the marker for the region in response to the position of any overlapping marker region.

10. The method according to claim 9, wherein, The step of defining the positions of the markers for each of the one or more first regions comprises, for each of the one or more first regions: In response to failing to identify any overlapping marker regions, if there are a predetermined number of marker regions closest to the region, the position of the marker for the region is defined in response to the positions of the predetermined number of marker regions closest to the region.

11. The method according to any one of claims 1 to 10, wherein, The step of defining the position of each marker for the one or more first regions includes, for each of the one or more first regions, further defining the position of the marker for the region in response to the centroid of the region.

12. The method according to any one of claims 1 to 11, wherein, The step of defining the position of each marker for the one or more first regions includes, for each of the one or more first regions, if there is a centroid of any other region, further defining the position of the marker for the region in response to the centroid of any other region.

13. A computer program product comprising instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 12.

14. A processor (910) for defining the position of one or more markers for a medical image of an object, the processor being configured to perform the method according to any one of claims 1 to 12.

15. A medical imaging system, comprising: An imaging system 930 for generating a medical image; And A processor according to claim 14, coupled to the imaging system to define the position of one or more markers for the ultrasound image.

Images