System and method for supporting peer review and contouring of medical images - Patents.com

Abstract

A method and system for contouring medical images for display on a display device is described. The method includes the steps of: providing at least one medical image to be contoured; providing a window width parameter W and a window level parameter L; displaying the medical image according to the W, L parameters; and performing the following contouring steps, which include: determining a local image intensity of the at least one medical image for each position of the contour when the contour is generated; determining whether the window width parameter W and the window level parameter L are suitable for each position of the contour on the medical image according to the determined local image intensity when the contour is generated; if the parameters are not suitable, alerting a user that at least one of the parameters W and L is not suitable for the most recently generated position on the contour; adjusting at least one of the window width parameter W and the window level parameter L to be suitable for the most recently generated position on the contour, and displaying the medical image according to the adjusted W, L settings. A method and system for reviewing previously contoured medical images is also described.

Description

[Technical field]

[0001] The present invention relates to the field of medical imaging, and in particular to contouring, reviewing or editing, for example by a radiation oncologist, of anatomical or target structures for radiation treatment planning in medical images of a patient. The present invention provides a tool to ensure that a user is using proper scaling of the display of a medical image when contouring a structure or reviewing the contour of a structure on a medical image. [Background technology]

[0002] In medical imaging, it may be necessary to depict structures or other objects on a medical image of a patient. For example, in radiation therapy planning, it is necessary to take a volumetric (3D) image of the patient, for which the treatment plan can be designed and the delivery of the radiation dose can be estimated. A 3D image / volume image is an image containing a volume that is usually created in medical imaging from several 2D images. An element of a 3D image is known as a voxel. A 2D image is an image containing a slice in a 2D plane. This slice may have a given thickness and thus constitute a volume in the medical image as a cross-section of the patient. An element of a 2D image is usually called a pixel.

[0003] Such 3D images are called simulated CT (computed tomography) images. Part of this process involves the outline of the anatomical and target structures on the patient's image so that the estimated radiation dose delivery to each of the structures can be calculated and reported to the clinical staff performing the treatment planning. In radiation therapy planning, this image is usually a CT image, but MRI (magnetic resonance imaging) images may also be used.

[0004] The size and location of the tumor as shown on the image are important indicators that contribute to the decision process regarding the choice of the best treatment to be offered to the patient. In order to make the best clinical choice, accurate delineation of both healthy organs and cancerous regions on the patient's images is required. For example, in radiation treatment planning, accurate delineation is required when planning to maximize the radiation dose to the tumor while minimizing the dose to healthy tissues.

[0005] The process of contouring or delineating is known in the field of radiation oncology as "contouring". The term "segmentation" is sometimes used as well, which may be used to refer to both the process and the result. Auto-segmentation or auto-contouring refers to when the contouring task is performed automatically by an image processing system (Sharp, 2014; Lustberg, 2018). Semi-automatic or interactive / manual tools also exist (Ramkumar et al., 2016).

[0006] Automation in the contouring process not only helps to reduce delineation time, but also improves the consistency of image segmentation and reduces inter- and intra-observer variability (Brouwer et al., 2012; Vinod et al., 2016; Vinod et al., 2016(a); Patrick et al., 2021). However, current automated solutions still require manual editing to correct errors in the automated system (Lustberg et al., 2018; Brouwer et al., 2020). Furthermore, automated solutions are still limited in terms of the anatomical regions that can be contoured or the image modalities that are supported. Therefore, manual contouring is still routinely performed in the clinic.

[0007] Manual contouring is typically performed using a workstation with a suitable image display. Today, most medical imaging techniques are three-dimensional, typically involving a stack of 2D medical image slices, but contours can be drawn around each structure for each image slice. 3D images are usually displayed slice-by-slice on a screen in one of three planes: axial, sagittal, coronal (these are well-known anatomical planes in medical imaging), or according to the image acquisition plane. Images may also be displayed by resampling in any plane. Multiple 2D images can also be displayed simultaneously, on the same screen or on multiple screens. The clinician can then manually generate new contours or review and / or edit the existing set of displayed contours.

[0008] Contours are commonly displayed as an overlay on medical image slices. Contours can be filled or displayed as curves with a specific line width. Color is commonly used to improve contrast, as most medical images are scalar-valued (non-color) images and are therefore displayed in grayscale. Medical images produced by current medical imaging techniques, including X-ray, CT, MRI PET (Positron Emission Tomography), typically have scalar values ​​coded with 12-16 bits per value (pixel). This corresponds to 4,096-65,536 stored values. These stored values ​​can be mapped to real-world values. For example, a CT image may have 1024 stored values, which can be mapped to real-world units of 0 Hounsfield units of value. Furthermore, for some 3D image modalities, a different mapping to real-world values ​​can be applied to each slice, such that the dynamic range of the 3D image is apparently much larger. For example, a PET image may have 1024 stored values ​​that map to 136.79 Bq / mL (Becquerels per milliliter) on one slice of the image, while on a different slice, the 1024 stored values ​​may map to 2440.99 Bq / mL. This conversion between stored values ​​and real-world values ​​is determined by the file format standard in which the medical image is stored (e.g., DICOM). Further references to stored values ​​may be interpreted as including the stored values ​​after conversion to real-world units according to the medical image file format.

[0009] Therefore, the range of storage values ​​in medical images is much wider than the range that current medical displays can support, which generally support 8 to 10 bits. Guidelines and recommendations for assessing the display quality of display devices used in medicine are regularly published and updated (Bevins et al., 2019). Practical recommendations on how to implement quality assurance processes according to these guidelines are also available (Bevins et al., 2020). Interestingly, most of the patterns and quality assessment processes described in (Bevins et al., 2019) use an 8-bit representation (256 shades of gray). This is not surprising, since a stronger limitation to the visualization of images is the inability of human observers to simultaneously distinguish a wider range of gray shades (between 700 and 900 according to (Kimpe and Tuytschaever, 2007)).

[0010] Due to these two limitations, display technology and human perception, a common technique in image processing techniques is to define a display range, known as a window, within which image intensity is mapped to displayed shades of gray. This is usually defined by two values: ● W: Window width: this is typically the length of the dynamic range of intensities (eg a window from a to b has width W = ba). ● L: Window level: intensity value at the center of the window (eg a window from a to b has L=(a+b) / 2).

[0011] The image is then scaled so that only image intensity values ​​between LW / 2 and L+W / 2 are rendered on the display. The restriction of numerical values ​​to a given range, here between LW / 2 and L+W / 2, is known as clamping. Clamping is the restriction of numerical values ​​to a given range [ab]. Specifically, values ​​are linearly scaled if they are within the given range; set to a lower value a if they are less than a; set to a higher value b if they are greater than b. Intensity values ​​lower than LW / 2 are clamped and all are displayed on the screen as the lowest value (black). Similarly, intensity values ​​greater than L+W / 2 are clamped and displayed on the display as the highest value (white). Intermediate image intensity values ​​are usually linearly scaled to cover the full range of the device's display. Such a transformation is commonly called window / leveling or clamped linear intensity transformation. In some cases, the window level parameters W, L are simply called the window. Exemplary terms such as lung window or bone window refer to W,L settings that are commonly used or recommended for viewing lung tissue or bone anatomy. The unclamped intensity values ​​of a (W,L) setting are intensity values ​​ranging from LW / 2 to L+W / 2, commonly written as [LW / 2,L+W / 2], and sometimes simply referred to as a window.

[0012] It should be noted that scalar intensity values ​​are often displayed as shades of gray, but color mapping may be used to convert the scalar intensity values ​​into a color display range, so the term "shades of gray" should be interpreted as the scalar image values ​​before mapping to display colors.

[0013] Recommendations regarding useful ranges of gray shades are made given the clinical question the physician needs to answer or given the task at hand: for example, a useful range of intensity values ​​for seeing lung structures on a CT scan is different from a useful range of intensity values ​​for liver structures or head and neck structures.

[0014] Guidelines are available on appropriate values ​​of W, L for CT (Bongartz, 2000). As an example, in a typical CT chest examination, the recommended window width (W) for soft tissue is 300-600 Hounsfield Units (HU) and the window level (L) is 0-30 HU. However, the recommended window width (W) for lung parenchyma is 800-1,600 HU and the window level (L) is -500-700 HU. A specific exemplary value for displaying lung tissue could be set as W=1000 HU and L=-500 HU, thus defining a range of intensities from [-500-1000 / 2,-500+1000 / 2]=[-1000,0], referenced by the lung window.

[0015] FIG. 3 shows several example intensity transformations for some typical window W, L parameter sets recommended for displaying CT images. In this figure, examples of recommended (W, L) parameter sets for CT are shown: bone anatomy 304, liver 305, general chest 306, lungs 307. For clarity of the figure, an x-axis 313 is also depicted at 314. This x-axis represents the stored image values ​​mapped to real-world (Hounsfield units) values; the range shown (-1000 to 2000 HU) is typical for CT imaging. The percentage-graded y-axis 311 gives the corresponding shades of grey, as indicated by the colour bar 310 from black to white. The window levels 308 of the different transformations are always rendered in the middle (50%) of the shades of grey 301. The window level L 308 and window width W 209 are shown for the transformation 312 of the bone anatomy, depicted in bold lines. In this example, a clamped linear intensity transformation is shown for L=480HU 208 and W=2500HU 309, thus defining a range of unclamped image intensities from -770 to +1700HU, linearly scaled to be displayed between 0% and 100% of shades of grey. Image intensities lower than -770HU 316 are clamped and displayed as the lowest value (black) on the display, and image intensities higher than +1700HU 215 are also clamped and displayed as the highest value (white) on the display. In the clinic, the quality of the display devices is regularly checked and the clinical team is trained and educated on guidelines regarding the appropriate (W,L) parameters to be used. In some cases, the clinician may need to change the (W,L) settings several times during the contouring. Indeed, organs and tumors may be surrounded by different tissue types, which generally require different (W,L) settings to "optimally" display and visualize the border between normal and tumor tissue on the screen. An example is a lung tumor invading the chest wall near the ribs, as shown in Figure 11. Different (W,L) settings are required to assess the correct boundaries between different tissue types.For example, the window / level settings typically used for lungs may be best for assessing the boundary between a tumor 1101 and a lung 1102, while the window / level settings typically used for bones may be best for assessing the boundary between a tumor 1101 and a rib 1103.

[0016] To further illustrate this, Figures 12a, 12b, 12c and 12d show an example of an intensity profile of the CT image values ​​as they are loaded in Figure 12a, and after application of the CT-lung window 307 in Figure 12b, the CT-bone window 304 in Figure 12c and the CT-chest window 306 in Figure 12d. In this example, the intensity profile corresponds to the anatomical region indicated by the dashed line in Figure 11, starting at 1107 and ending at 1108. A CT image intensity profile 1201 is shown in Figure 12a. The y-axis 1203 represents the loaded CT image intensity values ​​in Hounsfield units. The x-axis 1202 represents the spatial location graded in pixels, with the first pixel on the intensity profile corresponding to location 1107 located in the lungs and the last pixel corresponding to location 1108 located in the image background. As can be seen in FIG. 11, this line profile spans several tissue types: lung, tumor, bone, fat, muscle, fat, and finally air (outside the body). These different regions can be observed on the line profile 1201, which are labeled by letters A-F and summarized in the legend 1210. The boundaries between the different tissues are indicated by vertical dashed lines labeled 1204-1209. FIG. 12b shows the intensity profile 1221 after application of a CT-lung window of (W,L)=(1000,-500) 307 for presentation purposes. The x-axis 1222 is the same as 1202 in FIG. 12a. The boundaries between the different tissues, here labeled 1224-1229, are also in the same position as in FIG. 12a. The y-axis 1223, graded in percentages, gives the corresponding shades of grey, as indicated by the color bar 1230 from black to white. The intersections of intensity profile 1221 with horizontal lines 1231 and 1232 indicate clamped pixels. In this example, the boundary 1225 between the tumor and bone is not visible on the displayed image because both the tumor and bone intensity values ​​are outside the range of display values ​​and are therefore clamped.Also, the boundaries 1227 and 1228 between the muscle tissue and the surrounding fat tissue cannot be accurately delineated due to the clamping image intensity of the muscle pixels and the low image contrast at the boundary locations when both tissues are displayed at the highest gray shade. In the lung window, the image contrast is sufficient to recognize the correct boundary locations of 1224 and 1229.

[0017] FIG. 12c shows the intensity profile 1241 after application of a CT-bone window with (W,L)=(2500,480) 304. The x-axis 1242 is the same as 1202 in FIG. 12a. The boundaries between the different tissues, here labeled 1244-1249, are also in the same position as in FIG. 12a. The y-axis 1243, graded in percentages, gives the corresponding shades of grey, as indicated by the colour bar 1250 from black to white. The intersections of the intensity profile 1241 with the horizontal lines 1251 and 1252 indicate the clamped pixel intensities. In this example, only the boundaries with the bone tissues 1245 and 1246 have enough contrast to be accurately drawn or reviewed. The large window width W=2500 of the bone window makes it impossible to distinguish the soft tissues (muscle, tumour, fat etc.) and therefore boundaries such as 1247 and 1248 cannot be accurately drawn or reviewed. Accurate discrimination between soft tissues requires displaying the image with smaller W values. This is illustrated in FIG. 12d, which shows the intensity profile 1261 after application of a CT-chest window of (W,L)=(350,40) 306. The x-axis 1262 is the same as 1202 in FIG. 12a. The boundaries between the different tissues, here labeled 1264-1269, are also in the same positions as in FIG. 12a. The y-axis 1263, graded in percentages, gives the corresponding shades of grey, as indicated by the colour bar 1270 from black to white. The intersections of the intensity profile 1261 with the horizontal lines 1271 and 1272 indicate clamped pixels. In this example, the lung tissue A and the image background F are clamped, making it difficult to review or delineate the contours at the boundary locations 1264 or 1269. Also, the bone tissue C is clamped, making it difficult to recognise the correct positions of the bone boundaries 1265 and 1266.

[0018] Current medical image visualization devices allow the user to quickly switch between different (W,L) settings and may also allow the user to freely and interactively (e.g., via a computer mouse) adjust the contrast of the displayed image. In other words, the user sets custom (W,L) parameters for the displayed image. Although such technical features are useful, they are limited since there is nothing to prevent the user from contouring, reviewing or editing contours with inappropriate (W,L) parameters. Furthermore, guidelines regarding specific (W,L) parameters may only be defined for image modalities that use standardized intensity values, such as Hounsfield units for computed tomography, whereas MRI is not a quantitative technique and the range of image intensities obtained is highly diverse. In other words, current solutions facilitate contouring by allowing the application of variables (W,L) to the displayed image to improve contrast visualization, but do not proactively inform the user that contouring is being performed with a suboptimal image window width / level. This may therefore be a source of errors in the contouring and review process. If such errors go undetected, they may affect the radiation treatment plan and cause harm to the patient. [Prior art documents] [Non-patent literature]

[0019] [Non-Patent Document 1] Bevins N, Flynn M, Silosky M, Marsh R, Walz-Flannigan A, Badano A. AAPM Report 270: Display Quality Assurance. American Association of Physicists in Medicine; 2019. https: / / www.aapm.org / pubs / reports / RPT_270.pdf [Non-Patent Document 2] Bongartz G at al; “European guidelines on quality criteria for computed tomography”, 2000, EUR 16262 EN, https: / / op.europa.eu / en / publication-detail / - / publication / d229c9e1-a967-49de-b169-59ee68605f1a [Non-Patent Document 3] Bevins NB, Silosky MS, Badano A, Marsh RM, Flynn MJ, and Walz-Flannigan AI. Practical application of AAPM Report 270 in display quality assurance: A report of Task Group 270. Med Phys 2020. [Non-Patent Document 4] Lustberg T, et al. “Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer”, Radiotherapy and Oncology, 126(2):312-7, 2018. [Non-Patent Document 5] Sharp G, et al. “Vision 20 / 20: perspectives on automated image segmentation for radiotherapy.” Medical physics, 41, 5, 2014. [Non-Patent Document 6] Ramkumar, A. et al. "User Interaction in Semi-Automatic Segmentation of Organs at Risk: a Case Study in Radiotherapy." Journal of Digital Imaging, vol. 29, pp. 264-277, 2016

Waterfall 7

Fashion 8

Wood 9

Wood 10

[0020] Therefore, there is a need for a method and system for notifying a user while contouring, reviewing or editing one or more contours on a medical image that one or more of the current image window width and level parameters used to display the image with the one or more contours are inappropriate for the task of contouring, editing or reviewing the contours. Such a system and method are described and disclosed by the present invention. [Means for solving the problem]

[0021] According to the present invention, there is provided a method for contouring a medical image displayed on a display device, the method comprising the steps of: providing at least one medical image having one or more structures to be contoured; providing a window width parameter W and a window level parameter L for the at least one medical image; displaying the medical image according to the W, L parameters; and performing the following contouring steps, the contouring steps comprising the steps of: determining a local image intensity of the at least one medical image for each position of the contour when a contour is generated for a structure on the image; determining whether the window width parameter W and the window level parameter L are suitable for contouring each position of the contour on the medical image according to the determined local image intensity when the contour is generated; if at least one of the parameters is not suitable for contouring, providing a warning that at least one of the parameters W and L is not suitable for the most recently generated position on the contour; adjusting at least one of the window width parameter W and the window level parameter L to be suitable for contouring the structure at the most recently generated position on the contour and displaying the medical image according to the adjusted W, L settings.

[0022] In a further embodiment of the present invention, there is also provided a method for reviewing previously contoured medical images displayed on a display device, the method comprising the steps of: providing at least one medical image and at least one contour on the medical image to be reviewed; providing for the medical image a window width parameter W and a window level parameter L; displaying the medical image and displaying the at least one contour according to the W, L parameters; selecting at least one contour on the previously contoured image for review; and performing the following contour review steps, the contour review step comprising: selecting at least one medical contour for at least one portion of the selected contour to be reviewed; determining a local image intensity of the medical image to be reviewed; determining according to the determined local image intensity that at least one of a current window width parameter W and a window level parameter L is not suitable for reviewing at least one portion of the at least one contour; providing a warning that at least one of the parameters W and L is not suitable for reviewing at least one portion of the contour, adjusting at least one of the window width parameter W and the window level parameter L so that it is suitable for reviewing at least one portion of the contour, reviewing a further section of the contour portion using the adjusted parameters, and displaying the medical image according to the adjusted W, L settings.

[0023] In an embodiment of the invention, the contouring or contour review steps are repeated until one or more structures to be contoured have been contoured or all of at least one selected contour on a previously contoured image have been reviewed.

[0024] Preferably, adjusting at least one of the window width parameter W and the window level parameter L to be suitable for contouring at least one structure or for reviewing at least one portion of a contour is performed by a user. In an alternative embodiment of the invention, adjusting at least one of the window width parameter W and the window level parameter L to be suitable for contouring at least one structure or for reviewing at least one portion of a contour is an automatic adjustment.

[0025] In an embodiment of the invention, the method further includes a contour review step, which includes the steps of detecting one or more portions of the selected contour that are suitable for review using the current window width and window level parameters, and reviewing the detected one or more contour portions.

[0026] More preferably, the method also includes the step of editing or correcting one or more of the contour portions after the detected contour portions have been reviewed.

[0027] Preferably, the method further comprises the steps of determining whether the parameters W, L are suitable for the current task of contour generation or contour review, and if one or more of the detected parameters are not suitable for the current task, providing a warning that one or more of the parameters are not suitable and suggesting at least one new parameter suitable for the current task.

[0028] Preferably, the parameter suggestions for one or more parameters are automatic. More preferably, the parameter suggestions for one or more parameters are applied automatically.

[0029] In an embodiment of the present invention, determining the appropriateness of at least one of the window width and window level parameters is performed using a machine learning algorithm.

[0030] Preferably, the machine learning algorithm uses information from at least one of local image intensity and spatial information of the image.

[0031] Preferably, the method further comprises, when a contour is generated or selected for review, determining a patch around either the most recently generated part of the contour or the part of the contour for review and evaluating the local image intensity within the patch to determine whether at least one of the W, L parameters is suitable or needs to be adjusted.

[0032] More preferably, the local image intensities within the patch used to determine the suitability of the parameters are calculated from one or more of the mean intensity value, maximum intensity value, minimum intensity value, median intensity value, configurable percentiles of intensities, or a statistical measure between the distribution of patch intensity values ​​and a predefined parametric function of the W and L values.

[0033] Preferably, the local image intensity is compared to a predetermined or user configurable threshold function for at least one of the current W, L parameters.

[0034] More preferably, adjusting at least one of the window width parameter W and the window level parameter L is performed by the user in response to one or more of the following: a pop-up window with suggestions for new W, L settings; a context sensitive menu accessed via the display; a mouse hover over an image or outline area; or an interaction mechanism such as a mouse click, a keyboard hotkey or voice control.

[0035] More preferably, the alert provided to the user is a visual or audio alert. Preferably, the alert is provided by one or more of the following: a message in the user interface; a change in color of the contour being created or edited; a change in style or thickness of the contour being created or edited; a change in color of the area surrounding the contour being created or edited.

[0036] Preferably, the medical image is a CT scan, an MRI scan or a PET scan.

[0037] More preferably, when a contour is created or reviewed, the contour is saved or provided as output.

[0038] In an embodiment of the present invention, after the contour editing or creation is complete, the above steps are repeated so that multiple contours are created or reviewed on the medical image.

[0039] Preferably, at least one of the window width parameter W and the window level parameter L at the start of the contouring or review process is one of: image modality dependent; workflow dependent; set from previous use of the system; or user configured or defined from parameters of stored images.

[0040] In an embodiment of the invention, the medical image is a 2D medical image, a 3D medical image or a time series of medical images.

[0041] Further preferably, a minimum value of at least one of the W, L parameters is set for contouring, editing or contour review. Preferably, a maximum value of at least one of the W, L parameters is set for contouring, editing or contour review. In a preferred embodiment of the invention, the contouring is one of manual contouring, semi-automatic contouring or automatic contouring.

[0042] In a further embodiment of the present invention, a system for analysing medical images is also provided, the system comprising: a display for displaying at least one medical image to be contoured; and a processor for setting a window width parameter W and a window level parameter L for the display device to correct values ​​for a start position of a contour of a structure on the at least one medical image to be contoured, the processor determines a local image intensity of the at least one medical image for each position of the contour when the contour is generated, the processor determines whether the window width parameter W and the window level parameter L are correct for each position of the contour on the medical image according to the determined local image intensity when the contour is generated, and if at least one of the parameters is incorrect, warns a user that at least one of the parameters W and L is incorrect for the current position on the contour and adjusts at least one of the window width parameter W and the window level parameter L to be correct for the current position on the contour.

[0043] In a further embodiment of the present invention, there is also provided a system for enabling a user to review previously contoured medical images, the system comprising: a display for displaying at least one previously contoured medical image to be reviewed using default window width parameters W and window level parameters L; and a processor for selecting at least one contour on the previously contoured image for review, the processor being configured to: detect one or more portions of the selected contour suitable for review using the default window width and window level parameters, thereby enabling a user to review the detected one or more contour portions; detect that at least one of the default window width parameters W and window level parameters L is not suitable for a further section of the contour to be reviewed, alert the user that at least one of the parameters W and L is not suitable for the further section of the contour; adjust at least one of the window width parameters W and window level parameters L to be correct for the further section of the contour; and enable the user to review the further section of the contour portion using the adjusted parameters.

[0044] In an embodiment of the present invention, the display displays the medical image according to the adjusted W, L settings.

[0045] Further preferably, the processor is configured to automatically detect at least one of a current window width parameter W and a window length parameter L and determine whether the detected parameters are suitable for a current task of contour generation or contour review.

[0046] According to an embodiment of the invention, there is also provided a computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the above-mentioned method. [Brief description of the drawings]

[0047] Further details, aspects and embodiments of the present invention are now described, by way of example only, with reference to the drawings, in which like reference numbers are used to identify like or functionally similar elements, and in which elements are illustrated for simplicity and clarity and have not necessarily been drawn to scale.

[0048] [Figure 1] 1 is a simplified block diagram of an example medical image processing system. [Diagram 2] FIG. 1 illustrates a simplified process of loading medical images into a visualization station for manual contouring, contour review or editing. [Diagram 3] FIG. 1 illustrates typical applicable window (W,L) settings for displaying CT images. [Figure 4] FIG. 13 is a schematic diagram of the intensity at contour locations that may be at the edge of an application window W, L parameters of a displayed CT image. [Diagram 5] FIG. 13 is a schematic diagram showing how intensity at contour locations may be at the edge of an application window W, L parameters of a display image with non-normalized intensity values. [Figure 6] 1 is a flow chart of an embodiment according to the present invention. [Figure 7] 4 is a flow chart of an alternative embodiment according to the present invention. [Figure 8] 4 is a flow chart of an alternative embodiment according to the present invention. [Figure 9a] FIG. 2 is a schematic diagram of a 2D image slice during a contouring operation. [Figure 9b] FIG. 1 is a schematic diagram of a 2D image slice during a contour review operation. [Figure 10] FIG. 1 is a schematic diagram of the proposed suitable new window W, L parameters for the display image system. [Figure 11] FIG. 2 shows an example of a simplified style of axial image slices of a patient's chest. [Figure 12a] FIG. 2 shows an example of an intensity profile of a CT image of a patient. [Figure 12b]FIG. 13 shows an example of the intensity profile of a CT image of a patient after application of the CT-Lung window W, L settings to display the image. [Figure 12c] FIG. 13 shows an example of the intensity profile of a CT image of a patient after application of the CT-Bone window W, L settings to display the image. [Figure 12d] FIG. 13 shows an example of the intensity profile of a CT image of a patient after application of the W, L settings of the CT-Chest window to display the image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0049] Referring now to FIG. 1, there is shown a simplified block diagram of an example of a medical image processing system 100 configured to enable displaying to a user medical images for use in the method of the present invention. Preferably, the medical images are CT or MRI images, although other imaging methods such as PET scans may be used. In the illustrated example, the medical image processing system 100 includes one or more user terminals 101, including, for example, workstations, configured to access medical images and / or contours stored, for example, in a database 102 or other data storage device. In the illustrated example, a single database 102 is shown. However, it will be appreciated that the user terminal 101 may be configured to access medical images from two or more data storage devices. Furthermore, in the illustrated example, the database 102 is shown as being external to the user terminal 101. However, it will be appreciated that the user terminal 101 may also be configured to access medical images stored locally, in a local storage module, indicated at 110, on one or more internal storage elements, such as a memory element indicated at 103 or a disk element indicated at 109. The user terminal 101 further includes one or more signal processing modules, such as a signal processing module generally designated 104. The signal processing module is configured to execute computer program code stored, for example, in the local storage module 110. In the illustrated example, the signal processing module 105 is configured to execute computer program code including one or more of the automated processing components to detect suitable portions of the displayed contour for review. In the illustrated example, the signal processing module 104 is further configured to execute computer program code including one or more image and contour display components 106 configured to display images with window / level W, L settings and to display visual alerts to the user regarding the suitability of the current W, L display settings for the contouring or contour review process, such as on a display screen 107.The medical image processing system 100 may further include one or more user input devices, generally indicated at 108, for enabling a user to interact with computer program code or the like executing on the signal processing module 104.

[0050] FIG. 2 shows a simplified process of loading medical images into a visualization station or other display for contouring, reviewing or editing contours of structures on the medical images. Preferably, the method uses an image processing system such as that shown in FIG. 1. The process begins in step 201 with loading one or more medical images (which may be 2D or 3D, or a time series of medical images). It is understood that the process of loading medical images involves conversion between stored values ​​and real-world values, as determined by the file format standard in which the medical images are stored (e.g., DICOM). Preferably, the medical images are either CT images or MRI images, and are 10-16 bit images. Optionally, associated contours for one or more medical images may also be loaded. These may have been generated, for example, by an automatic contouring system, and therefore may require review and potentially editing. The image intensity values ​​are then converted (202) with the appropriate image window width / level and quantized (implicitly depending on the bit depth of the data type used) (203) before sending to the display system (8-bit, 10-bit) for display on the screen (204). If existing contours are loaded with the image data, these are also preferably displayed as an overlay of the image on the screen. The contouring, editing or review process can then begin (205). The one or more contours generated during contouring can be generated manually, semi-automatically or automatically. In an embodiment of the invention, at least one of the image window parameters W, L may need to be adjusted one or more times while the user is contouring the image (206) or while reviewing / editing an existing contour on the image. Once the contouring or review process is finished, the contour is saved or stored (207) for further use in the clinical workflow.

[0051] FIG. 3 shows some examples of intensity transforms 300 for some typical window W, L parameter sets recommended for displaying CT images, according to an embodiment of the present invention. For the sake of clarity of this figure only, an x-axis 313 is also depicted at 314. This axis represents stored image values ​​mapped to real-world values, and the range shown (-1000 to 2000 HU) is typical of CT imaging. The y-axis 311, graduated in percentages, gives corresponding shades of gray, as shown by the color bar 310 from black to white. In this figure, examples of recommended W, L parameters are shown for the bone anatomy 304, liver 305, general chest 306, and lungs 307. Of course, other parameter sets may be provided for these anatomy or other anatomical features. Window level L 308 and window width W 309 parameters are shown for the intensity transform 304 of the bone anatomy. The window level L 308 is rendered in the middle (50%) 301 of the available shades of grey for display. For clamped linear transformations, the window level value is always rendered in the middle (50%) of the available shades of grey. The intersections of the horizontal dashed line 301 with the linear parts of the different transformations 304, 305, 306, 307 define the corresponding window levels on the x-axis. In this diagram, the intersections are labeled 312 only for bone 304; it can be seen that there are intersections for all of the other settings 305-307, but these are simply not labeled for ease of understanding the diagram. Simply for clarity of this diagram, the clamped linear intensity transformation is highlighted for bone window setting 304 (bold line) with L=480HU 308 and W=2500HU 309, thus defining a range of unclamped image intensities from -770 to +1700HU, linearly scaled to be displayed between 0% and 100% shades of grey. Image intensities below -770HU 316 are clamped and displayed as the lowest value (black) on the display, and image intensities above +1700HU 315 are also clamped and displayed as the highest value (white) on the display. This clamping is not shown for the other window settings 305, 306 and 307, but is assumed.Although not commonly used for CT, the linear portion of the transformation can be replaced with a non-linear transformation. Horizontal dashed lines 303 (corresponding to 100% on the y-axis) and 302 (corresponding to 0% on the y-axis) represent the maximum and minimum display values.

[0052] FIG. 11 shows an example in simplified style of an axial image slice 1100 of a patient's chest. Several different tissue types are represented diagrammatically in a cross-section 1105 of the body. Seven tissue types can be identified using the legend 1106. In this example, the patient has a tumor 1101 in the left lung 1102 that has invaded the chest wall 1104 and is near the rib cage 1103. The tumor tissue is adjacent to three different normal tissue types, and each of these different normal tissue types requires different W, L settings to allow the boundary between the normal tissue type and the tumor tissue to be "optimally" displayed on the screen. Assuming that the image slice is taken from a CT scan and that the bone window 304 and the lung window 307 are the two extreme window settings, most of the tumor boundary will appear saturated when using either of the two windows. For example, when using a lung window of (W,L)=(1000,-500), image intensities higher than 0 HU are clamped; soft tissue (including tumor tissue) and bone tissue are likely to have intensity values ​​higher than 0 HU on a CT scan and will appear saturated or with very low contrast on the display when using the lung window.

[0053] To further illustrate this, Figures 12a, 12b, 12c and 12d show an example of an intensity profile of CT image values ​​loaded in Figure 12a and after application of the CT-lung window 307 in Figure 12b, the CT-bone window 304 in Figure 12c and the CT-chest window 306 in Figure 12d. In this example, the intensity profile corresponds to the anatomical region indicated by the dashed line in Figure 11, starting at 1107 and ending at 1108. A CT image intensity profile 1201 is shown in Figure 12a. The y-axis 1203 represents the loaded CT image intensity values ​​in Hounsfield units. The x-axis 1202 represents the spatial location graded in pixels, with the first pixel on the intensity profile corresponding to location 1107 located in the lungs and the last pixel corresponding to location 1108 located in the image background. As can be seen in FIG. 11, this line profile spans several tissue types: lung, tumor, bone, fat, muscle, fat, and finally air (outside the body). These different regions can be observed on the line profile 1201; they are labeled by letters A-F and summarized in the legend 1210. The boundaries between the different tissues are indicated by vertical dashed lines labeled 1204-1209. FIG. 12b shows the intensity profile 1221 after application of a CT-lung window of (W,L)=(1000,-500) 307 for presentation purposes. The x-axis 1222 is the same as 1202 in FIG. 12a. The boundaries between the different tissues, here labeled 1224-1229, are also in the same position as in FIG. 12a. The y-axis 1223, graded in percentages, gives the corresponding shades of grey, as indicated by the color bar 1230 from black to white. The intersections of intensity profile 1221 with horizontal lines 1231 and 1232 indicate clamped pixels. In this example, the boundary 1225 between the tumor and bone cannot be seen on the displayed image because both the tumor and bone intensity values ​​are outside the displayed value range and are therefore clamped. Also, the boundaries 1227 and 1228 between the muscle tissue and the surrounding adipose tissue cannot be accurately delineated due to the clamping of the muscle pixels and due to the low image contrast at the boundary locations since both tissues are displayed in the highest shades of gray.In the lung window, the image contrast is adequate to recognize the correct border locations of 1224 and 1229.

[0054] FIG. 12c shows the intensity profile 1241 after application of a CT-bone window with (W,L)=(2500,480) 304. The x-axis 1242 is the same as 1202 in FIG. 12a. The boundaries between the different tissues, here labeled 1244-1249, are also in the same position as in FIG. 12a. The y-axis 1243, graded in percentages, gives the corresponding shades of grey, as indicated by the colour bar 1250 from black to white. The intersections of the intensity profile 1241 with the horizontal lines 1251 and 1252 indicate clamped pixels. In this example, only 1245 and 1246 with the bone tissue have acceptable contrast to be accurately depicted or reviewed. The bone window. The large window width W=2500 makes it impossible to distinguish the soft tissues (muscle, tumour, fat etc.) and therefore boundaries such as 1247 and 1248 cannot be accurately depicted or reviewed. Accurate differentiation between soft tissues requires displaying the image with smaller W values. This is illustrated in FIG. 12d, which shows the intensity profile 1261 after application of a CT-chest window of (W,L)=(350,40) 306. The x-axis 1262 is the same as 1202 in FIG. 12a. The boundaries between the different tissues, here labeled 1264-1269, are also in the same positions as in FIG. 12a. The y-axis 1263, graded in percentages, gives the corresponding shades of grey, as indicated by the colour bar 1270 from black to white. The intersections of the intensity profile 1261 with the horizontal lines 1271 and 1272 indicate clamped pixels. In this example, the lung tissue A and the image background F are clamped, which makes it difficult to review or draw the contours at the boundary locations 1264 or 1269. Also, the bone tissue C is clamped, which makes it difficult to recognise the correct positions of the bone boundaries 1265 and 1266.

[0055] According to an embodiment of the present invention, a method is provided for automatically detecting whether the current window width / level parameters used for displaying a medical image provide sufficient contrast for accurate depiction of structures on the medical image at the correct boundaries while contouring the structures on the medical image. The displayed image contrast is expected to be optimal while the user contours at a position where the image intensity is close to the window level value used to display the image. The image contrast gradually decreases while moving from the window center towards one of the two window edges where the clamped image intensity appears saturated on the screen.

[0056] This is illustrated by the diagram in FIG. 4. This diagram shows a schematic diagram 400, where image intensities at contour locations may be at the edge of the applied W, L parameters of a displayed image with unnormalized intensity values. The x-axis 414 represents real-world image values. The range shown (-1000 to 2000 HU) is typical of CT images. The y-axis 411, graded in percentages, gives the corresponding shades of grey, as shown by a colour bar 410 going from black to white. The image is displayed on a display device using the W, L parameters corresponding to an intensity transformation 404, with a window width W 409 and a window level L 408, which is rendered to the middle (50%) 401 of the shades of grey available for display. Image intensity values ​​greater than the window level by half the window width are clamped and displayed as white 403. Image intensities less than the window level by half the window width are clamped and displayed as black 402. Therefore, image contrast is assumed to be optimal only around 50% of the range of grey shades represented by 407. Image intensity values ​​below 412 are displayed in the darkest grey shade 405 and image intensity values ​​above 413 are displayed in the lightest grey shade 406. This clamping is not explicitly shown in this figure but can be interpreted from Figure 3. In both grey shade ranges 405 and 406, the contrast of the displayed image is deemed inadequate for contouring or reviewing contours.

[0057] FIG. 5 is equivalent to FIG. 4 when the image modality used for contouring has non-quantified intensity values ​​such as MRI, since image intensity values ​​are highly dependent on acquisition and reconstruction parameters. Window widths and window levels are typically defined in percentages of image intensity values. The x-axis 514 represents real-world image intensity values ​​ranging from minimum to maximum image intensity values, graded in percentages mapping minimum values ​​to 0% 515 and maximum values ​​to 100% 516. The percentage-graded y-axis 511 gives the corresponding shades of grey as indicated by a colour bar 510 going from black to white. The image is displayed using W,L corresponding to an intensity transformation 504 with a window width W 509 and a window level L 508, which is rendered to the middle (50%) 501 of the shades of grey available for display. Image intensity values ​​that are half the window width greater than the window level are clamped and displayed as white 503. Image intensities half the window width below the window level are clamped and displayed as black 502. Image contrast is therefore assumed to be optimal only around 50% of the range of grey shades represented by 507. Image intensity values ​​below 512 are displayed in the darkest grey shade 505 and image intensity values ​​above 513 are displayed in the lightest grey shade 506. This clamping is not explicitly shown in this figure but can be interpreted from FIG. 3. In both grey shade ranges 505 and 506 the contrast of the displayed image is deemed inadequate for contouring or reviewing contours.

[0058] The present invention is a system that allows feedback to a user while contouring or delineating structures on a medical image when the current window width and level used to display image slices is inappropriate. Preferably, the feedback is provided to the user automatically.

[0059] An embodiment of the present invention is illustrated by Fig. 6. The system loads at least one medical image (which may be 2D or 3D, or a time series of medical images) in step 601, which shows one or more structures to be contoured. Preferably, the images are CT or MR images. The system applies default W,L settings to display the images. This image loading process may have been initiated by the user or automatic, and the default W,L settings may be image modality dependent or workflow dependent, may persist from previous use of the system, or may be user configured, or may be loaded from stored image parameters. The image is then displayed with the default W,L parameters. It is anticipated that such a system has the usual viewing capabilities of a medical image visualization tool, and that the user may choose to change the view. At some point, the user may want to start a contouring process of one or more structures on the medical image. In one embodiment of the present invention, a user may select a manual, semi-automatic or automatic contouring tool in step 603 and adjust at least one of the W, L parameters in step 602 as necessary before commencing contouring of the selected structure in step 604. The system and method determines the local image intensity of at least one medical image for each location of the contour as the contour is generated. The method also determines, in step 605, whether the current W, L parameters are appropriate for contouring of each location of the structure on the medical image based on the local image intensity at the location of the newly drawn or edited contour as the contour is drawn, preferably this is a dynamic determination.

[0060] The system does not interfere with the contouring process while the current W, L parameters are suitable. If one or more of the W, L parameters are not suitable for contouring, the user is warned that at least one of the parameters W and L is not suitable for the most recently generated position on the contour. The system warns the user in step 606 as soon as a newly drawn or edited contour is in an image location where the contrast of at least one displayed medical image is locally, i.e. where at least one of the W, L parameters is not suitable for contouring. Preferably, the warning is a visual or audio warning. In one embodiment of the invention, the warning is provided by one or more of the following: a message in the user interface; a change in color of the contour being generated or edited; a change in style or thickness of the contour being generated or edited; a change in color of the area around the contour being generated or edited.

[0061] The user may adjust at least one of the W, L parameters before stopping the contouring and resuming contouring of the structure on the medical image (607). In one embodiment of the invention, in response to the alert, at least one of the window width parameter W and the window level parameter L is adjusted to be appropriate for contouring of the most recently generated position on the contour, and the medical image is displayed on the display device according to the adjusted W, L settings.

[0062] In one embodiment of the invention, the method may further comprise the step of determining whether the parameters W, L are suitable for the current task of contour generation, and if the detected parameters are not suitable for the current task, a warning is provided that the parameters are not suitable and at least one new parameter suitable for the current task is suggested. Preferably, the warning is provided to the user.

[0063] In one embodiment of the present invention, the adjustment of at least one of the window width parameter W and the window level parameter L to suit the current task of contour generation is performed by a user, or the adjustment of at least one of the window width parameter W and the window level parameter L to suit the current task is an automatic adjustment.

[0064] In one embodiment of the invention, adjusting at least one of the window width parameter W and the window level parameter L is performed by the user in response to one or more of the following: a pop-up window with appropriate notification text and accept / reject buttons appearing on the screen; a context-dependent menu accessed via the display. The menu for parameter adjustment can be accessed from an annotation displayed next to the portion of the contour having the new suggested window (W,L) settings. In one embodiment of the invention, the context-dependent menu is displayed as soon as the system detects a mouse over (also known as a mouse hover) over the portion of the contour having the new proposed window (W,L) settings. In an alternative embodiment of the invention, the one or more updated parameter settings are applied directly as soon as the system detects a mouse over over the portion of the contour having the new proposed window (W,L) settings, with the option for the user to accept these using an interaction mechanism (mouse click, keyboard hotkey or voice control). In a further embodiment of the invention, the system comprises an audio feedback and voice control system for applying / accepting / rejecting the proposed W,L parameter settings. In yet a further example of the present invention, another pane of the system's user interface is used to list all new suggestions along with options to navigate and apply / accept / reject the proposed settings.

[0065] In an embodiment of the invention, the user may repeat steps 604-607 to complete contouring of the structure in step 608. In an embodiment of the invention, the process of contouring the structure (steps 602-608) may be repeated to contour multiple structures on the medical image, resulting in multiple contours being created on the medical image. The system then saves, stores or outputs (609) the representations / contours of the structures created by the user for further use in the clinical workflow.

[0066] A second embodiment of the present invention is illustrated by FIG. 7. This embodiment provides a system and method for reviewing existing contours on previously contoured medical images. The system loads at least one medical image (which may be 2D or 3D, or a time series of medical images) with an associated set of existing contours for review in step 701. Preferably, the images are CT or MRI images. The system applies default or pre-configured W,L settings or W,L parameters stored in the image file to display the image with the associated contours. It is anticipated that such a system has the usual viewing capabilities of a medical image visualization tool, and that the user may choose to change the view. This image loading process may have been user initiated or automatic, and the default W,L settings may be image modality or workflow dependent, may persist from a previous use of the system, or may be user configured, or may be loaded from stored image parameters. The image is then displayed with the default W,L parameters.

[0067] Optionally, the user may first select one or more structures for contour review 702 and then adjust at least one of the W, L parameters in step 703, if necessary, before starting the contour review process in step 704. The system and method determine, for the contour to be reviewed, the local image intensity of at least one medical image for each location of the contour. The system detects (705) which parts of the displayed contour can be properly reviewed and which parts of the contour cannot be properly reviewed based on the current W, L settings. Preferably, this is detected automatically. The system does not interfere with the contour review process while the current W, L parameters are suitable. When necessary, the system warns the user that the displayed image contrast is not suitable for reviewing the displayed contour (706). If it is determined that one or more of the W, L parameters are not suitable for reviewing at least one part of the existing contour according to the determined image intensity, the user is warned that at least one of the parameters W and L is not suitable for reviewing at least one part of the contour. In one embodiment of the invention, at least one of the window width parameter W and the window level parameter L is then adjusted so as to be suitable for reviewing at least one portion of the contour, and a further section of the contour is reviewed using the adjusted parameters.

[0068] In one embodiment of the invention, the method may further comprise the step of determining whether the parameters W, L are suitable for the current task of contour review, and if the detected parameters are not suitable for the current task, a warning is provided that the parameters are not suitable and at least one new parameter suitable for the current task is suggested. Preferably, the warning is provided to the user.

[0069] In one embodiment of the present invention, adjusting at least one of the window width parameter W and the window level parameter L to be suitable for reviewing at least one portion of the contour is performed by a user, or adjusting at least one of the window width parameter W and the window level parameter L to be suitable for reviewing at least one portion of the contour is an automatic adjustment.

[0070] Preferably, the alert is a visual or audio alert. In one embodiment of the invention, the alert is provided by one or more of the following: a message in a user interface; a change in color of the contour being created or edited; a change in style or thickness of the contour being created or edited; a change in color of the area surrounding the contour being created or edited.

[0071] In one embodiment of the invention, the adjustment of at least one of the window width parameter W and the window level parameter L is made by the user in response to one or more of the following: a pop-up window with appropriate notification text and accept / reject buttons appearing on the screen; a context-sensitive menu accessed via the display. The menu can be accessed from an annotation displayed next to the portion of the outline having the new suggested window (W,L) settings. In one embodiment of the invention, the context-sensitive menu is displayed as soon as the system detects a mouse over (also known as a mouse hover) on the portion of the outline having the new proposed window W,L settings. In an alternative embodiment of the invention, the settings are applied directly as soon as the system detects a mouse over on the portion of the outline having the new proposed window (W,L) settings, with an option for the user to accept these using an interaction mechanism (mouse click, keyboard hotkey or voice control). In another embodiment of the invention, the system comprises an audio feedback and voice control system for applying / accepting / rejecting the proposed settings. In yet another embodiment of the invention, a separate pane of the system's user interface is used to list all the new suggestions, with options to navigate and apply / accept / reject the proposed settings.

[0072] The user may adjust at least one of the W, L parameters before stopping the review and restarting the contour review process (707). The user completes the contour review in step 708. The contour review process (steps 702-708) may be repeated for multiple structures on the medical image, resulting in multiple contours being reviewed on at least one medical image. The system then saves, stores, or outputs the contours reviewed by the user for further use in the clinical workflow (709).

[0073] During the contour review process (steps 702-708), the user may need to adjust one or more of the displayed contours. Preferably, this contour adjustment is done manually, but may also be done semi-automatically or automatically. The user can then switch to the contouring process described in FIG. 6. Such a transition may be triggered as soon as the user selects the contouring tool. Preferably, the transition is triggered automatically. The contour review process can also be resumed after the user has exited the contouring mode, for example by deselecting the contouring tool. Preferably, the contour review is resumed automatically. In one embodiment of the present invention, the adjustment of at least one of the window width parameter W and the window level parameter L to be suitable for reviewing at least one portion of the contour is performed by the user. Alternatively, the adjustment of at least one of the window width parameter W and the window level parameter L to be suitable for reviewing at least one portion of the contour is an automatic adjustment.

[0074] In these embodiments of the invention for contour generation and contour review, a minimum value of at least one of the W and L parameters is set for contour generation, editing or contour review, or a maximum value of at least one of the W and L parameters is set for contour generation, editing or contour review.

[0075] A further example of the present invention is illustrated by Fig. 8. A key aspect of this embodiment is that the system and method can suggest at least one suitable window W, L parameter to the user while performing a contouring, editing or review task. Preferably, this is an automatic suggestion. The system loads at least one medical image (which may be 2D or 3D, or a time series of medical images) with or without an associated existing contour set in step 801. Preferably, the image is a CT or MRI image. The user then starts one of the tasks of contouring, editing the loaded contour or reviewing the loaded contour in step 802. Preferably, the contouring is manual contouring, semi-automatic contouring or automatic contouring. The system checks (803) whether the current window W, L used to display the image and the associated contour on the screen is suitable for the selected task of contouring, editing or review and warns the user if the displayed image contrast is not suitable (804). In one embodiment of the present invention, the warning is a visual or audio warning. In a following step 805, the system suggests at least one new window W, L setting to the user that is appropriate. Preferably, when a contour is generated or selected for review, a patch around either the most recently generated portion of the contour or the portion of the contour for review is determined and the local image intensity within the patch is reviewed to determine whether at least one of the W, L parameters is appropriate or needs to be adjusted.

[0076] In one embodiment of the invention, the method may further comprise the step of determining whether the parameters W, L are suitable for the current task of contour creation, editing or contour review, and if the detected parameters are not suitable for the current task, a warning is provided that the parameters are not suitable and at least one new parameter suitable for the current task is suggested. Preferably, the warning is provided to the user.

[0077] When the task is contour creation or editing of an existing contour, it is expected that there will be no conflicts regarding the determination of the new window parameter settings (as this is based on the local image intensity at a single location of the newly drawn or edited contour, see step 605). Therefore, the system can be optionally configured to automatically apply at least one new parameter setting 805. In a contour review task, conflicts may arise because the current window width and level may not be appropriate at different parts of the displayed contour. The system then alerts the user that multiple settings for the W and L parameters are proposed, allowing the user to select the appropriate settings. Preferably, the alert is a visual or audio alert. In an embodiment of the invention, the alert is provided by one or more of the following: a message in the user interface, a change in color of the contour being created or edited, a change in style or thickness of the contour being created or edited, a change in color of the area around the contour being created or edited. Optionally, in such a scenario, one or more of the settings can also be applied automatically. The portion of interest on the contour currently being contoured, edited or reviewed can be determined automatically, for example, by using eye-tracking hardware and software connected to the system to identify the location of the user's gaze, or can be indicated by the location of the user's mouse on the screen. Thus, even when there is a conflict between several contour locations with different suitable W,L settings, the system can still automatically apply the suggested suitable settings by identifying the contour of interest on the patient image, as indicated by the user's mouse or eye-tracking system. Optionally, the user can manually adjust one or more of the W,L parameters and then resume the task (806).

[0078] In one embodiment of the present invention, adjusting at least one of the window width parameter W and the window level parameter L to make at least one portion of the contour suitable for contouring, editing or reviewing is performed by a user, or adjusting at least one of the window width parameter W and the window level parameter L to make at least one portion of the contour suitable for contouring, editing or reviewing is an automatic adjustment.

[0079] The user completes the contouring / editing / review process in step 807. Steps 802-807 may be repeated for multiple structures. The system then saves, stores or outputs (808) the contour for further use in the clinical workflow.

[0080] An embodiment of the present invention may also automatically detect when the window W, L parameters are not appropriate based on local image intensity. Figures 9 and 10 (similar to Figure 4 but with new details numbered 1015, 1016 and 1017) are used to detail the underlying method. Figure 9a shows a 2D image slice 911 during a manual or semi-automatic contouring operation. The user starts contouring a structure on the image at 901 and proceeds around the structure as indicated by 905. As shown, the contour is generated counterclockwise, but may be generated in other directions. The already drawn part of the structure's contour is indicated by the continuous curve 902. The missing part (not yet drawn by the user) is indicated by the dashed curve 904. The location of the newly drawn part of the contour is indicated by 903. The shaded portion 913 of the figure is the local image patch that is preferably automatically and dynamically determined while the user is contouring. The image intensity over that local patch is then analyzed, preferably automatically, to determine whether the local image contrast is sufficient or adequate based on the current W, L parameter settings.

[0081] FIG. 9b is equivalent to FIG. 9a, but for a contour review task instead of contour generation. It shows a 2D image slice 912 with a loaded or automatically generated contour 910 to be reviewed. The system then automatically detects that two portions 907 and 916 of the contour cannot be properly reviewed because at least one of the current W, L settings is not appropriate for these sections of the contour. The portion 907 bounded by dashed line segments 908 and 909 shows an example where the image contrast at the contour location 907 is inadequate because the current W, L parameters display the image intensity in the lightest shade of gray (as indicated by 406 in FIG. 4 and by 506 in FIG. 5). The portion 916 bounded by dashed line segments 914 and 915 shows an example where the image contrast at the contour location 916 is inadequate because the current W, L parameters display the image intensity in the darkest shade of gray (as indicated by 405 in FIG. 4 and by 505 in FIG. 5). 9b illustrates, by way of example, that the system may detect that there are at least one or more portions of at least one or more contours that cannot be properly reviewed because at least one of the current W, L settings is not appropriate. This detection relies on an automatic analysis of local image intensity on a narrow band 906 around the loaded contour. The narrow band is split into multiple patches (whether overlapping or not), and the patches are then analyzed, similar to the manual contouring process shown in 900a.

[0082] Thus, in both cases of contouring or review, the detection of the W,L settings relies on the analysis of the local image intensity of the image patch around the contour. As an illustrative example, assuming the image is a CT image, the W,L parameters are as shown in FIG. 10 by an intensity transformation 1004 with a window width W 1009 and a window level L 1008, which is rendered to the middle (50%) 1001 of the shades of grey available for display. The x-axis 1014 represents real-world image intensity, and the range shown (-1000 to 2000 HU) is typical of CT imaging. The percentage-graded y-axis 1011 gives the corresponding shades of grey, as indicated by the black to white colour bar 1010. Also assume that the minimum of the intensity values ​​of the local patch is greater than L+W / 2. Thus, all pixels within the local patch are clamped and therefore saturated and displayed as white 1003. This clamping is not explicitly shown in this figure, but can be interpreted from FIG. 3. The user will then draw (or review) a contour drawn on a pure white background, which is not even suitable for determining the boundaries of biological structures, present in the image data captured by the imaging sensor, but rendered incorrectly on the display device. The system therefore warns the user about this problem. Preferably, the warning is a visual or audio warning. If a pixel is clamped and displayed as white, the solution is to adjust this by shifting the window level, in order to improve the local contrast of such a patch. Preferably, the parameter is shifted to a higher value. This is illustrated in FIG. 10 by the intensity transformation 1015. In this example, the minimum image intensity value of the local patch is compared to L+W / 2. If the local patch is displayed in the whitest shade of gray 1006, the local contrast will still be very low, since only 20% of the dynamic range of the shades of gray is used. This is the case, for example, if the minimum image intensity value of the local patch is greater than the value illustrated by 1013. A similar analysis can be performed for the darkest shade of gray 1005.For example, when the maximum image intensity value of a local patch is less than the value indicated by 1012, pixels in the local patch having image intensity values ​​lower than LW / 2 are clamped and displayed as black 1002. In this scenario, a suitable suggestion for a new window setting is to shift the window level to a lower intensity value.

[0083] A third scenario is that for example a large portion of the pixels of the local patch (e.g. 45%) have intensity values ​​greater than the value (1013) and are therefore displayed in the whitest shade of grey 1016. Also, a large portion of the pixels of the local patch (e.g. 45%) have intensity values ​​less than the value 1012 and are therefore displayed in the darkest shade of grey 1005. In this scenario, only 10% of the local pixels are displayed in shades of grey within the optimal recommended range represented by 1007. The local patch appears as if it has been binarized in black and white. Therefore, a good suggestion for a proper window setting is to keep the window level 1008 unchanged and increase the window width from 1009 to 1016 to obtain the intensity transformation 1017. The described scenario can occur if the window width is set too small for the range of available image intensities in the local patch. A fourth plausible scenario is the opposite of the third scenario, where the window width is set so large that all pixel intensities are rendered in approximately the same shade of grey. Again, the user will be drawing (or reviewing) contours drawn on a flat grey background, which is not adequate to determine the boundaries of biological structures present in the image data, captured by the imaging sensor but rendered inaccurately on the display device. This is typical, for example, when drawing interfaces between soft tissues with the bone window 204 (as shown by the example in FIG. 12c). Therefore, a good suggestion for proper window settings is to certainly reduce the window width. The window level may also need adjustment, for example to move it around the central tendency estimate of the image intensity in the local patch.

[0084] For all of the various embodiments and implementations of the present invention, the detection that the current window W, L settings are inappropriate may be performed according to measurements from the local image intensity of the image patch around the contour. These may include at least one of the following: ○ The average intensity value of the image patch ○ The maximum intensity value of the image patch ○ The minimum intensity value of the image patch ○ The central intensity value of the image patch ○ The configurable percentile of the intensity of the image patch ○ The statistical measurement between the distribution of the image patch intensity values and a predetermined parametric distribution function of the W, L values

[0085] More preferably, the measured values from the local image intensity are calculated before or after the application of the intensity transformation.

[0086] More preferably, the measured values from the local image intensity are compared with a predetermined or user-configurable threshold function of the current W, L parameters. There are many different examples of possible thresholds, for example, the minimum intensity value of the image patch > L (only the upper half of the gray scale is used), the maximum intensity value of the image patch < L, the central intensity value of the image patch > L + W / 2, and half of the local patch is saturated. The same is true for the median intensity value of the image patch < L + W / 2.

[0087] More preferably, the detection that at least one of the current window W, L settings is inappropriate is performed using a machine learning algorithm. The machine learning algorithm receives the local image patch and the current window W, L parameters. The information about the local patch may include intensity measurements and spatial information about the image intensity within the patch. In one embodiment of the present invention, the machine learning algorithm uses information about the local image intensity.

[0088] More preferably, when performing manual contour formation, editing or contour review, a minimum allowable W value can be set.

[0089] More preferably, a maximum allowable W value can be set when performing manual contouring, editing or contour review.

[0090] To warn the user that the window W, L settings are inappropriate (step 606 in FIG. 6 , step 706 in FIG. 7 , step 804 in FIG. 8 ), various different approaches may be implemented. In one embodiment of the invention, a warning message or symbol is displayed on the system user interface that a portion of the contour is being drawn, drawn or displayed with inappropriate window W, L settings. In a further embodiment, the color of the contour is changed where the contour is being drawn, drawn or displayed with inappropriate window W, L settings. In a further embodiment, the curve style or thickness of the contour is changed where the contour is being drawn, drawn or displayed with inappropriate window W, L settings. In a further embodiment, the color of the area around the contour is changed where the contour is being drawn, drawn or displayed with inappropriate window W, L settings. In a further embodiment, a symbolic annotation is superimposed on the contour that is being drawn, drawn or displayed with inappropriate window W, L settings. In a further embodiment, audio feedback is provided to the user that inappropriate window W, L settings are being used. In a further embodiment, the system warns the user when drawing by preventing the contour from being drawn. The drawing cursor may be changed to indicate that drawing cannot or should not be performed with the current window W,L settings. When reviewing a contour, the system may warn the user of improper window W,L settings by not displaying or changing the curve style or thickness of the curve in the portion of the contour where the window W,L settings are improper. One or more of these approaches may be implemented alone or in combination.

[0091] Various approaches may be implemented to alert the user that one or more window settings have been proposed for the user to select. In an embodiment of the invention, the alert is a visual or audio alert. In one embodiment, a pop-up window with appropriate notification text and accept / reject buttons appears on the screen. In a further embodiment, a context-sensitive menu can be accessed on an annotation shown next to the outline portion with the new proposed window (W, L) settings. In a further embodiment, the context-sensitive menu is displayed as soon as the system detects a mouse over (also known as a mouse hover) of the outline portion with the new proposed window W, L settings. In an alternative embodiment, the settings are applied directly as soon as the system detects a mouse over on the outline portion with the new proposed window (W, L) settings, with an option for the user to accept these using an interaction mechanism (mouse click, keyboard hotkey or voice control). In a further embodiment, the system comprises an audio feedback and voice control system for applying / accepting / rejecting the proposed settings. In an alternative embodiment, a separate pane of the system's user interface is used to list all the new suggestions, with options to navigate through the proposed settings and apply / accept / reject them. One or more of these approaches may be implemented alone or in combination.

[0092] Embodiments of the present invention may be applied to any or all of the following: picture archiving and communication systems (PACS); advanced visualization workstations; image acquisition workstations; web-based or cloud-based medical information and imaging systems; radiation treatment planning systems (TPS); radiation therapy linear accelerator consoles; and radiation therapy proton beam consoles.

[0093] The present invention has been described with reference to the accompanying drawings. However, it will be appreciated that the present invention is not limited to the specific examples described herein and illustrated in the accompanying drawings. Moreover, since the illustrated embodiments of the present invention can be implemented in large part using electronic components and circuits known to those skilled in the art, details will not be described beyond the extent deemed necessary as illustrated above for the understanding and appreciation of the underlying concepts of the present invention and so as not to obscure or confuse the teachings of the present invention.

[0094] The present invention may be implemented in a computer program for running on a computer system, comprising at least code portions for performing the steps of a method according to the present invention when executed on a programmable apparatus such as a computer system, or for enabling a programmable apparatus to perform the functions of a device or system according to the present invention.

[0095] A computer program is a listing of instructions, such as a particular application program and / or an operating system. A computer program may include, for example, one or more of a subroutine, a function, a procedure, an object method, an object implementation, an executable application, an applet, a servlet, source code, object code, a shared library / dynamic load library, and / or other sequence of instructions designed to execute on a computer system. Thus, some examples describe a non-transitory computer program product having stored executable program code for automatic contouring of cone beam CT images.

[0096] The computer program may be stored internally in a tangible non-transitory computer readable storage medium or transmitted to the computer system via a computer readable transmission medium. All or a portion of the computer program may be provided on a computer readable medium that is permanently, removably or remotely coupled to an information processing system. The tangible non-transitory computer readable medium may include, by way of example and not limitation, any number of the following: magnetic storage media including disk and tape storage media; optical storage media such as compact disk media (e.g., CD ROM, CD-R, etc.) and digital video disk storage media; non-volatile memory storage media including semiconductor-based memory units such as FLASH memory, EEPROM, EPROM, ROM; ferromagnetic digital memory; MRAM; volatile storage media including registers, buffers or caches, main memory, RAM, etc.

[0097] A computer process typically includes a running (executing) program or part of a program, current program values ​​and state information, and resources used by an operating system to manage the execution of the process. An operating system (OS) is software that manages the sharing of a computer's resources and provides programmers with the interface used to access those resources. An operating system processes system data and user input, and responds by allocating and managing tasks and internal system resources as services to users and programs of the system.

[0098] A computer system may include, for example, at least one processing unit, associated memory, and a number of input / output (I / O) devices. When executing a computer program, the computer system processes information in accordance with the computer program and generates resultant output information via the I / O devices.

[0099] The present invention has been described herein above with reference to specific examples of embodiments thereof, however it will be apparent that various modifications and changes may be made thereto without departing from the scope of the present invention as set forth in the appended claims, which are not limited to the specific examples described above.

[0100] Those skilled in the art will recognize that the boundaries between logic blocks are merely exemplary, and that alternative embodiments may merge logic blocks or circuit elements, or impose alternative decompositions of functionality on the various logic blocks or circuit elements. Thus, it should be understood that the architectures depicted herein are merely exemplary, and that in fact many other architectures may be implemented which achieve the same functionality.

[0101] Any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Thus, any two components combined herein to achieve a particular functionality can be considered to be "associated" with one another such that the desired functionality is achieved, regardless of architecture or intermediate components. Similarly, any two components so associated can also be considered to be "operably connected" or "operably coupled" with one another to achieve the desired functionality.

[0102] Moreover, those skilled in the art will recognize that the boundaries between operations described above are merely exemplary. Multiple operations may be combined into a single operation, a single operation may be distributed into additional operations, and operations may be performed with at least partial overlap in time. Furthermore, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments.

[0103] However, other modifications, variations, and alternatives are possible, and the specification and drawings are accordingly to be regarded in an illustrative rather than a restrictive sense.

[0104] In the claims, any reference signs placed in parentheses shall not be construed as limiting the claims. The term "comprises" does not exclude the presence of other elements or steps than those recited in the claim. Furthermore, the terms "a" or "an" as used herein are defined as one or more. Also, the use of introductory phrases such as "at least one" and "one or more" in the claims should not be construed as meaning that the introduction of an element of another claim by the indefinite article "a" or "an" limits a particular claim containing such an introduced claim element to an invention containing only one such element, even if the same claim contains the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an". The same applies to the use of definite articles. Unless otherwise stated, terms such as "first" and "second" are used to arbitrarily distinguish between the elements they describe. Thus, these terms are not necessarily intended to indicate a temporal or other priority of such elements. The mere fact that certain measures are recited in mutually different claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. 1. A method of contouring a medical image displayed on a display device, comprising: providing at least one medical image having one or more structures to be contoured; providing a window width parameter W and a window level parameter L for the at least one medical image; displaying the medical image according to the W, L parameters; performing the following contouring steps: determining a local image intensity of the at least one medical image for each location of a contour generated for a structure on the image; When the contour is generated, determining whether the window width parameter W and the window level parameter L are suitable for contouring each position of the contour on the medical image according to the determined local image intensity; if at least one of the parameters is not suitable for contour generation, providing a warning that at least one of the parameters W and L is not suitable for the most recently generated position on the contour; adjusting at least one of the window width parameter W and the window level parameter L to be appropriate for contouring the structure at the most recently generated location on the contour, and displaying the medical image according to the adjusted W, L settings; performing a contouring step, including: A method comprising:

2. 1. A method for reviewing a previously contoured medical image displayed on a display device, comprising: providing at least one medical image and at least one contour on the medical image to be reviewed; providing a window width parameter W and a window level parameter L for the at least one medical image; displaying the medical image according to W, L parameters to display the at least one contour; selecting at least one contour of the previously contoured image for review; performing a contour review step of: determining a local image intensity of the at least one medical image for at least one portion of the selected contour to be reviewed; determining, according to the determined local image intensity, that at least one of a current window width parameter and a window level parameter L is not suitable for reviewing the at least one portion of the at least one contour; providing a warning that at least one of the parameters W and L is not suitable for reviewing the at least one portion of the contour, adjusting at least one of the window width parameter W and the window level parameter L so that it is suitable for reviewing the at least one portion of the contour, reviewing a further section of the contour using the adjusted parameters, and displaying the medical image according to the adjusted W, L settings; performing a contour review step, including: A method comprising:

3. the contouring step or the contour review step is repeated until the one or more structures to be contoured have been contoured or all of the at least one selected contour on the previously contoured image have been reviewed.

3. The method according to claim 1 or 2.

4. adjusting at least one of the window width parameter W and the window level parameter L to be suitable for contouring at least one structure or for reviewing the at least one portion of the contour is performed by a user; 3. The method according to claim 1 or 2.

5. adjusting at least one of the window width parameter W and the window level parameter L to be suitable for contouring at least one structure or for reviewing the at least one portion of the contour is automatic adjustment; 3. The method according to claim 1 or 2.

6. the contour review step further comprising the steps of: using current window width and window level parameters to find one or more portions of the selected contour suitable for review; and reviewing the found one or more contour portions. The method of claim 2.

7. and further comprising editing or correcting one or more of the detected contour portions after the contour portions have been reviewed. The method of claim 2.

8. determining whether the parameters W and L are appropriate for the current task of contour generation or contour review; if one or more of the detected parameters are not suitable for the current task, providing a warning that one or more of the parameters are not suitable and suggesting at least one new parameter that is suitable for the current task; The method of claim 1 or 2, further comprising:

9. the parameter suggestions for the one or more parameters are automatic; The method of claim 8.

10. the parameter suggestions for the one or more parameters are applied automatically; The method of claim 8.

11. The determination of the appropriateness of at least one of the window width parameter W and the window level parameter L is performed using a machine learning algorithm.

3. The method according to claim 1 or 2.

12. the machine learning algorithm uses information from at least one of the local image intensity and spatial information of the image. The method of claim 11.

13. when a contour is generated or selected for review, determining a patch around either the most recently generated portion of the contour or the portion of the contour for review, and evaluating the local image intensity within the patch to determine whether at least one of the W, L parameters is suitable or needs to be adjusted; The method of claim 1 or 2, further comprising:

14. The local image intensity within the patch used to determine the appropriateness of parameters is mean intensity value, maximum intensity value, minimum intensity value, Median intensity value, configurable percentiles of intensity, a statistical measure between the distribution of image patch intensity values ​​and a predetermined parametric function of W, L values; The method of claim 13, wherein the value is calculated from one or more of:

15. the local image intensity is compared to a predetermined or user-configurable threshold function for at least one of the current W, L parameters; 3. The method according to claim 1 or 2.

16. Adjusting at least one of the window width parameter W and the window level parameter L may include: A popup window with suggestions for new W, L settings, Context-sensitive menus accessed through the display, hovering the mouse over the image or the outline area; Interaction mechanisms such as mouse clicks, keyboard hotkeys or voice control, by a user in response to one or more of:

3. The method according to claim 1 or 2.

17. The alert provided to the user is a visual or audio alert.

3. The method according to claim 1 or 2.

18. The warning may be: Messages in the user interface, Changing the color of the contour being created or edited; Changing the style or thickness of the contour being created or edited; Changing the color of the area around the contour being created or edited; 18. The method of claim 17, wherein the method is provided by one or more of:

19. the medical image is a CT scan, an MRI scan, or a PET scan; 3. The method according to claim 1 or 2.

20. When the contour is created or reviewed, the contour is saved or provided as an output.

3. The method according to claim 1 or 2.

21. After the contour editing or creation is complete, the steps are repeated so that multiple contours are created or reviewed on the medical image.

3. The method according to claim 1 or 2.

22. At least one of the window width parameter W and the window level parameter L at the start of a contouring or review process is: Imaging modality dependent, Workflow dependent, Settings from previous use of the system, User configured or defined from parameters of stored images; 3. The method according to claim 1 or 2, wherein the method is one of:

23. The medical image is a 2D medical image, a 3D medical image, or a time series of medical images.

3. The method according to claim 1 or 2.

24. A minimum value of at least one of the W and L parameters is set for contouring, editing, or contour review.

3. The method according to claim 1 or 2.

25. A maximum value of at least one of the W and L parameters is set for contouring, editing, or contour review; 3. The method according to claim 1 or 2.

26. The contouring is one of manual contouring, semi-automatic contouring, or automatic contouring.

3. The method according to claim 1 or 2.

27. 1. A system for analyzing medical images, comprising: a display for displaying at least one medical image to be contoured; a processor for setting a window width parameter W and a window level parameter L for a display device to correct values ​​for the start position of a contour of a structure on said at least one medical image to be contoured; the processor determines a local image intensity of the at least one medical image for each location of the contour as the contour is generated; When the contour is generated, the processor determines, for each position of the contour on the medical image according to the determined local image intensity, whether the window width parameter W and the window level parameter L are correct; if at least one of the parameters is incorrect, warns a user that at least one of the parameters W and L is incorrect for the current position on the contour, and adjusts at least one of the window width parameter W and the window level parameter L so that it is correct for the current position on the contour. system.

28. 1. A system that allows a user to review previously contoured medical images, comprising: a display for displaying at least one previously contoured medical image to be reviewed using default window width and window level parameters; a processor for selecting at least one contour on the previously contoured image for review; wherein the processor: using the default window width and window level parameters to find one or more portions of the selected contour suitable for review, thereby allowing a user to review the found portions of the one or more contours; detecting that at least one of the default window width parameter W and window level parameter L is not suitable for further sections of the contour to be reviewed; alerting the user that at least one of the parameters W and L is not appropriate for the further section of the contour, adjusting at least one of the window width parameter W and the window level parameter L so that it is correct for the further section of the contour, and allowing the user to review the further section of the contour using the adjusted parameters; The system is configured as follows:

29. the display displays the medical image according to the adjusted W, L settings.

29. A system according to claim 27 or 28.

30. The processor: configured to automatically detect at least one of a current window width parameter W and a window length parameter L, and determine whether the detected parameters are suitable for a current task of contour generation or contour review; 29. A system according to claim 27 or 28.

31. A computer program comprising a plurality of instructions which, when executed by a computer, causes the computer to perform a method according to claim 1 or 2.

Images