Processing energy spectrum image data generated by computed tomography scanner

Abstract

A method and system for generating predictive indicators of possible pathologies. The spectral image data is processed to identify regions of interest representing different sets of one or more organs. Each region of interest is then processed using a respective set of one or more machine learning algorithms to generate a respective number of prediction indicators for each region of interest. At least one of the prediction indicators is then output.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging, and in particular to the field of processing projection data generated by a computed tomography scanner. Background Art

[0002] A computed tomography (CT) scanner is a well-established medical imaging device that uses a detector arrangement formed by an array of detectors or pixels to detect the interaction between X-ray radiation energy and radiative material in order to generate medical image data.

[0003] In the field of computed tomography imaging, the use of non-traditional imaging techniques, such as spectral computed tomography, is becoming increasingly common. A CT scanner employing spectral computed tomography is capable of generating images representing particles of a particular type and / or frequency incident on a detector of the CT scanner, e.g., depending on the modality of the CT scanner. The image data generated using this technique is often referred to as spectral image data, spectral-based image data, material image data, or material-based image data.

[0004] More specifically, spectral CT is a CT technique based on distinguishing at least two different X-ray photon energies or spectra. This allows for material separation based on different attenuation profiles of materials at different energy values. In this way, a CT scanner is able to produce different types of spectral image data representing different energy levels or effects. Examples of spectral image types include photoelectric effect images and Compton scattering images. Other examples include iodine image data, which indicates the presence of iodine; water image data, which indicates the presence of water; contrast agent image data, which indicates the presence of contrast agent; effective atomic number image data, which indicates the effective atomic number of an element in the image data, and so on.

[0005] For example, a CT scanner can generate photoelectric effect image data, which is image data generated in response to only particles that reach the detector due to photoelectric effect interactions. As another example, a spectral CT scanner can generate Compton scatter image data, which is image data generated in response to only particles that reach the detector due to Compton scatter effect interactions.

[0006] There is a continuing desire to improve the diagnostic usefulness of spectral image data, and in particular to assist clinicians, other users, or further processing systems in analyzing spectral image data. Summary of the invention

[0007] Various embodiments of the invention are defined by the claims.

[0008] According to an example of an embodiment according to the present invention, a computer-implemented method of processing spectral image data of an object generated by a computed tomography scanner is provided.

[0009] The computer-implemented method includes: processing the spectral image data to identify at least two regions of interest, each region of interest representing a different set of one or more organs; processing each region of interest using a corresponding one of a plurality of sets of one or more machine learning algorithms, each machine learning algorithm configured to process the region of interest to generate a prediction indicator indicating a likelihood that the region of interest contains at least one representation of a predetermined pathology; and outputting at least one prediction indicator generated by the processing of each region of interest.

[0010] One embodiment of the present invention proposes a method for assisting the automatic analysis of spectral image data. In particular, it is proposed to analyze the spectral image data of an object using a region-specific machine learning process. The spectral image data is processed to identify at least two different regions of interest. Each region is then processed using one or more region-specific machine learning algorithms to generate a number of prediction indicators. Each prediction indicator represents or indicates the likelihood that the region of interest contains a predetermined pathology, such as a specific disease, lesion, growth and / or abnormality.

[0011] Various forms of predictive indicators are contemplated, including location indicators identifying the location of any possible pathology, binary indicators identifying a binary probability that the region of interest contains the pathology, and / or numerical indicators indicating a numerical probability that the region of interest contains the pathology.

[0012] Embodiments are based on the recognition that spectral image data contains a large amount of information that is difficult for clinicians to accurately and quickly evaluate, particularly in high-pressure and time-sensitive environments, such as emergency departments in hospitals. By providing pointers or indicators to specific areas or regions of spectral image data that contain possible pathology, a significant reduction in wasted time for identifying areas of interest can be achieved.

[0013] It should be understood that the predictive indicator may also be passed to another processing system, for example, to perform a more complete or further automated analysis of the subject. Thus, it is not necessary that the predictive indicator be provided directly to the clinician, for example, via a user interface.

[0014] In some embodiments, each machine learning algorithm is further configured to generate a confidence score for the predictive indicator generated by the machine learning algorithm, the confidence score representing a confidence in the likelihood that the region of interest contains the at least one indication of the predetermined pathology.

[0015] The computer-implemented method may also include: in response to each confidence score generated by the set of one or more machine learning algorithms, selecting at least one of the prediction indicators generated by processing each region of interest; and outputting at least one prediction indicator, wherein it includes outputting the selected at least one prediction indicator.

[0016] According to one embodiment, the generated predictive indicators are effectively filtered based on their confidence score(s) to improve the relevance of the predictive indicators output by the method, thereby reducing false positives.

[0017] In some examples, outputting the at least one prediction indicator comprises controlling a user interface to provide a visual representation in response to the at least one prediction indicator. In this way, information about the prediction indicator can be provided to a user such as a clinician. The information can be used by the clinician as part of a clinical decision-making process, such that the method provides a tool for assisting clinicians in making clinical decisions.

[0018] Optionally, each machine learning algorithm is further configured to generate a confidence score for the predictive indicator generated by the machine learning algorithm, the confidence score representing a confidence in the likelihood that the region of interest contains the at least one indication of the predetermined pathology.

[0019] Controlling the user interface may also be responsive to each confidence score generated by a plurality of sets of one or more machine learning algorithms. The use of confidence scores facilitates improved control and selection of the most appropriate or relevant predictive indicators to provide to an individual, e.g., to avoid or minimize raising concerns about inaccurate results.

[0020] In some examples, controlling the user interface includes providing greater visual emphasis to any of the at least one prediction indicator having a higher confidence score than to any of the at least one prediction indicator having a lower confidence score. The method recognizes that the most confident prediction indicator is most likely to be associated with an actual or true situation fact, thereby reliably improving the guidance provided to the user in identifying the most appropriate region.

[0021] Controlling the user interface may include sorting the at least one prediction indicator based on their confidence metrics to determine an order of the at least one prediction indicator; and controlling the user interface to provide a visual representation of the determined order of the at least one prediction indicator.

[0022] At least one of the machine learning algorithms may include a location identification machine learning algorithm. The location identification machine learning algorithm is configured to process a region of interest to identify within the region of interest locations of any representation of a predetermined pathology that is predicted to be present with a likelihood greater than a predetermined likelihood. A prediction indicator generated by the location identification machine learning algorithm indicates any identified locations within the region of interest.

[0023] Identifying the location of possible pathology further improves the assistance of the clinician or additional processing system(s) by further narrowing the possible area for pathology to guide the clinician in quickly identifying areas for investigation.

[0024] In some embodiments, outputting each prediction indicator includes controlling a user interface to provide a visual representation in response to the at least one prediction indicator; and the visual representation includes: for each location identified by any location identification machine learning algorithm that provides the prediction indicator in the at least one prediction indicator, a visual representation in response to the identified location.

[0025] In some embodiments, controlling the user interface comprises, for each location identified by any location identification machine learning algorithm that provides the at least one predictive indicator, superimposing a visual representation of the identified location over the visual representation of the spectral image data.

[0026] In some embodiments, controlling the user interface includes: for each location identified by any location identification machine learning algorithm that provides the at least one predictive indicator, providing a visual representation of a portion of the image data near the identified location.

[0027] The method provides valuable and clinically relevant information to the user at the user interface for evaluating the most appropriate image data. This significantly reduces the user's workload for analyzing the spectral image data, saving valuable clinical time.

[0028] In some embodiments, each predetermined pathology comprises a disease, lesion, growth, or abnormality in a corresponding region of interest.

[0029] Processing the spectral image data to identify at least two regions of interest may include performing an image segmentation process on the spectral image data.

[0030] Processing the spectral image data to identify at least two regions of interest may include registering the spectral image data to an anatomical atlas that identifies expected regions of interest, and identifying regions in the registered spectral image data that fall within the expected regions of interest of the anatomical atlas as at least two regions of interest.

[0031] Also provided is a processing system for processing energy spectral image data of an object generated by a computed tomography scanner, the processing system being configured to: obtain the energy spectral image data at an input interface; process the energy spectral image data to identify at least two regions of interest, each region of interest representing a different set of one or more organs; process each region of interest using a corresponding one of a plurality of sets of one or more machine learning algorithms, each machine learning algorithm being configured to process the region of interest to generate a prediction indicator indicating the likelihood that the region of interest contains at least one representation of a predetermined pathology; and output at least one prediction indicator generated by the processing of each region of interest at an output interface.

[0032] The processing system may be adapted to perform any of the methods described herein, as will be apparent to the skilled person.

[0033] A system is also proposed, the system comprising a user interface and a processing system configured to control the user interface to provide a visual representation in response to each predictive indicator.

[0034] Also proposed is a non-transitory computer program product comprising computer program code which, when executed on a computing device having a processing system, causes the processing system to perform all the steps of any method described herein.

[0035] Any advantages of the computer-implemented method according to the present invention are similarly and similarly applicable to the system and computer program disclosed herein. It will be appreciated by those skilled in the art that two or more of the above-mentioned options, embodiments and / or aspects of the present invention may be combined in any manner deemed useful.

[0036] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] For a better understanding of the invention and to show more clearly how it may be put into practice, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0038] Figure 1 Methods for assisting clinicians are illustrated;

[0039] Figure 2 is a flow chart illustrating an embodiment of the inventive method; and

[0040] Figure 3 is a block diagram illustrating an embodiment of the inventive system. DETAILED DESCRIPTION

[0041] Various embodiments of the present invention will be described with reference to the accompanying drawings.

[0042] It should be understood that the detailed description and specific examples are intended for illustration purposes only and are not intended to limit the scope of the invention when indicating exemplary embodiments of the device, system and method. These and other features, aspects and advantages of the device, system and method of the present invention will become better understood from the following description, claims and drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0043] The present invention provides a mechanism for generating predictive indicators of possible pathology. Spectral (CT) image data is processed to identify regions of interest, which represent different sets of one or more organs. Each region of interest is then processed using a corresponding set of one or more machine learning algorithms to generate a corresponding number of predictive indicators for each region of interest. At least one of these predictive indicators is then output.

[0044] Embodiments are based on the recognition that spectral image data contains a large amount of data, which may potentially overwhelm a clinician trying to view the spectral image data. By performing region-specific processing on the spectral image data, a method of generating indicators on a region-by-region basis may be provided to the clinician, which means that the clinician can quickly identify the most relevant regions for investigation, thereby significantly reducing the amount of time spent identifying the most relevant regions.

[0045] Further embodiments recognize that certain portions of the spectral image data, such as certain spectral images in the spectral image data, provide more useful information to the clinician than other portions. Thus, the method provides a technique for selecting (one or more) spectral images to provide along with the predictive indicator to improve the contextual awareness and relevance of the output information.

[0046] The disclosed methods may be used in any suitable medical / clinical setting, for example, for analysis during an emergency medical situation. However, the embodiments may also be used for research or analytical purposes.

[0047] Some embodiments provide a mechanism for a diagnostic clinical radiology support system. The proposed method allows: marking and identifying relevant and important findings or predictions in an increased spectral imaging space; sorting the findings or predictions according to selected criteria (such as relevance and / or severity); and providing visualization with key or important spectral images, which allows confirmation of any prediction.

[0048] The present invention has the potential to improve existing CT reading workflows by focusing the reader's attention and optionally providing the most adequate or appropriate spectral results (eg, upon request).

[0049] More particularly, it has been recognized that spectral image data contains a large amount of complex and non-intuitive data. This makes it a difficult and time-consuming task for clinicians to accurately evaluate spectral data. This can have a significant impact on subject outcomes, particularly in urgent or time-sensitive situations, such as during medical emergencies. By providing pointers or indicators to specific areas or regions of spectral image data, a significant reduction in wasted time and thus improved patient outcomes can be achieved.

[0050] In the context of the present disclosure, an image or image data contains a representation of an element in an image domain representation, or a version of an element is present in an image or image data. Spectral image data may be formed from a plurality of spectral images - each spectral image representing a different type of spectral image data. Techniques for generating spectral image data have been previously described and are established in the art.

[0051] Figure 1 An overview of a system 100 or method employed by some embodiments of the present invention is provided.The system 100 is conceptually formed of three processing functional blocks or groups.

[0052] The first block 110 represents a training engine that generates or trains region-specific machine learning algorithms. Each trained region-specific machine learning algorithm is configured to process a portion of the spectral data containing a representation of a region of interest, which is a collection of one or more organs.

[0053] The second block 120 represents a prediction component. The prediction component is configured to perform reasoning using a region-specific machine learning algorithm. In particular, the machine learning algorithm is used to process the spectral image data of the object. The second block 120 performs a process of identifying portions of the spectral image data representing different regions of interest and then processing each portion using a corresponding region-specific machine learning algorithm.

[0054] The third block 130 represents a reporting and information component. This component is configured to use the results of the reasoning performed by the prediction component to alert or guide the clinician to identify the most relevant outputs of the machine learning algorithm(s) and / or the relevant portions of the spectral data that correspond to or best represent these outputs.

[0055] Purely by example, if each machine learning algorithm is configured to generate a bounding box that identifies a particular pathology (such as a disease, lesion, or abnormality), then the third block may be able to identify the most clinically relevant or confident bounding box - optionally along with the most relevant and conclusive portion of the spectral image data that produced the bounding box(es).

[0056] A more complete understanding of the system 100 will be apparent from the following description, which describes a method of embodying the method of the system 100 .

[0057] Figure 2 A method 200 according to an embodiment is illustrated. The method 200 is configured for processing spectral image data of an object. The method 200 may effectively represent actions taken by a prediction component, and optionally the reporting and information components previously described.

[0058] The method 200 may be performed by a processing system. The processing system may be configured to receive energy spectrum image data at an input interface. The processing system may further include a processing unit for processing the received energy spectrum image data.

[0059] Spectral image data is image data generated by a computed tomography scanner, and in particular a computed tomography scanner operating in a spectral imaging mode.

[0060] The spectral image data may include a plurality of different spectral images. Preferably, the spectral image represents different types of spectral image data. For example, the spectral image may include a photoelectric effect image, a Compton scattering image, and the like. Other examples include an iodine image indicating the presence of iodine; a water image indicating the presence of water; a contrast agent image indicating the presence of a contrast agent; an effective atomic number image, and the like.

[0061] In some examples, method 200 includes a step 205 of obtaining spectral image data. This may include retrieving spectral image data from a memory and / or from a computed tomography scanner. In some examples, step 205 includes generating spectral image data, for example, using a known spectral imaging method. Of course, if spectral image data already exists or has been previously obtained, step 205 may be omitted.

[0062] The method 200 includes a step 210 of processing the spectral image data to identify at least two regions of interest, each region of interest representing a different set of one or more organs. An organ is a biological or anatomical organ of a subject represented by the spectral image data.

[0063] For example, step 210 can be performed by registering the spectral image data to an anatomical atlas that identifies the expected region of interest. Typically, the spectral image data represents a portion of the body. It is possible to register the portion of the body, and therefore the spectral image, to an anatomical atlas. The method can utilize a shape matching algorithm, etc. Then, identifying the region of interest in the spectral image data is performed by identifying each region in the registered spectral image that corresponds to the location of the expected region of interest in the anatomical atlas.

[0064] Another method for performing step 210 is to perform an image segmentation process on the spectral image data. This may include executing a series of one or more image segmentation algorithms, for example, iteratively subdividing the representation of the object in the spectral image data into regions of interest, or individually identifying different regions of interest.

[0065] The method 200 then performs a step 220 of processing each region of interest using a corresponding one of a plurality of sets of one or more machine learning algorithms. Thus, each region of interest (i.e., each portion of the spectral image data representing the identified region of interest) is processed using a corresponding set of one or more machine learning algorithms. Thus, each set of one or more machine learning algorithms processes a different region of interest.

[0066] In this manner, step 220 may include inputting each region of interest into each machine learning algorithm in a corresponding set of machine learning algorithms. Different regions of interest are provided to different sets of one or more machine learning algorithms.

[0067] Each machine learning algorithm is configured to process the region of interest to generate a prediction indicator indicating a likelihood that the region of interest contains at least one representation of a predetermined pathology. More particularly, each machine learning algorithm processes a portion or part of the spectral image data representing the region of interest to generate the prediction indicator.

[0068] Each predetermined pathology may include a disease, lesion, growth or abnormality in a corresponding region of interest.Other suitable pathologies that may be identified by a machine learning algorithm processing spectral image data will be apparent to the skilled person.

[0069] Therefore, step 220 is configured to generate at least one prediction indicator for each region of interest. Each region of interest is processed using a different set of one or more machine learning algorithms. The number of machine learning algorithms in each set can be greater than 1, so that the number of prediction indicators generated for each region of interest can also be greater than 1.

[0070] It is not necessary for each machine learning algorithm to process the entire portion of the spectral image data representing the region of interest. For example, if the spectral image data includes multiple spectral images, the machine learning algorithm can process a portion of one or more (e.g., a subset rather than all) of the multiple spectral images representing the region of interest.

[0071] Methods for generating machine learning algorithms for use in step 220 are established in the art. Embodiments of the present invention propose the use of region-specific algorithms, for example, algorithms trained to process specific regions or zones of energy spectrum image data. Therefore, when generating machine learning algorithms, the training data used to train each algorithm may only contain data for a specific region and / or may contain annotations relevant to a specific region, such as ground truth identification. For the sake of completeness, a more complete description of how to train a machine learning algorithm is provided later in this disclosure.

[0072] The method then performs a step 320 of outputting at least one prediction indicator generated by the processing of each region of interest.

[0073] In some examples, step 230 includes outputting all prediction indicators generated by step 220. Thus, the method may move directly from step 220 to step 230.

[0074] In other examples, step 230 includes outputting only a subset, i.e., and not necessarily all, of the prediction indicators generated by step 220. Thus, method 200 may include an optional step 225 of performing some further processing or filtering of the prediction indicators generated by step 220 to identify at least one prediction indicator output by step 230. Step 225 thus includes selecting a subset of the prediction indicators generated in step 220.

[0075] In some examples, step 230 includes outputting the at least one predictive indicator to an additional processing system for further analysis and / or processing.For example, the at least one predictive indicator may be provided to a pathology analysis kit for analyzing a subject for a particular pathology or condition.

[0076] In a preferred example, step 230 includes controlling the user interface to provide a visual representation in response to each of the at least one predictive indicator(s). Thus, step 230 may effectively perform the operation of the reporting and information component.

[0077] To improve relevance and reduce clinician uncertainty, some embodiments utilize a confidence score or metric for the result(s) produced by each machine learning algorithm.A confidence metric is an alternative label to a confidence score.

[0078] Thus, in some examples, each machine learning algorithm is further configured to generate a confidence score for the prediction indicator. The confidence score represents a confidence in the likelihood that the region of interest contains at least one indication of a predetermined pathology. The confidence score is a numerical value.

[0079] In this manner, step 220 may be configured to generate a confidence score or confidence metric for each prediction indicator.

[0080] Methods for configuring machine learning algorithms to generate confidence scores are well established in the art. By way of example only, multiple methods for generating confidence scores are disclosed in "Quantitative evaluation of confidence measures in a machine learning world." (Proceedings of the IEEE International Conference on Computer Vision. 2017) by Poggi, Matteo, Fabio Tosi, and Stefano Mattoccia. Lambrou, Antonis, Harris Papadopoulos, and Alex Gammerman "Reliable confidence measures for medical diagnosis with evolutionary algorithms." (IEEE Transactions on Information Technology in Biomedicine 15.1 (2010): 93-99) disclose other methods that are particularly useful for medical applications.

[0081] If performed, step 225 may process the prediction indicators generated in step 220 using confidence scores to identify at least one prediction indicator output by step 230 .

[0082] In particular, step 225 may include identifying or selecting those predictive indicators whose confidence scores satisfy one or more predetermined criteria.

[0083] For example, step 225 may include selecting only the predictive indicators having one of the X largest confidence scores, where X is a real integer. Example values ​​of X include 5, 10, or 20. In this way, only the most relevant and / or most confident results of pathology identification are identified, for example, and provided to the individual. This improves the relevance and contextual understanding of the predictive indicators.

[0084] As another example, step 225 may include selecting only prediction indicators having a confidence score greater than a threshold confidence score. This avoids providing inaccurate information to the clinician. The threshold confidence score may be set by the clinician and / or based on clinical guidelines, practice, or one or more policies. For example, the clinician may modify the value of the threshold confidence score at the user interface.

[0085] In another example, step 225 may include selecting only predictive indicators having a confidence score greater than an indicator-specific threshold confidence score. The indicator-specific threshold confidence score may vary for each predictive indicator and, therefore, for each machine learning algorithm. For example, the value of the indicator-specific threshold may depend on the region processed by the corresponding machine learning algorithm and / or the pathology investigated by the machine learning algorithm.

[0086] The method thus effectively controls the (allowed) false positive rate for the selected prediction indicator. The method recognizes that, for different regions or pathologies, different levels of false positives can be considered as acceptable to clinicians. For example, clinicians can tolerate a higher false positive level of a certain deception of finding cancer that is very harmful to the object, but may adhere to a low false positive rate for a simple diagnosis of cancer. Therefore, the value of each indicator-specific threshold value can be set by a clinician and / or based on clinical guidelines, practice or one or more strategies. For example, a clinician can modify the value of each indicator-specific threshold value at a user interface.

[0087] Step 230 may be configured to control the output of at least one prediction indicator using the confidence metric. For example, if step 230 includes controlling a user interface to provide a visual representation of the prediction indicator(s), then step 230 may be configured to provide greater visual emphasis to any prediction indicator having a higher confidence score than to any prediction indicator having a lower confidence score.

[0088] Providing greater visual emphasis can be achieved by using colors that control the visual representation of the prediction indicator in response to the confidence score. For example, different levels of confidence scores can be set at different colors or color saturation levels. By way of example, more confident prediction indicators can be colored green and less confident indicators can be colored red. Prediction indicators that are in between can be colored on a gradient or scale between green and red.

[0089] As another example, providing greater visual emphasis may be achieved by controlling the position or visual prominence of the prediction indicator(s) in response to their confidence score. For example, if the prediction indicators are presented in a list, the order of the prediction indicators in the list may be responsive to the confidence score—e.g., indicators associated with higher scores may be positioned higher in the list.

[0090] As another example, the size of each visual representation of the prediction indicator(s) may be responsive to the confidence score(s) of each prediction indicator. For example, a prediction indicator with a greater confidence score may have a larger or more prominent visual representation at the user interface than a prediction indicator with a smaller confidence score.

[0091] In at least one example, the step of controlling the user interface includes sorting all prediction indicators to be visually represented based on their confidence metrics to determine an order of the prediction indicators, and controlling the user interface to provide a visual representation of the determined order of the prediction indicators.

[0092] This provides a mechanism for effectively controlling the order of presentation of the prediction indicators, and therefore the visual emphasis of the prediction indicators, in response to the confidence score of each prediction indicator.Those skilled in the art will be aware of methods for ordering values ​​based on their magnitude.

[0093] In at least one embodiment, each confidence metric is weighted, biased, or modified in response to context information. This process may be performed as part of step 220.

[0094] For example, the context information may identify one or more desired or target regions of interest. Modifying the confidence score may include, for example, weighting the prediction metric more heavily for regions of interest that include, are included by, or are near the desired region of interest compared to more distant regions of interest.

[0095] The method recognizes that the underlying clinical setting or context will affect the relevance of different predictive indicators. For example, if a clinician is investigating chest pain, then predictive indicators for regions of interest near the chest (such as the heart or lungs) will have greater clinical relevance than regions further from the chest. As another example, if a clinician is investigating leg pain, then predictive indicators for the buttocks, legs, and / or feet will have greater clinical relevance than regions of the head or heart.

[0096] For example, the context information may identify one or more desired pathologies. Each machine learning algorithm is configured to generate an indicator for a particular pathology. Modifying the confidence score may include, for example, weighting the prediction metric for the desired pathology more heavily than for other pathologies.

[0097] The approach recognizes that one or more pathologies of the machine learning approach will affect the relevance of different predictive indicators.

[0098] The weighting or modification may be performed, for example, by multiplying the confidence measure by a determined value, which may be within a predetermined scale, such as 0 to 1. In this context, heavier weighting means multiplying by a larger value. Other methods for performing weighting, including methods based on non-linear weighting or division, will be apparent to the skilled person.

[0099] The approach provides a mechanism for providing more contextually relevant control of confidence scores and providing contextually useful information to individuals at the user interface.

[0100] Thus, step 220 may comprise modifying each confidence metric in response to the context information.Thus, the method may comprise step 240 of obtaining context information.

[0101] The context information may include indicators of the target region of interest and / or pathology.Step 220 may include weighting the predicted indicators of the regions of interest based on or in response to the proximity or relevance of each region of interest to the target region of interest and / or pathology.

[0102] Any of the methods described herein that utilize confidence values, such as selecting predictive indicators having confidence values ​​that satisfy one or more predetermined criteria, can be adapted to utilize weighted confidence values ​​instead. This makes the results more clinically relevant, i.e., relevant to an ongoing or related medical procedure.

[0103] In some examples, at least one set of one or more machine learning algorithms includes multiple machine learning algorithms configured to identify the same pathology in a region of interest associated with the set in step 220. Thus, for at least one region of interest, multiple predictive indicators can be generated for the same pathology.

[0104] In such an example, for any region processed using the ensemble including multiple machine learning algorithms, the confidence scores for the prediction indicators associated with the same pathology can be used to select one (e.g., only one) of the prediction indicators for output. In particular, the prediction indicator with the highest confidence score for that particular pathology in that particular region of interest can be selected.

[0105] Thus, step 220 may include, for at least one of the regions of interest, generating a plurality of prediction indicators for the same pathology with corresponding confidence scores. For each such region of interest, one of the plurality of prediction indicators for the same pathology may be selected for further use based on the confidence score of each prediction indicator, e.g., for use or output in step 230. The selection may be performed in step 220 or step 225 (if performed).

[0106] This approach allows the most relevant or reliable predictive indicators to be used in further processing, thereby increasing the accuracy and relevance of the predictive indicators to the clinician.

[0107] The method is particularly advantageous when the spectral image data comprises a plurality of spectral images, and each machine learning algorithm for identifying the same pathology using a particular region of interest is configured to process a different set of one or more spectral images to produce a predictive indicator.

[0108] The method recognizes that different spectral images are advantageous for identifying certain pathologies and / or that imaging conditions may affect which spectral images are best able to identify a particular pathology with the greatest confidence at any given time. The method thereby maintains a high level of accuracy in generating predictive indicators.

[0109] In some examples, at least one of the machine learning algorithms includes a location recognition machine learning algorithm.

[0110] The location identification machine learning algorithm is configured to process the region of interest to identify within the region of interest the location of any representation of a predetermined pathology that is predicted to be present with a likelihood greater than a predetermined likelihood. For example, identifying the location of a pathology within the image data may be performed by executing a segmentation technique trained to identify a particular pathology.

[0111] Thus, step 220 may be configured such that the prediction indicator generated by the location identification machine learning algorithm indicates any identified location within the region of interest. For example, this may take the form of a bounding box or contour that identifies the location of the predicted pathology.

[0112] In some examples, where step 230 includes providing a visual representation of the prediction indicator(s), step 230 is configured to, for each identified location of the at least one prediction indicator(s), superimpose the visual representation of the identified location on the visual representation of the spectral image data. The method provides the clinician with useful and contextual information about the identified location.

[0113] Purely by way of example, if each machine learning algorithm is configured to generate a bounding box that identifies a particular pathology, such as a disease, lesion, or abnormality, then step 230 may include providing a visual representation of the bounding box(es) superimposed on a visual representation of the spectral image data at the relevant location. In some examples, this process of superimposing the visual representation of the identified location is performed only in response to user input at the user interface. For example, a user may be able to select or otherwise identify a particular visual representation of a predictive indicator, and an appropriate overlay for that predictive indicator may be provided.

[0114] In other examples, the process is performed for each of the at least one prediction indicator(s).

[0115] In some examples, for each identified location of the at least one prediction indicator(s), step 230 is configured to provide a visual representation of a portion of the spectral image data near the identified location. In particular, the visual representation of the portion of the spectral image data may include at least a visual representation of the identified location.

[0116] The size of the portion may be smaller than the (total) size of the (total) spectral image data. In particular, the portion may represent only the region of interest of the spectral image data that is processed to generate the prediction indicator and / or the identified location.

[0117] This method provides clinicians with useful and specific clinical information.

[0118] If the spectral image data includes more than one spectral result, ie, more than one spectral image, step 230 may be configured to select and display one of the spectral images or a portion of one of the spectral images for display at the user interface together with the prediction indicator.

[0119] In some examples, the process of displaying one or more of the spectral images is performed only in response to user input at the user interface. For example, a user may be able to select or otherwise identify a particular visual representation of a prediction indicator, and an appropriate spectral image for that prediction indicator may be provided.

[0120] In other examples, the process is performed for each of the at least one prediction indicator(s).

[0121] The selection of the spectral image can be based on the type of pathology associated with the predictive indicator. The method recognizes that some spectral images are more clinically useful than other spectral images for specific pathologies. For example, an effective atomic number image (also called a Z image) can be selected for a predictive indicator for a lesion-type pulmonary embolism. Therefore, the method provides more contextually useful information.

[0122] As another example, the selection of the spectral image may also or additionally be based on characteristics of the spectral image. By way of example, the spectral image with the maximum contrast may be selected as the spectral image to be provided together with the prediction indicator. As another example, the spectral image with the maximum signal-to-noise ratio may be selected as the spectral image to be provided together with the prediction indicator.

[0123] Thus, for each spectral image of the spectral image data, step 230 may include the step of performing one or more image analysis techniques on the spectral image to identify one or more image properties or characteristics. Step 230 may then include using the one or more image properties or characteristics to determine which spectral image to provide alongside the prediction indicator, for example, selecting the spectral image with maximum contrast, maximum image-to-noise ratio, etc.

[0124] As yet another example, the selection of a spectral image may also or additionally be responsive to one or more user-defined settings. Different users may have different preferences for different types of spectral images, which may be taken into account.

[0125] In some examples, the selection of a spectral image may be based on the pathology type and one or more user-defined settings. Thus, a user or individual may be able to define preferred spectral images for different types of pathologies, such that the pathology type associated with the prediction indicator may be used along with the user preferences to select a spectral image.

[0126] It has been previously described how a machine learning algorithm may process only a portion of one or more (e.g., a subset or non-all) of a plurality of spectral images representing an area of ​​interest to produce a prediction indicator. In such an embodiment, the selection of a spectral image may be based on the spectral image(s) used to produce the corresponding prediction indicator. For example, the selected spectral image may be one of the spectral images(s) used to produce the prediction indicator. In this manner, step 230 may include selecting the spectral image(s) used or processed by the machine learning algorithm to generate the prediction indicator as the spectral image(s) to be provided along with the prediction indicator.

[0127] The proposed method for selecting a spectral image from spectral images of spectral image data can be combined with the concept of providing a visual representation of a portion of the spectral image data near the identified pathological location. In particular, the portion of the spectral image data can be a portion or part of a specific spectral image or spectral result containing the identified pathological location. The method for selecting a specific spectral image has been previously described.

[0128] Figure 3 A system 300 according to an embodiment is illustrated. The system includes a processing system 310, which is itself an embodiment, and a user interface 320. The processing system 310 includes a user interface 311, which is configured to obtain spectral image data. The spectral image data includes a spectral representation of an object or individual, for example, an individual who has undergone or is undergoing a spectral imaging process. For example, the user interface 311 can obtain or retrieve the spectral image data from the memory 350. Alternatively, the spectral image data can be obtained from a computed tomography scanner (e.g., operating in a spectral imaging mode).

[0129] The processing system 310 includes a processing unit, processing circuit or module 315 configured to process the spectral image data to identify at least two regions of interest, each region of interest representing a different set of one or more organs. Methods for appropriately processing spectral image data that the processing unit 315 may employ have been previously described.

[0130] The processing unit 315 is also configured to process each region of interest using a corresponding one of a plurality of sets of one or more machine learning algorithms, each machine learning algorithm being configured to process the region of interest to generate a prediction indicator indicating the likelihood that the region of interest contains at least one representation of a predetermined pathology. The processing unit 315 may be configured to output at least one prediction indicator generated by the processing of each region of interest via the output interface 312. The output may be controlled by the processing unit 315 of the processing system 310. In some examples, the processing unit 315 is configured to select at least one prediction indicator from all prediction indicators generated using the set of machine learning algorithms. The selection may be responsive to a confidence score generated by each machine learning algorithm in the set of one or more machine learning algorithms.

[0131] The user interface 320 of the system 300 may be configured to provide a visual representation of the (selected) at least one prediction indicator. The operation of the user interface 320 may be controlled by the processing unit 315, for example, via the output interface 312. By way of example, the processing unit 315 may be configured to generate display data processed by the user interface 320 to provide a visual representation of the (one or more) prediction indicators. Thus, the processing unit 315 may be configured to control the user interface 320 via the output interface 312 to provide a visual representation in response to the at least one prediction indicator.

[0132] A skilled person will readily appreciate how the processing system 310 may be appropriately configured to perform any of the previously described methods and / or approaches. For example, the processing system 310 may be configured to generate a confidence value for each prediction indicator, select a subset of (one or more) prediction indicators using the confidence values, control visual representations based on the confidence values, control which visual representations are provided based on the confidence values, control visual emphasis of prediction indicators responsive to confidence values, weight confidence values ​​using contextual information, and the like.

[0133] Embodiments utilize a machine learning algorithm to process regions of interest of spectral image data. A machine learning algorithm is any self-training algorithm that processes input data to produce or predict output data. Here, the input data includes spectral image data or a region of spectral image data, and the output data includes an indicator indicating the likelihood of a particular pathology (such as a disease, lesion, growth, or abnormality) being present in the corresponding region of interest.

[0134] Suitable machine learning algorithms for the present invention will be apparent to those skilled in the art. Examples of suitable machine learning algorithms include decision tree algorithms and neural networks. Other machine learning algorithms (such as logistic regression, support vector machines, or naive Bayes models) are suitable alternatives.

[0135] The structure of a neural network is loosely inspired by the human brain. A neural network comprises layers, each layer comprising multiple neurons. Each neuron comprises a mathematical operation. In particular, each neuron may comprise a different weighted combination of a single type of transformation (e.g., transformations of the same type, sigmoid transformations, etc., but with different weights). In the process of processing input data, the mathematical operation of each neuron is performed on the input data to produce a numerical output, and the output of each layer in the neural network is sequentially fed into the next layer. The last layer provides the output.

[0136] Methods for training machine learning algorithms are well known. Typically, such methods include obtaining a training data set, which includes training input data entries and corresponding training output data entries. An initialized machine learning algorithm is applied to each input data entry to generate a predicted output data entry. The error between the predicted output data entry and the corresponding training output data entry is used to modify the machine learning algorithm. The process can be repeated until the error converges and the predicted output data entry is sufficiently similar to the training output data entry (e.g., ±1%). This is generally referred to as a supervised learning technique. For example, in the case where the machine learning algorithm is formed by a neural network, the mathematical operation (weighting) of each neuron can be modified until the error converges. Known methods for modifying neural networks include gradient descent, back propagation algorithms, etc.

[0137] The training input data entries correspond to example spectral image data or portions of spectral image data. The training output data entries correspond to indicators indicating the likelihood of the presence of a particular pathology, such as a disease, lesion, growth or abnormality, in a corresponding region of interest.

[0138] Embodiments utilize processing units and / or processing systems. The processing unit / system can be implemented in many ways using software and / or hardware to perform the various functions required. A processor is an example of a processing unit / system that uses one or more microprocessors that can be programmed using software (such as code) to perform the required functions. However, the processing unit / system can be implemented with or without a processor, and can also be implemented as a combination of dedicated hardware that performs some functions and processors (e.g., one or more programmable microprocessors and associated circuits) that perform other functions. Examples of processing system components that can be used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0139] In various embodiments, a processor or processing unit / system may be associated with one or more storage media, such as volatile and non-volatile computer memory, such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs or instructions that perform the desired functions when executed on one or more processors and / or processing units / systems. The various storage media may be fixed to the processor or processing unit / system or may be transportable so that one or more programs / instructions stored thereon may be loaded into the processor or processing unit / system.

[0140] It will be understood that the disclosed method is preferably a computer-implemented method. Thus, a concept of a computer program is also proposed, and the computer program includes a code unit for implementing any described method when the program is run on a processing system (such as a computer). Therefore, the different parts, rows or blocks of the code of the computer program according to the embodiment can be performed by a processing system or a computer to perform any method described herein. In some alternative embodiments, the function described in (one or more) block diagrams or (one or more) flow charts may not occur in the order described in the accompanying drawings. For example, two frames shown in succession can actually be performed substantially concurrently or frames can sometimes be performed in reverse order, depending on the function / action involved.

[0141] By studying the drawings, the disclosure and the claims, those skilled in the art can understand and implement the variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word includes but does not exclude other elements or steps, and the word one or one does not exclude multiple. A single processor or other unit can implement the functions of several items recorded in the claims. Although specific measures are recorded in mutually different dependent claims, this does not indicate that the combination of these measures cannot be used advantageously. If the computer program is as described above, it can be stored / distributed on a suitable medium (such as an optical storage medium or solid-state medium provided together with other hardware or as part thereof), but it can also be distributed in other forms, such as distributed via the Internet or other wired or wireless telecommunications systems. If the term is suitable for use in the claims or the specification, it should be noted that the term is suitable for being intended to be equivalent to the term being configured as. If the term arrangement is used in the claims or the specification, it should be noted that the term arrangement is intended to be equivalent to the term system, and vice versa. Any figure mark in the claims should not be interpreted as a limitation on the scope.

Claims

1. A computer-implemented method of processing spectral image data of an object generated by a computed tomography scanner, the computer-implemented method comprising: processing the spectral image data to identify at least two regions of interest, each region of interest representing a different set of one or more organs; processing each region of interest using a corresponding set of a plurality of sets of one or more machine learning algorithms, each machine learning algorithm configured to process the region of interest to generate a prediction indicator indicating a likelihood that the region of interest contains at least one indication of a predetermined pathology; and At least one prediction indicator generated by the processing for each region of interest is output.

2. The computer-implemented method of claim 1 , wherein: Each machine learning algorithm is further configured to generate a confidence score for the prediction indicator generated by the machine learning algorithm, the confidence score representing a confidence in the likelihood that the region of interest contains the at least one indication of the predetermined pathology; The computer-implemented method further comprises: selecting at least one of the prediction indicators generated by the processing of each region of interest in response to each confidence score generated by the set of one or more machine learning algorithms; and Outputting at least one prediction indicator includes outputting the selected at least one prediction indicator.

3. The computer-implemented method of claim 1 or 2, wherein: Outputting the at least one predictive indicator includes controlling a user interface to provide a visual representation in response to the at least one predictive indicator.

4. The computer-implemented method of claim 3, wherein: Each machine learning algorithm is further configured to generate a confidence score for the predictive indicator generated by the machine learning algorithm, the confidence score representing a confidence in the likelihood that the region of interest contains the at least one indication of the predetermined pathology; and Controlling the user interface is also responsive to each confidence score generated by the set of one or more machine learning algorithms.

5. The computer-implemented method of claim 4, wherein: Controlling the user interface includes providing greater visual emphasis to any one of the at least one prediction indicator having a higher confidence score than to any one of the at least one prediction indicator having a lower confidence score.

6. A computer-implemented method according to any one of claims 4 to 5, wherein: Controlling the user interface comprises: sorting the at least one prediction indicator based on a confidence metric of the at least one prediction indicator to determine an order of the at least one prediction indicator; and The user interface is controlled to provide a visual representation of the determined order of the at least one predictive indicator.

7. A computer-implemented method according to any one of claims 1 to 6, wherein: At least one of the machine learning algorithms comprises a location identification machine learning algorithm, wherein: The location identification machine learning algorithm is configured to process the region of interest to identify within the region of interest locations of any representations of the predetermined pathology that are predicted to be present with a likelihood greater than a predetermined likelihood; and The predicted indicators generated by the location identification machine learning algorithm are indicative of any identified locations within the region of interest.

8. The computer-implemented method of claim 7, wherein: Outputting each predictive indicator comprises controlling a user interface to provide a visual representation in response to the at least one predictive indicator; and For each location identified by any location identification machine learning algorithm that provides the at least one predictive indicator, the visual representation includes a visual representation responsive to the identified location.

9. The computer-implemented method of claim 8, wherein: Controlling the user interface includes, for each location identified by any location identifying machine learning algorithm that provides the at least one predictive indicator, superimposing a visual representation of the identified location on the visual representation of the spectral image data.

10. The computer-implemented method of claim 8 or 9, wherein: Controlling the user interface includes, for each location identified by any location identifying machine learning algorithm that provides the at least one predictive indicator, providing a visual representation of a portion of the image data proximate the identified location.

11. The computer-implemented method of any one of claims 1 to 10, wherein: Each predetermined pathology includes a disease, lesion, growth or abnormality in a corresponding region of interest.

12. The computer-implemented method of any one of claims 1 to 11, wherein: Processing the spectral image data to identify at least two regions of interest includes: performing an image segmentation process on the energy spectrum image data; or The spectral image data is registered to an anatomical atlas that identifies expected regions of interest; and regions of the registered spectral image data that fall within the expected regions of interest of the anatomical atlas are identified as the at least two regions of interest.

13. A processing system for processing spectral image data of an object generated by a computed tomography scanner, the processing system comprising: a memory storing a plurality of instructions; as well as a processor circuit coupled to the memory and configured to execute the plurality of instructions to: Obtaining the energy spectrum image data at an input interface; processing the spectral image data to identify at least two regions of interest, each region of interest representing a different set of one or more organs; processing each region of interest using a corresponding set of a plurality of sets of one or more machine learning algorithms, each machine learning algorithm configured to process the region of interest to generate a prediction indicator indicating a likelihood that the region of interest contains at least one indication of a predetermined pathology; and At least one prediction indicator generated by the processing for each region of interest is output.

14. The processing system of claim 13, wherein: Each machine learning algorithm is further configured to generate a confidence score for the predictive indicator generated by the machine learning algorithm, the confidence score representing a confidence in the likelihood that the region of interest contains the at least one indication of the predetermined pathology; and The processor circuit is configured to execute the plurality of instructions to perform the following operations: selecting at least one of the prediction indicators generated by the processing of each region of interest in response to each confidence score generated by the set of one or more machine learning algorithms; and Outputting at least one prediction indicator includes outputting the selected at least one prediction indicator.

15. A non-transitory computer program product comprising computer program code which, when executed on a computing device having a processing system, causes the processing system to perform all of the methods according to any one of claims 1 to 12.

Images