TOOLS AND METHOD FOR THE ANALYSIS OF MEDICAL IMAGES OF PATIENTS

Description

[0001] The current invention relates to a tool to analyse medical images of patients, using a database comprising preloaded medical images.

[0002] Specifically, the tool is designed to analyse medical images of patients by using an algorithm.

[0003] In healthcare, time, especially wasted time, contributes to the overcrowding of hospitals, anxious patients and a reduced outcome of the patients.

[0004] It is commonly accepted that the sooner a patient gets the right care in the right place with the right resources, the higher the chances are to obtain a positive outcome.

[0005] The value of time is highly associated with patients' lives in healthcare.

[0006] A known problem in healthcare are the inefficiencies in the process starting from the first appearance of symptoms of a patient to be examined by the right doctor with the right expertise until obtaining optimal care. Very often, valuable time is already wasted from even before a patient has been diagnosed with a certain disease due to, for example, the complexity of the disease, long waiting lists to visit certain expert doctors, et cetera.

[0007] Furthermore, it is generally known that these inefficiencies increase the cost of proper healthcare dramatically.

[0008] Document WO 2015 / 134530 Al discloses methods and a personalized system for retrieving similar image-based subject data. A method for adaptive learning of imaging data comprises accessing a subject database comprising subject imaging data. Next a search of the subject database is conducted using one or more search parameters which can include one or more image data associated with a subject undergoing treatment. Next at least one match of the search can be provided on a user interface.

[0009] Document US 2019 / 267133 Al discloses an appointment scheduling device for scheduling an appointment for a patient to visit a health provider includes an embedder a predictor and a scheduler. The embedder receives input data about the patient. The input data is associated with a request to schedule the appointment with the health provider. The embedder generates an embedding based on the input data. The predictor receives the embedding and predicts an appointment parameter based on the embedding. The scheduler schedules the appointment based on the appointment parameter.

[0010] Finally, document US 2019 / 371439 Al discloses an apparatus for determining similarity between medical data sets for a plurality of patients or other subjects comprises at least one data store and a processing resource.

[0011] The purpose of the current invention is to provide a solution to the aforementioned and other disadvantages.

[0012] The current invention relates to a tool to analyse medical images of patients as defined in appended claim 1, the tool using a database comprising preloaded medical images, whereby the tool has an input interface for loading a plurality of medical images of different patients to be analysed, wherein the tool is provided with an algorithm to prioritise patients of which the medical images have been loaded according to the similarity between their medical images to be analysed and the preloaded medical images already provided in the database, for generating a disease similarity index, wherein the said index is used to rank and prioritise the handling of patients, the patients of which the loaded medical images have the highest similarity index are ranked with the highest priority.

[0013] An advantage of a tool according to the invention is that it provides an easy and fast manner to guide healthcare practitioners to decide which patients should be handled first due to a higher priority proposed by the tool, for example, to perform extra tests to diagnose patients with a certain disease or referring patients immediately to a doctor with the necessary expertise for optimal care.

[0014] The algorithm used to prioritise patients is based on the number of cases for the most relevant disease weighted by their computed similarity for each image to be analysed, multiplied by a tunable coefficient K and by the number of cases for other relevant diseases and divided by a tunable parameter Rmax.

[0015] In a preferred embodiment the tunable coefficient K is determined by an interactive iterative process wherein a subset of selected data preloaded in the database is chosen as a validation set and wherein the remaining preloaded data in the database are used as a training set, and wherein at the start a value for the tunable coefficient K is chosen and then running the algorithm to prioritise the medical images in the validation set using data from the training set and investigating if the performance of the prioritisation is acceptable, and in case in which the performance is not acceptable to repeat this algorithm each time with a different value for the coefficient K until an acceptable performance is achieved.

[0016] In yet another preferred embodiment, the tunable parameter Rmax is determined, depending on the obtained value of the tunable coefficient K, in such a way that the similarity index is limited to a value of about one hundred percent.

[0017] The preloaded medical images provided in the database are preferably present in the database in the form of a semantical representation and the medical images of different patients to be analysed are first converted into a semantical representation in order to be compared with the semantical representations of the preloaded medical images in the database by means of cosine similarity which has been proven to give optimal results.

[0018] The semantical representations of medical images are achieved by a truncated deep neural network like DenseNet 121 and reduced by average pooling, which generates a 1024-components vector.

[0019] An advantage of this is proven practise.

[0020] Furthermore, the tool can be used to prioritise patients according to the plausible severity of a certain disease predicted by the tool.

[0021] An advantage of this is that patients potentially suffering from a certain disease with a higher severity can be examined and / or treated by the doctor first.

[0022] The database preferably contains medical images of patients taken at different stages of progression of a certain disease, and that the tool for each stage of progression provides the medical images of the best and worst possible cases, allowing the comparison of the progression of the disease of a patient with the best and worst case of the preloaded medical images in the database.

[0023] This allows to evaluate progression of the patient and of the treatment of the disease compared to the best and worst possible outcome, for example to evaluate the effectiveness of a treatment for the disease in concern.

[0024] The medical images used by the tool need to be in a certain format to obtain optimal results and therefore the medical images can be preprocessed by removal of annotations, resizing the image to l000xl000 pixels and normalization by the mean and the standard deviation.

[0025] In a preferred embodiment the tool is accommodated with an input interface to upload medical images from an external platform at distance, for example from a computer in a hospital or the like.

[0026] The invention also relates to a method to analyse medical images of patients, as defined in appended claim 9.

[0027] With the intention of better showing the characteristics of the invention, a tool and a method to analyse medical images is described hereinafter, by way of an example without any limiting nature, with reference to the accompanying drawings wherein: figure 1 demonstrates a schematic representation of a tool according to the invention; figure 2 shows a technical overview with an output of a similarity analysis according to the invention; figure 3 shows a schematic representation of a similarity algorithm according to the invention; figure 4 shows an overview of the priority algorithm according to the invention; figure 5 shows a schematic representation to obtain a tunable coefficient K and a tunable parameter Rmax according to the invention;

[0028] Figure 1 shows a schematic representation of an overview of the overall use of a tool 1 according to the invention.

[0029] For example, a patient 2A is advised to go to a hospital to obtain a medical image 3 of an area of interest, such as but not limiting to an X-ray or a computerised tomography scan, which is anonymised by eliminating personal information, randomised and stored in a database 4 at the hospital to ensure records can be tracked by healthcare practitioners.

[0030] The medical image 3A of this patient 2A can be analysed by the before mentioned tool 1 containing a suitable input interface to upload medical images 3 from an external platform 5 at the hospital, for example from a computer or a digital network in the hospital or the like.

[0031] The medical image 3 is analysed by using several algorithms 7, which analysis can afterwards be coupled to the medical image 3 when stored in the database 4 as well as the final diagnosis and treatment of the patient.

[0032] The medical image 3A of the patient 2A to be analysed can be compared by similarity to the preloaded medical images 6 present in the database 4.

[0033] Medical images 3A, 3B, 3C, 30, 3E of different patients 2A, 2B, 2C, 20, 2E to be analysed can then be prioritised by ranking the medical images 3 according to similarity with the preloaded medical images 6 and / or importance and / or severity.

[0034] Figure 2 shows a technical overview and an output 7 of a similarity analysis. In order to perform a similarity analysis the medical images 3 are analysed by the similarity algorithm 8.

[0035] The preloaded medical images 6 provided in the database 4 are present in the database 4 in the form of a semantical representation 9 and the medical images 3 of different patients 2 to be analysed are first converted into their semantical representations 9.

[0036] After the convergence of the medical images 3 into their semantical representations 9, the semantical representations 9 of the medical images 3 of the patients 2 to be analysed are compared with the semantical representations 9 of the preloaded medical images 6 in the database 4 by means of cosine similarity 10.

[0037] The output 7 of such a similarity analysis, as shown in figure 2, will provide a percentage of similarity to the five most similar cases 6A, 6B, 6C, 60, 6E found in the database 4 which will also be provided.

[0038] For example, the medical image 3A of patient 2A, in this case a computerized tomography scan of his lungs, undergoes such a similarity analysis. The output 7 provides a percentage of similarity, in this case eighty percent similarity, and the five most similar cases 6A, 6B, 6C, 6D, 6E present in the database 4. The tool 1 will also provide the final diagnosis of the patients from those most similar cases 6A, 6B, 6C, 60, 6E.

[0039] So, the similarity output 7 provides a healthcare practitioner with the following information: the medical image 3A of patient 2A is eighty percent similar to the five most similar cases, 6A, 6B, 6C, 60, 6E, and that the patients from these cases 6A, 6B, 6C, 60, 6E were ultimately diagnosed with for example coronavirus disease 2019 and therefore the medical image 3A of patient 2A demonstrates eighty percent similarity to coronavirus disease 2019.

[0040] It should be noted that for each medical image 3 of patients 2 to be analysed, the five most similar cases 6 to which the medical image 3 will be compared by means of cosine similarity 10 can and probably will be different for each patient 2 of which a medical image 3 has been loaded in the tool 1.

[0041] As shown in figure 3, the similarity algorithm 8 contains several steps. The medical images 3 of different patients 2 to be analysed can be preprocessed 11 by removal of annotations 12, resizing 13 the image to 1000x1000 pixels and normalisation 14 by the mean and the standard deviation.

[0042] Following the preprocessing 11, medical images 3 are converted to their semantical representations 9by a truncated deep neural network and reduced by average pooling in order to generate a multiple components vector, for example a 1024-components vector, which is then used for cosine similarity 10 analysis to provide a similarity ranking 15.

[0043] Figure 4 shows an overview to transform the similarity ranking 15 to a priority. In order to prioritise medical images 3 of different patients 2 to be analysed, the medical images 3 of those patients 2 are analysed by the priority algorithm 16.

[0044] Given the similarity analysis as described before, the preloaded medical images 6 present in the database 4 are ranked by similarity 15 after which the tool will select the top ten most similar cases 17 from the database 4.

[0045] In case of patient 2A of which a medical image 3A of a lung has been uploaded, the tool 1 will select the top ten most similar cases 17 related to lungs, not necessarily all related to the same disease but corresponding to the closest similar cases.

[0046] For each of these similar cases 17, the tool 1 will classify 18 the top ten most similar cases 17 according to their related diseases and count 19 the number of cases related to each disease within the top ten 17.

[0047] Following the above mentioned top ten most similar cases 17 for patient 2A, the tool can for example count 19 that five patients of these similar cases 17 were reported to be diagnosed with coronavirus disease 2019, three with another form of pneumonia and two were healthy.

[0048] The tool will then perform a priority rating 20 based on the number of cases for the most relevant disease weighted by their computed similarity for each image 3 to be analysed multiplied by a tunable coefficient K, to which the number of cases for other relevant diseases is added and divided by a tunable parameter Rmax.

[0049] To illustrate this further, in case of patient 2A coronavirus disease 2019 can be considered the most relevant disease based on the top ten most similar cases 17 in this instance. In the priority rating 20 the number of coronavirus disease 2019 cases is multiplied by a tunable coefficient K and added by the number of cases for other pneumonia and divided by a tunable parameter Rmax.

[0050] The outcome of priority algorithm 16 is a disease similarity index 21 for the most relevant disease, which in case of patient 2A could be a coronavirus disease 2019 similarity index.

[0051] The medical images 3 of different patients 2 to be analysed can then be ranked with the highest priority by means of disease similarity index 21.

[0052] Figure 5 shows a method to obtain a value for the tunable coefficient K and a value for the tunable parameter Rmax.

[0053] The tunable coefficient K is determined by an interactive iterative process wherein a subset of selected data preloaded 6 in the database 4 is chosen as a validation set 22A and wherein the remaining preloaded data 6 in the database 4 are used as a training set 22B, and wherein at the start a value 22C for the tunable coefficient K is chosen and then running the algorithm 16 to prioritise 220 the medical images 3 in the validation set 22A using data from the training set 22B and evaluating if the performance 22E of the prioritisation is acceptable, and in case the performance 22E is not acceptable to repeat the prioritisation algorithm 220 each time with a different value 22C for the coefficient K until an acceptable performance 22E is achieved.

[0054] When the performance 22E of the value 22C for the tunable coefficient K is acceptable, tunable parameter Rmax 23A is determined in such a way that the similarity index is limited to a value of about one hundred percent.

[0055] The tool 1 can also be used to prioritise patients 2 according to the plausible severity of a certain disease, in which case the patient would be examined by the doctor prior to less severe cases.

[0056] Furthermore, the tool 1 can be used to predict at which stage of progression of a certain disease a patient 2 is and can provide the worst and / or best similar medical images for the relevant disease, allowing healthcare practitioners to evaluate the effectiveness of the treatment of the particular patient and to how the disease could progress compared to best and worst case.

[0057] The current invention is by no means limited to the embodiments described as an example and shown in the drawings, but a tool to analyse medical images according to the invention can be realized in all kind of variants without departing from the scope of the invention as defined by the claims.

Claims

1. Tool (1) to analyse medical images (3) of patients (2), comprising a database (4) having preloaded medical images (6) of patients, each preloaded image having been diagnosed with a disease, and an input interface for loading a plurality of medical images (3) of different patients (2) to be analysed, wherein the tool (1) is provided with an algorithm (16) to prioritise patients to be analysed (2) of which the medical images (3) have been loaded according to the similarity between their medical images (3) to be analysed and the preloaded medical images (6) already provided in the database (4) for generating a disease similarity index (21), wherein the said disease similarity index (21) is used to rank and prioritise the handling of patients (2), the tool (1) being characterised in that the outcome of the algorithm (16) is a disease similarity index (21) for the most relevant disease, wherein, to obtain the most relevant disease for a medical image (3) to be analysed, the tool (1) ranks the preloaded medical images (6) and selects the top ten most similar cases (17) from the database (4) and, for each of these similar cases (17), the tool (1) classifies (18) the top ten most similar cases (17) according to their related diseases and counts (19) the number of cases related to each disease within the top ten, the most relevant disease being the disease with the highest count; and the tool (1) performs priority rating based on the number of cases for the most relevant disease weighted by their computed similarity for each image (3) to be analysed multiplied by a tunable coefficient K and by the number of cases for other relevant diseases and divided by a tunable parameter Rmax.

2. Tool (1) according to claim 1, wherein the tunable coefficient K is determined by an interactive iterative process wherein a subset of selected data preloaded 6 in the database 4 is chosen as a validation set 22A and wherein the remaining preloaded data 6 in the database 4 are used as a training set 22B, and wherein at the start a value 22C for the tunable coefficient K is chosen and then running the algorithm 16 to prioritise 220 the medical images 3 in the validation set 22A using data from the training set 22B and evaluating if the performance 22E of the prioritisation is acceptable, and in case in which the performance 22E is not acceptable to repeat the prioritisation algorithm 220 each time with a different value 22C for the coefficient K until an acceptable performance 22E is achieved.

3. Tool according to claim 2, wherein the tunable parameter Rmax 23A is determined, depending on the determined value 22C of the tunable coefficient K, in such a way that the similarity index (21) is limited to a value of one hundred percent.

4. Tool (1) according to claim 1, wherein the preloaded medical images (6) provided in the database (4) are present in the database (4) in the form of a semantical representation (9) and the medical images (3) of different patients (2) to be analysed are first transformed in to a semantical representation (9) in order to be compared with the semantical representations (9) of the preloaded medical images (6) in the database (4) by means of cosine similarity (10).

5. Tool (1) according to claim 4, wherein the semantical representations (9) of medical images (3) are achieved by a truncated deep neural network DenseNet 121 and reduced by average pooling, which generates a 1024-components vector.

6. Tool (1) according to any of the previous claims, wherein the tool (1) can be used to prioritise patients according to the plausible severity of a certain disease.

7. Tool (1) according to claim 1, wherein medical images (3) used by the tool (1) are preprocessed (11) by removal of annotations (12), resizing (13) the image to 1000x1000 pixels and normalization (14) by the mean and the standard deviation.

8. Tool (1) according to claim 1, wherein the input interface is suitable for loading medical images (3) from an external platform at distance.

9. Method to analyse medical images (3) of patients (2) suitable for use in a tool (1) according to any of the previous claims, the method making use of a database (4) with preloaded medical images (6) and an algorithm (16) to prioritise patients to be analysed (2); wherein the method is used to prioritise said medical images (3) of different patients (2) to be analysed and the method comprising the following steps: - loading the medical images (3) of the patients (2) to be prioritised in the tool (1); - comparing each of the loaded images (3) of different patients (2) with the preloaded medical images (6) to determine a disease similarity index (21); and, - ranking by highest priority each loaded image (3) of the patients (2) according to their disease similarity index (21), the method being characterised in that the outcome of algorithm (16) is a disease similarity index (21) for the most relevant disease wherein, to obtain the most relevant disease for a medical image (3) to be analysed, the tool (1) ranks the preloaded medical images (6) and selects the top ten most similar cases (17) from the database (4) and, for each of these similar cases (17), the tool (1) classifies (18) the top ten most similar cases (17) according to their related diseases and counts (19) the number of cases related to each disease within the top ten, the most relevant disease being the disease with the highest count; and the tool (1) is based on the number of cases for the most relevant disease weighted by their computed similarity for each image (3) to be analysed multiplied by a tunable coefficient K and by the number of cases for other relevant diseases and divided by a tunable parameter Rmax.

10. Method according to claim 9, wherein the step of comparing medical images (3) of different patients (2) with preloaded medical images (6) of a database (4) based on similarity comprises the following steps: - preloaded medical images (6) in the database (4) are transformed to a semantical representation (9) of each medical image (6); - medical images (3) of different patients (2) to be analysed are loaded into the tool (1); - medical images (3) of different patients (2) to be analysed are converted into a semantical representation (9); and, - the semantical representations (9) of the preloaded medical images (6) in the database (4) and the semantical representations (9) of the medical images (3) of different patients (2) to be analysed are compared by means of cosine similarity (10).

11. Method according to claim 10, wherein the step of converting medical images (3, 6) into semantical representations (9) comprises the following steps: - medical images (3, 6) are loaded into a database (4) or the tool (1); - medical images (3, 6) are elaborated by a truncated deep neural network DenseNet 121; and, - medical images (3, 6) are reduced by average pooling, which generates a 1024-components vector.