Patient-specific artificial neural network training system and method

Abstract

The invention relates to a training system and a training method for training artificial neural networks, in particular for medical applications, using patient-specific features. A patient-specific artificial neural network training system includes: an input interface configured to receive image data from a patient; the computing device further comprises a classification module, a separator module and a training module. And an output interface configured to output a diagnostic signal; wherein the classification module is configured to acquire the received image data as input and to generate a first classification signal for each image of the image data, and wherein the separator module is configured to separate the received image data into at least a first data set and a second data set according to a reliability criterion based on the first classification signal, and wherein the training module is configured to train the artificial neural network using the first data set, or only a portion thereof, as a training data set.

Description

Technical Field

[0001] The present invention relates to a training system and method for training artificial neural networks, preferably for medical applications, taking into account patient-specific characteristics. In particular, the present invention relates to improving the performance of artificial neural networks as classifiers and tools to assist diagnosis and making them more adaptable to the characteristics of each patient.

[0002] By way of example, the present invention will find application primarily in the fields of hematology and immunology, and in particular in the classification of leukocytes, but the principles of the invention have a much broader scope. Background Art

[0003] In the field of medical diagnosis, the introduction of computer-assisted applications and in particular artificial intelligence and machine learning has opened up a wide range of possibilities. In particular, computerized clinical decision support systems have continued to evolve since their introduction four decades ago. In the past few years, several countries have moved to HIMSS Level 6, and the use of electronic health records (EHR) has become the rule rather than the exception.

[0004] Artificial neural networks offer one of the most promising ways to assist in medical decision making. In samples where large amounts of training data are available, AI entities can compete with decisions made by unassisted humans.

[0005] Even more problematic are cases where the data is sparse. Again, for rare cases that are only sparsely covered by the training data, the number of outliers may be too small for the neural network to recognize them and extract meaningful information.

[0006] In medicine and biology, this can happen with markers for rare diseases, but it can also happen in the diagnosis of a specific patient. The data for a specific patient is sparse compared to the average data collected from a patient population. Deviations from the average are expected, and in some cases, can hamper diagnosis. For example, a patient may show low iron levels in the blood or low blood pressure, which are not indicative of disease. In a white blood cell differential, a patient may show, for example, abnormal basophils with lower granules than average, which is more of a feature than a marker of disease.

[0007] Before diagnosing a patient, a human expert will try to gather as much information about the patient as possible in order to accommodate patient-specific characteristics to the extent possible and arrive at a correct diagnosis. Incorporating this patient-specific knowledge into an artificial neural network is challenging, but at the same time, an important issue for the improvement of computerized assisted diagnosis systems. Summary of the invention

[0008] The object of the present invention is to provide a training system and method for training an artificial neural network for use as an auxiliary diagnosis tool in medical applications, with an emphasis on adapting the artificial neural network to patient characteristics in order to improve the accuracy of diagnosis.

[0009] The objects of the invention are achieved by an apparatus having the features disclosed in claim 1 and by a method having the features specified in claim 14. Preferred embodiments of the invention having advantageous features are given in the dependent claims.

[0010] A first aspect of the present invention provides a patient-specific artificial neural network training system, comprising: an input interface configured to receive image data from a patient; a computing device, further comprising a classification module, a separator module and a training module; and an output interface configured to output a diagnostic signal; wherein the classification module is configured to acquire received image data as input and generate a first classification signal for each graph of the acquired image data, wherein the separator module is configured to separate the received image data into at least a first data set and a second data set according to a first reliability criterion based on the first classification signal, and wherein the training module is configured to use the first data set or only a portion of the first data set as a training data set to train the artificial neural network.

[0011] Input interface and / or output interface are broadly understood as entities that can obtain image data and transmit it for further processing. Therefore, each of them can include at least a central processing unit, a CPU, and / or at least one graphics processing unit, a GPU, and / or at least one field programmable gate array, a FPGA, and / or at least one application-specific integrated circuit, an ASIC and / or any combination of the foregoing. Each of them can also include a working memory operably connected to at least one CPU and / or a non-transient memory operably connected to at least one CPU and / or a working memory. Each of them can be partially and / or completely implemented in a local device and / or such as partially and / or completely implemented in a remote system through a cloud computing platform.

[0012] The input interface and / or the output interface may be implemented in hardware and / or software, wired and / or wireless, and any combination thereof. Any of the interfaces may include an interface to an intranet or the Internet, to a cloud computing service, to a remote server, and / or the like.

[0013] Image data broadly describes any visual information that can be used for further processing and analysis in medical applications. In the context of blood cell classification, image data may consist of images of artificially stained blood samples (or: stained blood smears) using any of the staining methods used in blood cell classification. Some of these image data may include selected sub-image portions or crops of larger images, where segmentation has been applied and / or the crop contains the object to be classified together with some surrounding background. Hereinafter, image data, image datasets or images will be used indiscriminately to describe information in the form of images.

[0014] Likewise, the training data set in the present invention mainly refers to the training images. Hereinafter, the two terms will be used without distinction in meaning.

[0015] Whenever an artificial neural network is mentioned in the present invention, it should be understood as a computerized entity capable of implementing different data analysis methods broadly described under the terms artificial intelligence, machine learning, deep learning or computer learning. The artificial neural network can be a generative adversarial network (GAN), a convolutional neural network (CNN) or any other neural network. The artificial neural network to be trained will also be referred to as a refined classifier or a patient-specific artificial neural network.

[0016] A computing device should be broadly understood as an entity capable of processing data. The computing device can be implemented as any device, which includes at least a central processing unit CPU, and / or at least one graphics processing unit GPU, and / or at least one field programmable gate array FPGA, and / or at least one application-specific integrated circuit ASIC and / or any combination of the foregoing or is composed of them. It may also include a working memory operably connected to at least one CPU and / or a non-transient memory operably connected to at least one CPU and / or working memory. The computing device can execute software, apps or algorithms with different data processing capabilities. It can be implemented partially and / or completely in a local device and / or partially and / or completely in a remote system such as a cloud computing platform.

[0017] The diagnostic signal may include an indication that a particular disease or condition may be present or even exists in the patient.Once the patient-specific artificial neural network has analyzed the classified image, the diagnostic signal may include a list of possible or probable diseases associated with the classified image.

[0018] A second aspect of the present invention provides a computer-implemented patient-specific training method, preferably to be used with the patient-specific training system of the first aspect, comprising the following steps: (a) acquiring image data from a patient; (b) generating a first classification signal for each image of the acquired image data; (c) separating the image data into at least a first data set and a second data set based on the first classification signal according to a first reliability criterion; (d) initializing an artificial neural network; and (e) training the artificial neural network using the entire first data set or a portion of the first data set.

[0019] In particular, the method according to the second aspect of the present invention can be performed using the image forming apparatus according to the first aspect of the present invention. Therefore, the features and advantages disclosed in the present invention in combination with the image forming apparatus are also disclosed for the method, and vice versa.

[0020] According to a third aspect, the present invention provides a computer program product comprising executable program code configured to perform the method according to the second aspect of the present invention when executed.

[0021] According to a fourth aspect, the present invention provides a non-transitory computer-readable data storage medium comprising executable program code, which executable program code is configured to perform the method according to the second aspect of the present invention when executed.

[0022] The non-transitory computer-readable data storage medium may include or consist of any type of computer memory, in particular semiconductor memory (such as solid-state memory). The data storage medium may also include or consist of a CD, DVD, Blu-ray disc, USB memory stick, etc.

[0023] According to a fifth aspect, the present invention provides a data stream comprising an executable program code or being configured to generate an executable program code, the executable program code being configured to perform the method according to the second aspect of the present invention when executed.

[0024] One of the main ideas of the present invention is to provide a training system configured to train an artificial intelligence neural network for classification of medical image data, which is able to capture patient-specific features. This is achieved by processing a set of images from a patient by a classification module (or a pre-trained classifier) ​​and dividing the images into two groups based on the output. The first group includes images that have been identified with high confidence, and the second group includes images that cannot be identified with a good enough confidence. A training image dataset is generated from the first set of images, and a patient-specific artificial neural network is trained with the image dataset. The patient-specific artificial neural network is a refined classifier that contains more information about the patient than the pre-trained classifier.

[0025] The device as described above provides a simple implementation of a method for training an artificial neural network for medical image classification, which can be used as a subsequent diagnostic aid. One step consists of acquiring medical images, which in most cases have previously undergone a staining process. In another step, each image acquires a classification signal, which for example indicates the likelihood that the image belongs to different categories of the classifier. Based on the classification signal and a quantitative labeling criterion, the images can be separated into a first data set and a second data set. The (untrained) artificial neural network is initialized and trained with a portion of the first data set, which contains images that can be identified with higher certainty.

[0026] One advantage of the present invention is that by training an artificial neural network with the most indicative cells to perform refined classification, patient-specific features are acquired and learned by the artificial neural network. Thus, the classification accuracy and diagnostic accuracy of the patient-specific artificial neural network is higher than that which can be achieved with an artificial neural network that has been trained with data from many patients. This process of acquiring patient-specific features mimics how humans incorporate patient-specific characteristics into their diagnosis.

[0027] Another advantage of the invention is that the performance of the patient-specific artificial neural network can be easily tested by inputting the images of the second data set into the already trained artificial neural network. In this test run, the quality of the refined classifier can be evaluated by comparing its classification signal with the classification signal achieved with the pre-trained classifier. If the training has been successful, the reliability measure of the images of the second data set should increase, and in some cases one may even be able to recognize and mark all or part of the images of the second data set with a certain confidence.

[0028] Advantageous embodiments and further developments emerge from the dependent claims and from the description of various preferred embodiments illustrated in the drawings.

[0029] According to some of the embodiments, improvements or embodiment variants, the classification module comprises a pre-trained artificial neural network structure.

[0030] According to some of the embodiments, improvements or embodiment variants, the classification signal indicates a probability value associated with the first set of categories for each image. In other words, the artificial neural network can subsequently process the probability distribution associated with the image and assign a category label to the image.

[0031] According to some of the embodiments, improvements or embodiment variants, the first data set includes all images above a threshold probability value and the second data set includes all images at or below the same threshold probability value. The first data set contains very indicative cells of different categories in the first group of categories. Introducing an absolute probability threshold is a simple way to separate this first data set from less certain cases that will define the second data set. It is also possible and advantageous in some applications to divide the images into more than two data sets by, for example, setting different probability thresholds.

[0032] According to some of the embodiments, improvements or embodiment variants, the first data set includes all images above a relative threshold probability value between the categories, and the second data set includes all images at or below the same relative threshold probability value between the categories. In some cases, it is advantageous to define the reliability of the classification based on a relative probability metric. This reliability criterion emphasizes the distinguishability of one category relative to other categories.

[0033] According to some of the embodiments, improvements or variants, the training module further comprises a selector unit configured to determine a second set of categories and assemble a training data set based on information from the first data set and / or the second data set. Although the first set of categories is determined by the pre-trained artificial neural network of the classification unit, the second set of categories can be determined before the artificial neural network is trained. This depends on the result of the classification performed by the classification unit. For example, if the representation of the first data set only fills a subset of the categories in large quantities, it may be preferable to train the artificial neural network using only those fully represented categories. On the other hand, the second data set may indicate that the unclear situation is mainly caused by confusion between the two categories. In this case, it may be advantageous to focus the refined classification on these two categories because this is the most likely situation where the classification can be improved. The selection of a specific training data set (e.g., the proportion of images of each category in the selected category) is also a parameter that can be adjusted depending on the category distribution of the first data set. In a specific preferred embodiment, the second set of categories may be a subset of the first set of categories. In certain other embodiments, the second set of categories may contain categories that do not exist in the first set of categories.

[0034] According to some of the embodiments, improvements or embodiment variants, the selector unit further comprises a threshold distinction subunit configured to select a category in the first set of categories as a category in the second set of categories if the number of images classified in the category in the first set of categories is above a certain threshold value. In other words, the configuration of the second set of categories can be based on a quantitative criterion that selects only those categories that are statistically significant by defining a lower threshold of representation for each category.

[0035] According to some of the embodiments, improvements or variants, the selector unit also includes a sampling subunit, which is configured to sample a data subset from the first data set and / or amplify the first data set so that each category in the second set of categories has a statistically significant number of images in the training data set. Even if a threshold is introduced, some categories may be underrepresented relative to other selected categories. This may cause unwanted biases in the training of the artificial neural network. In order to avoid this possible bias, it is advantageous to balance the amount of representation of each category. Typically, the sampling subunit will randomly select representatives of over-represented categories to balance the distribution of training images according to the second set of categories. The sampling subunit can also apply amplification techniques to generate other training data for poorly represented or rare categories, for example by rotating, mirroring, flipping, scaling and / or resampling the image data. Typically, increasing the amount of qualified training data by amplification will result in better performance of the artificial neural network.

[0036] According to some of the embodiments, improvements or embodiment variants, the selector unit also includes an artificial intelligence subunit that is pre-trained to recognize a specific diagnosis and is configured to select training images based on a second reliability criterion. In some cases where a specific disease or abnormality is the target of patient image analysis, the second set of categories should be associated with markers of the specific disease or abnormality. In this case, using a pre-trained artificial neural network to identify the specific disease or abnormality can more efficiently select the training data set. Depending on the available data, the artificial neural network can be trained with general data or patient-specific data. The reliability criterion can be a probability distribution among different categories for which the artificial neural network has been pre-trained.

[0037] According to some of the embodiments, improvements or variants, the training module is also configured to generate a second classification signal associated with each image in the training data set, wherein the second classification signal indicates a probability value associated with the second set of categories for each image of the training data set.

[0038] According to some of the embodiments, improvements or embodiment variants, the selector unit further comprises a training scenario subunit of an artificial neural network configured to select a training module. The artificial neural network may be an artificial neural network based on a classification module or based on a different model. The training scenario subunit is capable of processing different models, the performance of which may be evaluated and compared later.

[0039] According to some of the embodiments, improvements or embodiment variants, the training scene subunit is further configured to add patient-specific non-image data to the training data set for training the artificial neural network of the training module. Thus, the artificial neural network training data set is augmented by patient-based metadata.

[0040] According to some of the embodiments, improvements or embodiment variants, the patient-specific non-image data comprises information about at least a previously diagnosed disease. This prior patient diagnosis selects the kind of algorithm to be used, in particular the kind of algorithm that is pre-trained to recognize a specifically diagnosed disease. Depending on the availability of data, the artificial neural network can be trained based on generic data or on data from the patient himself.

[0041] According to some of the embodiments, improvements or embodiment variants, the patient-specific training system also includes a quality control module, which is configured to input the image data of the second data set into the already trained artificial neural network of the training module and compare the second classification signal with the first classification signal for each image. In order to test the quality of the refined classification, one can use the artificial neural network of the training module with the second data set (i.e., the data set that is considered uncertain in the classification performed by the classification module). The probability distribution obtained by the pre-trained artificial neural network can then be compared with the probability distribution of the patient-specific artificial neural network. If the images of the second data set can be better recognized (i.e., their probability distribution now enables safe recognition or at least an improvement in recognition), successful training is achieved.

[0042] According to some of the embodiments, improvements or embodiment variants, the patient-specific training system further comprises a verification module configured to input the image data of the first data set into the already trained artificial neural network of the training module and to compare the second classification signal with the first classification signal for each image. The verification step may be defined by inputting images of the first data set that were not used for training into the already trained artificial neural network. The refined classifier should preferably correctly label the image with an increased confidence level.

[0043] Although here, above and below, some functions are described as being performed by modules, it should be understood that this does not necessarily mean that such modules are provided as separate entities from each other. In the case where one or more modules are provided as software, these modules can be implemented by program code segments or program code fragments that can be different from each other but can also be intertwined or integrated with each other.

[0044] Similarly, where one or more modules are provided as hardware, the functionality of one or more modules may be provided by the same hardware component, or the functionality of several modules may be distributed across several hardware components, which do not necessarily correspond to the modules. Alternatively and additionally, one or more of such modules may also be associated with a more specific computing structure, as it may be implemented using FPGA technology. Therefore, any device, system, method, etc. that exhibits all features and functions attributable to a particular module should be understood to include or implement the module. In particular, all modules may be implemented by program code executed by a computing device (e.g., a server or cloud computing platform).

[0045] According to some of the embodiments, the method of the second aspect further comprises the following steps: validating the artificial neural network using a portion of the image data of the first data set, wherein a second classification signal is generated for each image and the second classification signal is compared with the first classification signal of the same image, and evaluating the performance of the artificial neural network using the image data of the second data set, wherein a second classification signal is generated for each image and the second classification signal is compared with the first classification signal of the same image.

[0046] According to some of the embodiments, the patient-specific training system is part of a computer-aided clinical diagnosis embedded within a diagnostic decision support system (DDSS) to assist humans in making decisions in the clinical workflow.

[0047] The above embodiments and implementations may be combined with one another as desired, as long as the combination is reasonable.

[0048] Another scope of applicability of the present method and apparatus will become clear from the following drawings, detailed description and claims. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the present invention, are given by way of illustration only, as various changes and modifications within the spirit and scope of the present invention will become clear to those skilled in the art. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Aspects of the present disclosure will be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, but emphasis is placed on clearly illustrating the principles of the present disclosure. Parts corresponding to the same elements in different figures are marked in the figures with the same reference numerals.

[0050] Figure 1 is a schematic illustration of a patient-specific training system according to an embodiment of the present invention;

[0051] Figure 2 is a schematic depiction of the structure of a selector unit according to some embodiments of the present invention.

[0052] Figure 3 is a block diagram illustrating an exemplary implementation of a patient-specific training method;

[0053] FIG4 is an example of possible image processing according to some embodiments of the present invention;

[0054] Figure 5 is a schematic block diagram illustrating a computer program product according to an embodiment of the third aspect of the present invention; and

[0055] Figure 6 is a schematic block diagram illustrating a non-transitory computer-readable data storage medium according to an embodiment of the fourth aspect of the present invention.

[0056] These figures may not be to scale, and for the sake of clarity and simplicity, certain components may be shown in generalized or schematic form. In some instances, in order to avoid obscuring the concepts of the present invention, well-known structures and devices are shown in block diagram form. Similarly, the numbering of steps in the method is intended to simplify its description. They do not necessarily mean a certain ordering of the steps. In particular, several steps can be performed simultaneously.

[0057] The detailed description includes specific details for the purpose of providing a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. DETAILED DESCRIPTION

[0058] Figure 1 is a schematic illustration of a patient-specific training system 100 according to an embodiment of the present invention. For illustrative purposes, Figure 1 The training system is designed for training an artificial neural network for leukocyte classification. However, it should be understood that this specific example does not limit the scope of the present invention.

[0059] The training system 100 comprises an input interface 10, a computing device 20 and an output interface 30. The input interface 10 is broadly understood as any entity capable of acquiring image data D1 and transmitting it for further processing.

[0060] Computing device 20 should be broadly understood as any entity capable of processing data. Figure 1 The computing device 20 illustrated in FIG. 1 comprises a classification module 2, a separator module 3 and a training module 4. The classification module 2 is configured to take image data D1 as input and classify it according to some classification criteria. In the simplest version, the classification module 2 comprises a pre-trained artificial neural network. Figure 1In the specific application described in , the artificial neural network has been pre-trained to classify different types of white blood cells, for example, into basophils C1, monocytes C2, lymphocytes C3, blasts C4 and neutrophils C5. The classification module 2 outputs a first classification signal that attributes a probability distribution to each of the images D1, measuring the likelihood that the image D1 belongs to each of the categories C1-C5.

[0061] The separator module 3 is configured to interpret the probability distribution of each image D1 and to separate the images D1 into at least a first data set D2 and a second data set D3. The first data set D2 contains images D1 that can be attributed with certainty to one of the categories C1-C5, while the second data set D3 contains images D1 whose identification is considered uncertain.

[0062] According to some of the embodiments of the present invention, the criterion for marking an image as determined is determined by a probability threshold. For example, if the associated probability is higher than 70%, the image becomes a recognition label of the category. The probability threshold can be predetermined, for example it can be part of an existing classification protocol, or it can be determined a posteriori based on the obtained probability distribution. Using this criterion of an absolute probability threshold, a first image with a probability distribution of (75, 5, 2, 10, 8)) will be retained in the first data set D2 and classified as a basophil (with a probability of 75%), while a second image with a probability distribution of (60, 20, 5, 5, 10) will be retained in the second database D3.

[0063] According to some of the embodiments of the present invention, in some cases, relative probabilities may be preferred. For example, if the probability difference between the highest probability and the next probability exceeds 30%, the image is marked as definitely identified. Returning to the previous example, using this criterion based on relative probability, the first image with a probability distribution of (75, 5, 2, 10, 8) and the second image with a probability distribution of (60, 20, 5, 5, 10) will both be kept in the first data set D2. These two images will be classified as basophils (with probabilities of 75% and 60%, respectively).

[0064] The training module 4 is configured to input the first data set D2 or only a part of the first data set to train the artificial neural network 41. It comprises or consists of the artificial neural network 41 to be trained, a selector unit 42, a quality control module 43 and a verification module 44.

[0065] The artificial neural network 41 may be, for example, a generative adversarial neural network (GAN) or a convolutional neural network (CNN). Figure 1 By way of example an artificial neural network 41 with three classes C1 ' to C3' is shown. As discussed before, the classes C1 '-C3' are preferably subsets of the classes C1-C5 of the classification module 2.

[0066] The selector unit 42 is configured to generate a training data set D4 from the first data set D2. In some embodiments, it is also configured to select the artificial neural network 41 to be trained and the categories C1'-C3' for classification. In order to perform these functions, the selector unit 42 can be assigned a threshold distinction subunit 421 and / or a sampling subunit 422 and / or an artificial intelligence subunit 423 and / or a training scene subunit 424. Figure 2 The structure of the selector unit 42 is discussed in detail.

[0067] Once the artificial neural network 41 has been trained, a quality control module 43 and a validation module 44 may be employed. The quality control module 43 is configured to input images of the second data set D3 that have not yet been used for training into the already trained patient-specific artificial neural network 41. In order to evaluate the performance of the patient-specific artificial neural network 41 compared with the pre-trained classifier of the classification module 2, the quality control module 43 compares the first classification signal and the second classification signal for each of the images of the second data set D3. As already discussed above, these classification signals are reliability measures, and in some embodiments, they will be implemented as probability distributions of all available categories of the classifier.

[0068] The quality control module 43 compares these probability distributions. In this way, the extent to which the artificial neural network 41 has learned patient-specific features can be quantitatively determined. The probability distribution of the images in the second data set D3 from the second classified signal should be better than the probability distribution of the first classified signal. Ideally, the improvement is good enough so that the images of the second data set D3 that could not be identified with sufficient confidence by the separator module 3 can now be reclassified and identified with a certain confidence. This reclassification of the image D1 can also be done by the separator module 3 using the same probability thresholds adopted when processing the first classified signal or modifying them.

[0069] The validation module 44 is configured to input images of the first data set D2 (preferably those images that were not used for training, in order to avoid, for example, overfitting) into the already trained patient-specific artificial neural network 41. The first classification signal of each of the images is compared with the corresponding second classification signal in order to validate the artificial neural network 41. Since these images have already been classified with a high degree of confidence by the separator module 3 based on the first classification signal, the second classification signal will simply confirm the classification, possibly with a higher degree of confidence.

[0070] The output interface 30 is broadly understood as any entity capable of generating a diagnostic signal based on information obtained from the training module 4. Therefore, it may include at least a central processing unit CPU, and / or at least one graphics processing unit GPU, and / or at least one field programmable gate array FPGA, and / or at least one application-specific integrated circuit ASIC and / or any combination of the foregoing. It may also include a working memory operably connected to at least one CPU and / or a non-transient memory operably connected to at least one CPU and / or a working memory. It may be implemented partially and / or completely in a local device and / or partially and / or completely in a remote system such as a cloud computing platform.

[0071] If the training system 100 is to be implemented as part of a computer-aided clinical diagnosis embedded within a diagnostic decision support system (DDSS), the output interface 30 can present the diagnostic signal on a user interface (such as a website, application, software, or electronic health record (EHR) front-end interface).

[0072] Both the input interface 10 and / or the output interface 30 may be implemented in hardware and / or software, wired and / or wireless, and any combination thereof. Either of the interfaces 10, 30 may include an interface to an intranet or the Internet, to a cloud computing service, to a remote server, and / or the like.

[0073] Figure 2 Schematic depiction of the structure of the selector unit 42 according to some embodiments of the present invention is shown. The selector unit 42 is configured to decide which training data set D4 among the first data set D2 is to be used to train the artificial neural network ( Figure 1 In addition, the type of artificial neural network 41 to be used, the training algorithm, and the category of the image classification to be used by the artificial neural network 41 can also be determined ( Figure 1 Selection of C1'-C3').

[0074] Figure 2 A possible configuration of the selector unit 42 is shown, including a threshold distinction subunit 421 , a sampling subunit 422 , an artificial intelligence subunit 423 , and a training scenario subunit 424 .

[0075] The threshold distinction subunit 421 is configured to, if the classification is in the first group of categories ( Figure 1 If the number of images in a specific category in C1-C5 in the first group of categories is higher than a certain threshold, then a specific category in the first group of categories C1-C5 is selected as the second group of categories ( Figure 1The reduction of data from the received image D1 to the first data set D2 means that it may be statistically significant to make the second set of categories C1'-C3' smaller than the first set of categories C1-C5. The threshold applied by the threshold distinction subunit 421 is a measure of the statistical importance of the training desired for the categories in the second set of categories C1'-C3'. A less complex option is to simply keep a fixed number of categories (two, three or more) among the first set of categories C1-C5 when training the artificial neural network 41. Preferably, these categories are the categories with the highest number of representatives used for training.

[0076] The sampling subunit 422 is configured to sample a data subset from the first data set D2 and / or to augment the first data set D2 so that each of the second set of categories C1'-C3' has a statistically significant number of representatives. The sampling subunit 422 can be used in conjunction with the threshold distinction subunit 421 to ensure that there is no imbalance between the second set of categories C1'-C3' selected to train the artificial neural network 41. It is important to keep the number of training representatives for each category almost equal, or at least avoid that the number of training representatives at a particular category is under-represented or over-represented. The sampling subunit 422 can perform a random selection to cut representatives from the most represented categories, or it can simply hierarchically cut representatives of those categories starting from those with lower recognition certainty. Depending on the capabilities of the data, image augmentation can also be used to balance the training data set D4, with the additional benefit that it increases the total number of representatives, thereby improving the expected classification quality of the artificial neural network 41.

[0077] The artificial intelligence subunit 423 is configured to perform the selection of the second group of categories C1'-C3' based on a specific diagnosis. The artificial intelligence subunit 423 includes at least an artificial neural network (not shown) pre-trained with data to identify a specific disease. Since in some of the embodiments, the training system 100 is intended to be part of a computer-aided diagnosis system, it may be determined to be advantageous to fix the second group of categories C1'-C3' by using a pre-trained artificial neural network. For example, one may want to use the training system 100 to confirm whether a patient who was diagnosed with blood cancer in the past has blood cancer again. The artificial intelligence subunit 423 can be trained with a blood sample of the patient when he / she was first diagnosed, and use the trained artificial neural network to determine the categories C1'-C3' of the artificial neural network 41. In this case, the amount of data available for training is obviously a limiting factor, but one can also supplement the patient's blood sample with blood samples of other patients diagnosed with the same type of blood cancer.

[0078] The training scenario subunit 424 is configured to select the model and algorithm of the artificial neural network 41. Since it is not always clear which artificial neural network 41 will perform better for a given application, one of the advantages brought by this operation is that different artificial neural network structures can be trained. The training scenario subunit 424 can then be combined with the quality control module 43 in order to determine which artificial neural network 41 is best suited for the application at hand.

[0079] The training scene subunit 424 is also configured to add patient-specific non-image data to the data used for training the artificial neural network of the training module 4. The use of metadata in some artificial neural networks 41 may certainly improve their performance, for example if the metadata contains information about the patient's previous conditions or diseases.

[0080] Figure 2 All of these subunits depicted in the description may be operated in combination as long as the corresponding operations do not cause contradictions or conflicts.

[0081] Figure 3 is a diagram showing that it is preferred to Figure 1 Block diagram of an exemplary embodiment of a training method M applied together with the training system 100 described in . A step M1 comprises acquiring an image dataset D1 of a patient, which in a preferred embodiment of leukocyte classification has in most cases previously undergone a staining and segmentation process. In another step M2, a first classification signal is generated for each of the acquired images D1. In most of the preferred embodiments, this is achieved by inputting the images D1 into a pretrained artificial neural network, which outputs a probability distribution for each image as a first classification signal. In a further step M3, based on the first classification signal, the image dataset D1 is divided into at least a first dataset D2 and a second dataset D3. The separation is performed using some quantitative criteria. As has been described above Figure 1 How this can be done is discussed in the description of . In a further step M4, the artificial neural network 41 is initialized and then trained in a further step M5 using the first data set D2 or a part thereof as a training data set D4. Figure 1 and Figure 2 How to define subsets in the first data set D2 is discussed in detail in the description of .

[0082] Once the artificial neural network 41 has been trained, it can be verified in a further step M6. In this step M6, a part of the images of the first data set D1 that was not used in the training data set D4 will be used. A second classification signal is generated for each image and the second classification signal is compared with the first classification signal of the same image. In the most preferred embodiment, this amounts to a comparison of two probability distributions. In a further step M7, the performance of the artificial neural network 41 is evaluated by using the images of the second data set D3. A second classification signal is generated for each image and the second classification signal is compared with the first classification signal of the same image. In the most preferred embodiment, this amounts to a comparison of two probability distributions.

[0083] FIG. 4 shows an example of possible image processing according to some embodiments of the invention. Figure 4A Six stained images N1-N6 are shown as examples of image data D1 received by the input interface 10. They show a normal monocyte N1, a dysplastic monocyte N2, a dysplastic metamyelocyte N3, a normal segmented neutrophil N4, and two dysplastic myelocyte cells N5, N6. These cells are from a patient with myelodysplastic syndrome (MDS), a blood cancer in which myelocytes, which are precursors of neutrophils, do not mature normally.

[0084] The images N1-N6 are processed by a classification module 2, which generates classifications preferably according to a set of categories ( Figure 1 The first classification signal is in the form of a probability distribution of each of the images N1-N6 (C1-C5 in FIG. 1 ). For simplicity, one can limit the categories to two types of white blood cells, such as monocytes C2 and neutrophils C5. In this case, where the purpose is to determine whether a patient has MDS, the pre-trained artificial neural network is preferably pre-trained with samples diagnosed with MDS.

[0085] Figure 4B The segmentation of the image performed by the separator module 3 is shown. Once the reliability criteria are applied, the probability distribution of each of the images N1-N6 is labeled. As discussed previously, according to some preferred embodiments, a good criterion is to apply a probability threshold. Figure 4B In the example of , after applying the threshold, images N4, N5 and N6 are characterized as neutrophils C5, while image N1 is identified as monocytes C2. Therefore, the first data set D2 includes images N1, N4, N5 and N6. In contrast, images N2 and N3 are uncertain and belong to the second data set D3.

[0086] According to the principles of the present invention, the classification can be refined by using the images N1, N4, N5 and N6 of the first data set D2 or parts thereof to train the artificial neural network 41 (see Figure 1 ). Figure 4B An example of a training data set D4 is shown, which includes N1, N4 and N6. This can be the result of applying the threshold distinction subunit 421 or the sampling subunit 422 of the selector unit 42. In the first case, N5 will be below a certain probability threshold. In other words, N5 will be an image with lower certainty inside the first data set D2 and will be excluded from the training data set D4. In the second case, the sampling subunit 422 will be used to reduce the ratio of 3:1 myelocytes to monocytes to a 2:1 ratio. Then, the exclusion of N5 can be a random process, or N5 can be specifically selected, for example because it is the least certain among the myelocytes.

[0087] Figure 4C It is shown that the artificial neural network 41 has been used Figure 4B The training dataset D4 is shown in the stage of training. Figure 4C Yes Figure 4B The expected illustration of a successful refinement of the classification when the images N2 and N3 in the second data set D3 are input to the artificial neural network 41 is shown. This operation is managed by the quality control module 43, which generates a second classification signal, preferably in the form of a probability distribution of the images N2 and N3, indicating how likely it is that the images N2 and N3 are myelocytes C5 or monocytes C2. We have already discussed above that the separator unit 3 determines whether the images can be identified (i.e. they are certain according to the selected reliability criteria). In Figure 4C In the example, N3 is identified as myelocyte C5, while N2 is labeled as monocyte N2.

[0088] Image N2 cannot be recognized using a pre-trained artificial neural network that has been trained to recognize generic monocytes. Instead, the refinement training adds patient-specific features that are learned by the artificial neural network 41. As a result, when trained using the patient's white blood cells, N2 is found to be similar to the patient's monocytes and can be correctly labeled. The end result is a trained artificial neural network 41 that is customized for the patient and can therefore provide more reliable information for aiding the diagnosis of MDS (in this particular example).

[0089] Figure 5A schematic block diagram illustrating a computer program product 300 according to an embodiment of the third aspect of the invention is shown. The computer program product 300 comprises an executable program code 350 which, when executed, is configured to perform a method according to any embodiment of the second aspect of the invention, in particular as already described with respect to the preceding figures.

[0090] Figure 6 A schematic block diagram illustrating a non-transitory computer-readable data storage medium 400 according to an embodiment of the fourth aspect of the present invention is shown. The data storage medium 400 comprises an executable program code 450 which, when executed, is configured to perform a method according to any embodiment of the second aspect of the present invention, in particular as has been described with respect to the preceding figures.

[0091] The non-transitory computer-readable data storage medium may include or consist of any type of computer memory, in particular semiconductor memory (such as solid-state memory). The data storage medium may also include or consist of a CD, DVD, Blu-ray disc, USB memory stick, etc.

[0092] The previous description of the disclosed embodiments is merely an example of possible implementations, and these examples are provided to enable any person skilled in the art to make or use the present invention. It will be readily apparent to those skilled in the art that various changes and modifications to these embodiments will occur, and the general principles defined in the present invention may be applied to other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present invention is not intended to be limited to the embodiments shown in the present invention, but should conform to the widest scope consistent with the principles and novel features disclosed in the present invention. Therefore, the present invention is not limited except according to the appended claims.

[0093] Reference numerals list

[0094] 2 Classification Module

[0095] 3 separator modules

[0096] 4 Training Modules

[0097] 10 Input interface

[0098] 20 Computing Devices

[0099] 30 Output interface

[0100] 41 Artificial Neural Networks

[0101] 42 Selector Unit

[0102] 43 Quality Control Module

[0103] 44 Verification Module

[0104] 100-patient-specific artificial neural network training system

[0105] 300 Computer program products

[0106] 350 executable code

[0107] 400 Non-transitory computer readable data storage medium

[0108] 421 Threshold Distinguishing Subunits

[0109] 422 Sampling subunit

[0110] 423 Artificial Intelligence Subunit

[0111] 424 Training Scenario Subunit

[0112] 450 executable code

[0113] D1 Image Data

[0114] D2 First dataset

[0115] D3 Second Dataset

[0116] D4 training dataset

[0117] N1 Normal monocytes

[0118] N2 Dysplastic mononuclear cells

[0119] N3 Dysplastic metamyelocytes

[0120] N4 Normal segmented neutrophils

[0121] N5 Dysplastic myelocytes

[0122] N6 Dysplastic myelocytes

[0123] C1-C5 Group 1 Category

[0124] C1'-C3'Second group category

[0125] M Computer-implemented patient-specific training method

[0126] M1-M7 method steps.

Claims

1. A patient-specific artificial neural network training system (100), comprising: An input interface (10) configured to receive image data (D1) from a patient; The computing device (20) further comprises a classification module (2), a separator module (3) and a training module (4); and An output interface (30) configured to output a diagnostic signal; The classification module (2) is configured to take the received image data (D1) as input and generate a first classification signal for each image of the image data (D1), wherein the separator module (3) is configured to separate the received image data (D1) into at least a first data set (D2) and a second data set (D3) according to a first reliability criterion based on the first classification signal, and The training module (4) is configured to use the first data set (D2) or only a part of the first data set as a training data set (D4) to train an artificial neural network (41).

2. The patient-specific training system (100) according to claim 1, wherein: The classification module (2) comprises a pre-trained artificial neural network structure.

3. The patient-specific training system (100) according to claims 1 to 2, wherein: The first classification signal indicates, for each image, a probability value associated with a first set of classes (C1-C5).

4. The patient-specific training system (100) according to claims 1 to 3, wherein: The first data set (D2) includes all images above a threshold probability value, and the second data set (D3) includes all images at or below the same threshold probability value.

5. The patient-specific training system (100) according to claims 1 to 3, wherein: The first data set (D2) includes all images between categories (C1-C5) that are above a relative threshold probability value, and the second data set (D3) includes all images between categories (C1-C5) that are at or below the same relative threshold probability value.

6. The patient-specific training system (100) according to claims 1 to 5, wherein: The training module (4) also includes a selector unit (42) configured to determine a second set of categories (C1'-C3') and assemble the training data set (D4) based on information from the first data set (D2) and / or the second data set (D3).

7. The patient-specific training system (100) according to any one of the preceding claims, wherein: The selector unit (42) also includes a threshold distinction subunit (421), which is configured to select a category in the first group of categories (C1-C5) as a category in the second group of categories (C1'-C3') if the number of images (D1) classified in the category of the first group of categories (C1-C5) is higher than a specific threshold value.

8. The patient-specific training system (100) according to any one of the preceding claims, wherein: The selector unit (42) also includes a sampling subunit (422), which is configured to sample a subset of data from the first data set (D2) and / or augment the first data set (D2) so that each category in the second group of categories (C1'-C3') has a statistically significant number of images in the training data set (D4).

9. The patient-specific training system (100) according to any one of the preceding claims, wherein: The selector unit (42) also includes an artificial intelligence sub-unit (423) pre-trained to recognize a specific diagnosis, configured to select the training data set (D4) based on a second reliability criterion.

10. The patient-specific training system (100) according to any one of the preceding claims, wherein: The training module (4) is also configured to generate a second classification signal associated with each image of the training data set (D4), wherein the second classification signal indicates a probability value associated with the second set of categories (C1'-C3') for each image of the training data set (D4).

11. The patient-specific training system (100) according to any one of the preceding claims, wherein: The selector unit (42) further comprises a training scenario sub-unit (424) configured to select the artificial neural network (41) of the training module (4).

12. The patient-specific training system (100) according to claim 11, wherein: The training scene subunit (424) is also configured to add patient-specific non-image data to the training data set (D4) used to train the artificial neural network (41) of the training module (4).

13. The patient-specific training system (100) according to claim 12, wherein: The patient-specific non-image data comprises at least information of a previously diagnosed disease.

14. A computer-implemented patient-specific training method (M), preferably to be used together with the patient-specific training system (100) according to the preceding claim, the method comprising the following steps: acquiring (M1) image data (D1) from a patient; generating (M2) a first classification signal for each image of the acquired image data (D1); separating (M3) the image data (D1) into at least a first data set (D2) and a second data set (D3) based on the first classification signal and according to a first reliability criterion; Initialize (M4) the artificial neural network (41); as well as The artificial neural network (41) is trained (M5) using the entire first data set (D2) or a portion of the first data set.

15. A diagnostic decision support system comprising a patient-specific training system (100) according to claims 1 to 13.

Images