PRESENTATIONS OF THE PRESENCE OF ACUTE RESPIRATORY DIFFICULTY SYNDROME

Abstract

A mechanism for predicting acute respiratory distress syndrome (ARDS) in an individual. Individual data are processed using different machine learning techniques to generate a variety of feature vectors. These feature vectors are then merged using a fusion algorithm to produce a merged feature vector. The merged feature vector is further processed using another machine learning technique to generate a predictive indicator for ARDS. The fusion algorithm is trained using at least one loss function that is independent of the performance of the machine learning algorithm that generates the predictive indicator from the merged feature vector.

Description

AREA OF INVENTION

[0001] The present invention relates to the field of generating predictive indicators. BACKGROUND OF THE INVENTION

[0002] Acute respiratory distress syndrome (ARDS) is a severe form of acute lung injury (ALI) that occurs when fluid accumulates in an individual's lungs, depriving the organs of the oxygen they need to function. Therefore, there is a need to generate precise predictive indicators for the onset of ARDS to prompt or alert a physician to check for its presence.

[0003] One of the most common approaches to manually checking for the presence of ARDS is to use the Berlin definition, which defines a number of parameters that characterize or indicate the occurrence of ARDS.

[0004] Existing automated approaches typically use only non-imaging or non-photon data, such as vital signs or laboratory test results. However, medical imaging data (e.g., chest X-ray data) is known to play a crucial role in accurately identifying potential ARDS, as such images can provide clues to the etiology and chronicity of the disease. Medical imaging data can also play a role in monitoring ARDS and identifying potential clinical complications.

[0005] There is therefore a clear need for a procedure to generate a predictive indicator for ARDS that considers both medical imaging data and non-medical data. In particular, it would be desirable to provide a precise and energy-efficient approach for generating such a predictive indicator for ARDS. BRIEF SUMMARY OF THE INVENTION

[0006] The invention is defined by the claims.

[0007] According to examples according to one aspect of the invention, a computer-implemented method for generating a predictive indicator for acute respiratory distress syndrome for an individual is provided, wherein the predictive indicator is used in predicting the course of the acute respiratory distress syndrome.

[0008] The computer-implemented procedure comprises: obtaining individual data of the individual, including: medical image data of the individual and medical non-image data of the individual; processing the medical image data using a first machine learning procedure to generate a first feature vector; processing the non-image data of the individual using a second machine learning procedure to generate a second feature vector, wherein the second feature vector comprises static information of the individual, which is information that does not change significantly during the course of medical treatment; merging the first feature vector and the second feature vector using a fusion algorithm to form a merged feature vector that represents the information of both the first feature vector and at least the static information of the second feature vector;and processing the merged feature vector using a third machine learning algorithm to generate a predictive indicator that specifies whether the individual has acute respiratory distress syndrome or not.

[0009] The fusion algorithm is a trainable algorithm and is trained using at least one loss measure generated by at least one loss function, wherein each loss function processes a plurality of training-merged feature vectors, independently of the third machine learning algorithm, to generate a respective loss measure, and wherein the respective loss measure represents a difference between a predetermined set of feature vectors merged by the loss function.

[0010] One advantage of merging feature vectors, including at least the static information, is the ability to use different types of machine learning methods, which can be optimized for the specific type of information (improving the characterization of different information types within a single feature vector). Furthermore, by selecting different loss functions for training the fusion algorithm, the precision of the predictive indicator can be ensured independently of the fusion algorithm. Thus, any changes to the fusion algorithm (e.g., during training) will not affect any of the loss measures generated by any of the loss functions proposed herein.

[0011] The predictive indicator can take the form of binary, categorical, or numerical data. For example, the predictive indicator could be a binary predictor of whether or not an individual has acute respiratory distress syndrome (ARDS). Alternatively, the predictive indicator could categorize the probability of an individual having ARDS (e.g., "Not likely," "Less likely than unlikely," "As likely as unlikely," "More likely than unlikely," "Probably likely," etc.). In yet another example, the predictive indicator could be a predicted probability that an individual has ARDS (e.g., on a scale of 0 to 1, 0 to 10, 0 to 100, 1 to 10, 1 to 100, and so on).

[0012] In the context of this disclosure, medical data means all data relevant to making a medical diagnosis or assessment of the individual. This may include demographic data (e.g., age, gender, etc.), patient history information (e.g., length of stay, previous medical treatments and / or diagnoses, etc.), and / or physiological information (e.g., vital sign information such as heart rate, respiratory rate, etc.).

[0013] The fusion algorithm can include a neural autoencoder network.

[0014] In some examples, each training-related feature vector is associated with a respective individual and a respective predictive indicator that indicates whether the respective individual has acute respiratory distress syndrome or not.

[0015] In some examples, the at least one loss function includes a loss function that: for each training-merged feature vector in a first subset of the plurality of training-merged feature vectors, determines a first measure of the difference between the training-merged feature vector and another training-merged feature vector associated with an identical or similar predictive indicator compared to the predictive indicator of the merged feature vector; and sums and averages the determined first measures of difference to produce a first loss measure.

[0016] The fusion algorithm can be trained to reduce the initial loss measure.

[0017] The at least one loss function may include a second loss function which: for each training-merged feature vector in a second subset of the plurality of training-merged feature vectors, determines a second measure of the difference between the training-merged feature vector and another training-merged feature vector associated with a different predictive indicator compared to the predictive indicator of the merged feature vector; and sums and averages the determined second measures of difference to produce a second loss measure.

[0018] In some examples, the fusion algorithm is trained to increase the second loss measure.

[0019] In some examples, the multitude of training-merged feature vectors is generated by, for each of a first multitude of instances of training individual data, comprising medical training image data and medical non-training image data: processing the medical training image data using the first machine learning procedure to generate a first training feature vector; processing the medical non-training image data using the second machine learning procedure to generate a second training feature vector; and merging the first training feature vector and the second training feature vector using the fusion algorithm to generate the training-merged feature vector.

[0020] The at least one loss function can include a third loss function which: for each training-merged feature vector in a third subset of the plurality of training-merged feature vectors: processes the training-merged feature vector using a reconstruction algorithm to generate a reconstructed first feature vector and a reconstructed second feature vector;a third measure of the difference between the reconstructed first feature vector and the first training feature vector that was merged to produce the training-merged feature vector, and a fourth measure of the difference between the reconstructed second feature vector and the second training feature vector that was merged to produce the training-merged feature vector, and summing and / or averaging each third measure of difference and / or fourth measure of difference to produce a third loss measure.

[0021] The fusion algorithm can be trained to reduce the third loss measure.

[0022] In some embodiments, the fusion algorithm is trained using at least one additional loss measure generated by another loss function;wherein the further loss function is configured to process a plurality of instances of training data, each instance of training data comprising: training individual data, comprising medical training image data and medical non-training image data of a training subject, and a predictive training indicator indicating whether or not the training individual has acute respiratory distress syndrome, wherein the further loss function comprises: for each instance of training data: processing the medical training image data and the medical non-training image data using the first machine learning procedure, the second machine learning procedure, the fusion algorithm, and the third machine learning procedure to generate a predictive intermediate indicator;Determine a fifth measure of the difference between the predictive intermediate indicator and the predictive training indicator, and sum and / or average each fifth measure of the difference to generate another loss measure.

[0023] The procedure may further include processing the individual's non-image data using a fourth machine learning procedure to generate a third feature vector, wherein the merging of the first feature vector and the second feature vector includes merging the first feature vector, the second feature vector, and the third feature vector using a fusion algorithm to form a merged feature vector to represent the information of the first feature vector, the second feature vector, and the third feature vector; the second feature vector represents static information of the individual, which is information that does not change significantly during the course of medical treatment; and the third feature vector represents dynamic information of the individual, which is information that is expected to change during the course of medical treatment.

[0024] The first machine learning algorithm, the second machine learning algorithm and / or the third machine learning algorithm can be trained using the at least one loss measure generated by the at least one loss function.

[0025] A computer-implemented method for training a fusion algorithm for use in a previously described procedure is also proposed. The computer-implemented method comprises training the fusion algorithm using at least one loss measure generated by at least one loss function that processes a variety of training-induced merged feature vectors independently of the third machine learning algorithm.

[0026] Furthermore, a computer program product is proposed which comprises computer program code means which, when executed on a computing device with a processing system, cause the processing system to perform all the steps of any method disclosed herein.

[0027] A processing system is also provided to generate a predictive indicator for acute respiratory distress syndrome for an individual, with the predictive indicator being used in predicting the course of acute respiratory distress syndrome.

[0028] The processing system is configured to: receive individual data, comprising: the individual's medical image data and the individual's medical non-image data; process the medical image data using a first machine learning procedure to generate a first feature vector; process the individual's non-image data using a second machine learning procedure to generate a second feature vector, the second feature vector comprising static information about the individual, which is information that does not change significantly during the course of medical treatment; merge the first feature vector and the second feature vector using a fusion algorithm to form a merged feature vector representing the information of the first feature vector and at least the static information of the second feature vector;and processing the merged feature vector using a third machine learning algorithm to generate a predictive indicator that indicates whether or not the individual has acute respiratory distress syndrome.

[0029] The fusion algorithm is a trainable algorithm and is trained using at least one loss measure generated by at least one loss function, wherein each loss function processes a plurality of training-merged feature vectors, independently of the third machine learning algorithm, to generate a respective loss measure, and wherein the respective loss measure represents a difference between a predetermined set of feature vectors merged by the loss function.

[0030] The processing system can be suitably adapted to perform the function(s) of any method disclosed herein, and vice versa.

[0031] These and other aspects of the invention will become apparent from the embodiment(s) described below and will be explained with reference to them. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] For a better understanding of the invention and to show more clearly how it can be implemented in practice, reference is now made only by way of example to the attached drawings, in which: Fig. 1 illustrates a suggested workflow; Fig. 2 illustrates a fusion algorithm; Fig. 3 illustrates a fusion algorithm and a reconstruction algorithm and Fig. 4 is a flowchart that illustrates a proposed procedure. DETAILED DESCRIPTION OF THE EXECUTION FORMS

[0033] The invention is described with reference to the figures.

[0034] It is understood that the detailed description and specific examples, while providing exemplary embodiments of the device, systems, and methods, are intended for illustrative purposes only and are not meant to limit the scope of protection of the invention. These and other features, aspects, and advantages of the device, systems, and methods of the present invention are better understood from the following description, the accompanying claims, and the accompanying drawings. It is understood that the figures are merely schematic and not drawn to scale. It is also understood that the same reference numerals are used in all figures to indicate the same or similar parts.

[0035] The invention provides a mechanism for predicting acute respiratory distress syndrome (ARDS) in an individual. Individual data is processed using various machine learning methods to generate a variety of feature vectors. These feature vectors are then merged using a fusion algorithm to produce a merged feature vector. This merged feature vector is further processed using another machine learning method to generate a predictive indicator for ARDS. The fusion algorithm is trained using at least one loss function that is independent of the performance of the machine learning algorithm that generates the predictive indicator from the merged feature vector.

[0036] Implementations are based on the understanding that improving the performance of the fusion algorithm, e.g., so that the merged feature vector contains the information from the individual data more precisely, improves the accuracy of ARDS prediction.

[0037] Fig. Figure 1 conceptually illustrates a workflow 100 for generating a predictive indicator 190 for acute respiratory distress syndrome (ARDS) for an individual. The workflow processes individual data 110 of the individual to generate the predictive indicator 190.

[0038] The individual data 110 of the individual includes medical image data 111 and non-medical image data 112.

[0039] Medical image data can include, for example, medical images containing a representation of at least one thorax of the individual. Medical image data can include, for example, ultrasound images, X-ray images, CT images, MRI images, PET images, and so on. Any modality or form of medical image data can be used in the proposed concepts.

[0040] These embodiments are particularly advantageous when the medical imaging data includes chest X-ray data or X-ray data of the individual. This is because X-ray data have been shown to provide reliable and consistent representations or indicators of ARDS. However, other forms of imaging data (e.g., of the chest or other respiratory elements) also contain indicators of ARDS.

[0041] Medical non-image data 112 can include, for example, static and / or dynamic information. Static information includes any data or information that is expected to remain unchanged during a hospital stay (e.g., demographic information). Dynamic information includes any data or information that is expected to change or vary during a hospital stay, such as physiological parameter values ​​(e.g., vital signs), results of one or more laboratory tests, and so on. An alternative term for dynamic information is, for example, time-series information, as it is information or data that can be associated with one or more timestamps.

[0042] Some examples of static information include: age, sex, ethnicity, medical history, length of stay. Some examples of dynamic information include measurements of one or more physiological parameters of the individual, such as those derived from laboratory tests (e.g., glucose, P / F ratio) and those derived from direct individual monitoring (e.g., heart rate, respiratory rate, oxygen saturation).

[0043] The medical image data for the individual also includes static information and may, for example, be stored in a Picture Archiving and Communication System (PACS) or obtained directly from the medical imaging device. The non-medical image data may, for example, be stored in one or more of the individual's medical records, such as in a clinical information system, from which it can be retrieved.

[0044] Workflow 100 includes processing the medical image data using a first machine learning method 120 to generate a first feature vector 125. Workflow 100 also includes processing the non-image medical data using a second machine learning algorithm 130 to generate a second feature vector 135. The second machine learning method is separate from and / or different from the first machine learning method.

[0045] The medical image data and / or medical non-image data can be pre-processed before being fed into their respective machine learning processes.

[0046] For example, histogram smoothing and / or Gaussian blurring can be used to preprocess the medical data beams to achieve an extended intensity range and moderately increased contrast for the first machine learning procedure.

[0047] As another example, non-image medical data can be preprocessed by performing one or more processes to: handle missing values; average values; remove outliers; and / or feature coding / normalization. Before the training phase of the multimodal model, it is possible to link each unique intensive care unit (ICU) admission with radiographic examinations.

[0048] The first and second machine learning methods can be any suitable machine learning methods capable of generating a feature vector (e.g., a reduced feature vector). For example, a convolutional neural network (CNN) can be used as the first machine learning algorithm to extract significant features from the image data. The second machine learning algorithm could, for example, include a neural network with fully connected layers or a long short-term memory (LSTM) network.

[0049] The workflow also includes merging the first feature vector and the second feature vector using a fusion algorithm 140. The fusion algorithm forms a merged feature vector 145 that represents the information from or contained in both the first feature vector 125 and the second feature vector. The merged feature vector 145 may, for example, be smaller than the combination of the first 125 and the second 135 feature vectors.

[0050] The fusion algorithm 140, for example, can be a trainable algorithm. Thus, the fusion algorithm itself can be a machine learning method. For example, the fusion algorithm can be or comprise a neural autoencoder network.

[0051] The workflow then processes the merged feature vector 145 using a third machine learning method 150 to generate the predictive indicator 190, which indicates whether or not the individual has acute respiratory distress syndrome. The predictive indicator can be in the form of a classification (e.g., a binary or categorical classification) or a probability. Thus, the predictive indicator can be formed from binary, categorical, or numerical data.

[0052] The third machine learning method could be, for example, a multi-layered perceptron or another feedforward neural network. However, other suitable machine learning methods could be used in alternative examples.

[0053] In some examples, the workflow 100 further includes processing the individual's non-image data using a fourth machine learning procedure 160 to generate a third feature vector 165. The fusion algorithm 140 can be configured to merge the first 125, second 135, and third 165 feature vectors to produce the merged feature vector 145. In this case, the merged feature vector 145 represents the information from the first, second, and third feature vectors.

[0054] The fourth machine learning method is separate from and / or different from the first and second machine learning methods (as well as from / to the third machine learning method).

[0055] In some examples, the second feature vector represents the individual's static information. As mentioned earlier, static information is information that does not change significantly during the course of medical treatment. The third feature vector can represent the individual's dynamic information. As mentioned earlier, dynamic information can include or be information that is expected to change or vary during the course of medical treatment.

[0056] This approach allows for the use of different forms or types of machine learning algorithms when processing different data types, in order to characterize or represent different data types in different feature vectors. For example, a feature vector representing static information can be generated using a fully convolutional network as the second machine learning method, which facilitates the recording or characterization of links between different static variables of the individual within a feature vector. As another example, a Long Short-Term Memory (LSTM) can be used as the fourth machine learning method to enhance the characterization or representation of time-dependent (i.e., dynamic) information in a feature vector.

[0057] Thus, the use of both the second and fourth machine learning methods can improve the characterization of individual information into feature vectors and therefore the overall precision of the workflow.

[0058] It should be noted that the first, second, and third machine learning algorithms, together with the fusion algorithm (and the fourth machine learning algorithm, if any), effectively represent submodules or subprocesses of a larger machine learning algorithm.

[0059] A machine learning algorithm is a self-learning algorithm that processes input data to generate or predict output data. In this context, for the larger / overall machine learning algorithm, the input data includes the individual's data, and the output data includes the predictive indicator (whether or not the individual has acute respiratory distress syndrome).

[0060] Suitable machine learning algorithms for use in the present invention (e.g., as any of the machine learning algorithms, including the fusion algorithm) are apparent to those skilled in the art. Examples of suitable machine learning algorithms include decision tree algorithms and artificial neural networks. Other machine learning algorithms, such as logistic regression, support vector machines, or naive Bayes models, are suitable alternatives. Embodiments are particularly advantageous if each machine learning algorithm is a neural network.

[0061] The structure of a neural network (sometimes called an artificial neural network) is inspired by the human brain. Neural networks consist of layers, with each layer comprising a multitude of neurons. Each neuron performs a mathematical operation. In particular, each neuron can comprise a differently weighted combination of a single conversion type (e.g., the same conversion type, sigmoid, etc., but with different weights). In the process of processing input data, each neuron's mathematical operation is performed on the input data to produce a numerical output, and the outputs of each layer in the neural network are fed sequentially to the next layer. The final layer provides the output.

[0062] Methods for training a machine learning algorithm are well-known. Typically, such methods involve obtaining a training dataset containing training input data entries and corresponding training output data entries. An initialized machine learning algorithm is applied to each input data entry to generate predicted output data entries. Any error between the predicted output data entries and the corresponding training output data entries is used to modify the machine learning algorithm. This process can be repeated until the error converges and the predicted output data entries are sufficiently similar to the training output data entries (e.g., ±1%). This is commonly known as supervised learning.

[0063] If the machine learning algorithm is formed from a neural network, for example, the (weightings of the) mathematical operations of the individual neurons can be modified until the error converges. Known methods for modifying a neural network include gradient descent, backpropagation algorithms, and so on.

[0064] Training the first, second, and third machine learning algorithms (and optionally the fusion algorithm and / or the fourth machine learning algorithm, if present) can be performed using a training dataset where the training input data entries correspond to exemplary instances of individual data, and the training output data entries provide a predictive (ground-truth) indicator for each instance of individual data. Each instance of individual data is processed (e.g., according to the workflow described earlier), and a predicted predictive indicator is generated for each instance. Any error between the predicted predictive indicator and the corresponding predictive (ground-truth) indicator is used to modify at least the first, second, and third (and, if present, the fourth) machine learning algorithms.The same error can also be used in some cases when training the fusion algorithm.

[0065] This technique follows established training procedures for machine learning algorithms, and other approaches are obvious to the expert.

[0066] This disclosure provides (e.g., additional) techniques for improving the performance of the fusion algorithm 140, such that the merged feature vector 145 more accurately represents the information from or contained in the first feature vector 125 and / or the second feature vector 135 (and / or the third feature vector 165, if any) and / or the individual. An improvement to the fusion algorithm enhances the precision of the predictive indicator 190 by reducing the risk of errors or by minimizing the introduction of any errors into the fusion process.

[0067] In particular, the present disclosure proposes an approach in which the fusion algorithm is trained using at least one loss measure generated by at least one loss function. Each loss function processes a plurality of training-merged feature vectors, independently of the third machine learning algorithm, to generate a respective loss measure.

[0068] In other words, the loss measure generated by any of the loss functions proposed herein (used to train the fusion algorithm) is independent of any predictive indicator generated by the third machine learning algorithm, which processes each merged vector produced by the fusion algorithm. Thus, any changes to the third machine learning algorithm (e.g., during training) have no effect on any of the loss measures generated by any of the loss functions proposed herein.

[0069] Fig.Figure 2 illustrates an example of a fusion algorithm 140 for improved contextual understanding. The fusion algorithm 140 is configured to merge a multitude of feature vectors 125, 135, 165 to form a merged feature vector 145. The merged feature vector 145 represents or contains information found in the multitude of feature vectors. In particular, the merged feature vector can be designed (through appropriate training) to include features that are important or necessary for accurate prediction of ARDS.

[0070] In the illustrated example, the multitude of feature vectors includes at least the first 125 and the second 135 feature vector previously described, and optionally the third feature vector 165.

[0071] The illustrated fusion algorithm is a neural autoencoder network. Thus, each feature vector 125, 135, 165 is fed as input into the fusion algorithm. The fusion algorithm then uses a series or sequence of connected layers to generate a (in some embodiments smaller) merged feature vector 145. In particular, each value in the merged feature vector 145 can respond, at least partially, to each value of every feature vector fed as input into the fusion algorithm. The neural autoencoder network is a type of neural network, examples of which have been described previously.

[0072] As previously mentioned, the proposed loss function(s) uses a multitude of training-merged feature vectors. This multitude of training-merged feature vectors can be generated from an initial multitude of instances of training individual data, comprising medical training image data and medical non-training image data.

[0073] For example, the multitude of training-merged Merlenal vectors can be generated by processing, for each instance of training individual data, the medical training image data using the first machine learning procedure to generate a first training feature vector; processing the medical non-training image data using the second machine learning procedure to generate a second training feature vector; and merging the first training feature vector and the second training feature vector using the fusion algorithm to generate the training-merged feature vector.

[0074] Of course, if a fourth machine learning method is to be used, then (for each instance of training individual data) the corresponding training-merged feature vector can be generated by further processing the non-training medical image data using the fourth machine learning method to generate a third training feature vector, and by adjusting the fusion step to include merging the first, second and third feature vectors to generate the training-merged feature vector.

[0075] A loss function processes at least a portion of the multitude of feature vectors merged during training to generate a loss measure. This loss measure can be used to update or modify the fusion algorithm, for example, to modify one or more weights or other parameter values ​​of the fusion algorithm.

[0076] Each time the fusion algorithm is updated, the multitude of feature vectors merged during training can be regenerated using the updated algorithm. The loss function(s) can then recalculate the loss measure(s) for this multitude of merged feature vectors. This allows changes to the fusion algorithm to propagate to the merged feature vectors, resulting in improved precision.

[0077] A first loss function for a fusion algorithm generates a first loss measure that penalizes differences between merged feature vectors of similar individuals (e.g., individuals with a similar or identical predictive indicator for ARDS). This first loss measure can be called a first contrastive loss L. kontrastiv1 be designated.

[0078] To generate an initial contrastive loss, each training-merged feature vector is associated with a specific individual and a corresponding predictive indicator that specifies whether or not that individual has acute respiratory distress syndrome. This association or correspondence can be reported in a dataset containing the multitude of training-merged feature vectors. The predictive indicator in this scenario can be a ground-truth indicator (e.g., defined or determined by an appropriately qualified physician).

[0079] The first loss function determines, for each training-merged feature vector in a first subset of the plurality of training-merged feature vectors, a first measure of the difference between the training-merged feature vector and another training-merged feature vector that is associated with the same or similar predictive indicator compared to the predictive indicator of the merged feature vector.

[0080] The precise criterion(s) for considering the predictive indicators of two training-merged feature vectors as similar may depend on the type or data format of the predictive indicator. For example, two training-merged feature vectors may be considered similar if a binary or categorical predictive indicator associated with each training-merged feature vector has the same value. Alternatively, two training-merged feature vectors may be considered similar if the difference between the numerical predictive indicator associated with each training-merged feature vector is less than a predetermined value or percentage (e.g., they differ by less than 10%).

[0081] The first loss function then sums or averages the determined first measures of difference to generate the first loss measure.

[0082] Equation (1) represents an approach for the first loss function, i.e., an approach to defining or determining the first loss measure L. kontrastiv1 . Lcontrastive1=∑imax(0,‖Ai−Ai+‖22) where A i represents the i-th merged feature vector of the first sub-proposition. The merged feature vector A i and the positive sample feature vector A i + They belong to the same class (e.g., they exhibit similar or the same predictive indicator(s)). The function ||·|| is any suitable loss or difference function, such as an L1 loss function or an L2 loss function.

[0083] The fusion algorithm can be used with the first loss measure L. kontrastiv1, e.g., to decrease or reduce the first loss measure, they can be trained. Approaches to training an algorithm have already been described, but usually involve modifying the value of one or more parameters of the algorithm (e.g., using a gradient descent approach).

[0084] A second loss function for a fusion algorithm generates a second loss measure that penalizes similarities between merged feature vectors from different individuals (e.g., individuals with dissimilar predictive indicators for ARDS). This second loss measure can be called the second contrastive loss L. kontrastiv2 be designated.

[0085] To generate a second contrastive loss, each training-merged feature vector remains associated with a specific individual and a specific predictive indicator indicating whether or not that individual has acute respiratory distress syndrome. Again, the predictive indicator in this scenario can be a ground-truth indicator (e.g., defined or determined by a suitably qualified physician).

[0086] The second loss function determines, for each training-merged feature vector in a second subset of the multitude of training-merged feature vectors, a second measure of the difference between the training-merged feature vector and another training-merged feature vector that is associated with a different or dissimilar predictive indicator compared to the predictive indicator of the merged feature vector.

[0087] The precise criterion(s) for considering the predictive indicators of two training-merged feature vectors as different or dissimilar may depend on the type or data format of the predictive indicator. For example, two training-merged feature vectors may be considered different or dissimilar if the binary or categorical predictive indicators associated with the training-merged feature vectors have different values. Alternatively, two training-merged feature vectors may be considered different or dissimilar if the difference between the numerical predictive indicator associated with each training-merged feature vector is greater than a predetermined value or percentage (e.g., they differ by less than 10%).

[0088] The second loss function then sums or averages the determined second measures of difference to obtain a second loss measure L. kontrastiv2 to produce.

[0089] Equation (2) represents an approach for the second loss function, i.e., an approach to defining or determining the second loss measure L. kontrastiv2 . Lcontrastive2=∑imax(0,‖Ai−Ai−‖22) where A i represents the i-th merged feature vector of the second sub-theorem. The merged feature vector A i and the positive sample feature vector A i - They belong to different classes (e.g., they exhibit dissimilar predictive indicators). The function ||·|| is any suitable loss or difference function, such as an L1 loss function or an L2 loss function.

[0090] The fusion algorithm can be used with the second loss measure L. kontrastiv2, e.g., to increase the second loss measure, can be trained. Approaches to training an algorithm have already been described, but usually involve modifying the value of one or more parameters of the algorithm (e.g., using a gradient descent approach).

[0091] Equation (3) represents a combined loss function that combines the previously described first and second loss functions to obtain a combined loss measure L kombiniert to produce. Lcombined=∑imax(0,‖Ai−Ai+‖22−‖Ai−Ai−‖22+m)

[0092] The boundary parameter m controls the distances between similar and dissimilar pairs. The combined loss L kombiniert The aim is to bring samples of the same class closer together and to move samples of different classes further apart based on these distances.

[0093] A third loss function aims to characterize, in the form of a third loss measure, the ability of a merged feature vector (generated by the fusion algorithm) to reconstruct the first, second, and (if present) third feature vectors. Thus, the third loss function aims to characterize how accurately or appropriately the merged feature vector represents the information contained in the first, second, and / or third feature vectors (e.g., whether information is lost). The use of such a loss function would improve the performance of the fusion algorithm.

[0094] The third loss measure can therefore be described as reconstruction loss L Rekonstruktion The term "reconstruction loss" is used to test or measure whether the merged feature vector has retained the distinctive features of the individual data.

[0095] The third loss measure can be generated by processing each training-merged feature vector in a third subset of the multitude of training-merged feature vectors (previously described).

[0096] In particular, determining the third loss measure may involve processing the training-merged feature vector using a reconstruction algorithm to generate a reconstructed first feature vector and a reconstructed second feature vector; determining a third measure of the difference between the reconstructed first feature vector and the first training feature vector that was merged to generate the training-merged feature vector; and determining a fourth measure of the difference between the reconstructed second feature vector and the second training feature vector that was merged to generate the training-merged feature vector, and summing or averaging each third measure of difference and / or fourth measure of difference to generate a third loss measure.

[0097] If the overall workflow employs a fourth machine learning method, the reconstruction algorithm can further generate a reconstructed fourth feature vector. Determining the third loss measure can involve determining a fifth measure of the difference between the reconstructed third feature vector and the third training feature vector, which was merged to generate the training-merged feature vector. Summing and / or averaging can involve averaging each of the third, fourth, and / or fifth measures of difference to generate the third loss measure.

[0098] Fig. Figure 3 illustrates an approach to generating the reconstructed feature vectors. In particular, it illustrates Fig.3. The fusion algorithm 140 is combined with the reconstruction algorithm 300. The reconstruction algorithm processes a merged feature vector 145 to generate the reconstructed feature vectors 325, 335, 365.

[0099] The reconstruction algorithm can effectively serve as the inverse of the fusion algorithm. Approaches to constructing such a reconstruction algorithm are apparent to a suitably qualified person. In particular, the reconstruction algorithm can be a trainable algorithm (e.g., trained using the third loss measure), such as a neural network that processes the merged feature vector to reconstruct the feature vectors that were originally used to generate the merged feature vector.

[0100] Equation (4) represents a reconstructive loss function that gives the third loss measure, i.e., the reconstruction loss L. Rekonstruktion -, generated. LReconstruction=1N∑i=1N(‖Iiv,I^iv‖+‖Tiv,T^iv‖+‖Siv,S^iv‖) where: IV represents the first feature vector 125, which is used when generating the i-th feature vector merged due to training; I^iv represents the reconstructed first feature vector 325 (generated using the i-th feature vector merged due to training); TV represents the second feature vector 135, which is used when generating the i-th feature vector merged due to training; T^iv represents the reconstructed second feature vector 335 (generated using the i-th feature vector merged due to training); Siv represents the third feature vector 165, which is used when generating the i-th feature vector merged due to training; S^iv represents the reconstructed third feature vector 365 (generated using the i-th feature vector merged due to training); N is the number of feature vectors merged due to training in the third subset of the plurality of feature vectors merged due to training, and the function ||·|| is any suitable loss or difference function, such as an L1 loss function or an L2 loss function.

[0101] In equation (4) the term ‖Siv,S^iv‖ in embodiments in which no third feature vector is generated.

[0102] The fusion algorithm can be trained to reduce the third loss measure.

[0103] The examples of loss functions described above are independent of the third machine learning algorithm.

[0104] However, the fusion algorithm can be (further) trained using one or more additional loss measures that utilize the third machine learning algorithm. An example of an additional loss measure is one used to generate the predictive indicator with a given first and second (and, if used, third) feature vector.

[0105] Thus, another loss function is also provided, configured to process a multitude of instances of training data, where each instance of training data includes training feature vectors (comprising a first, second and (if used) third feature vector) and a predictive training indicator that specifies whether the training individual associated with the training feature vectors has acute respiratory distress syndrome or not (which may be ground truth data).

[0106] The further loss function includes, for each instance of training data: processing the training feature vectors using the fusion algorithm to generate a merged intermediate feature vector; and processing the merged intermediate feature vector to generate a predictive intermediate indicator. The further loss function also includes, for each instance of training data, determining a fifth measure of the difference between the predictive intermediate indicator and the predictive training indicator. The further loss function also includes summing and / or averaging each of these fifth measures of difference to generate another loss measure.

[0107] This additional loss measure can be called classification loss L Klassifikation be designated.

[0108] Equation (5) provides an example of such a further loss function or classification loss function, which gives the further loss measure, i.e. a classification loss L Klassifikation , generated. LClassification=−1N∑i=1Np(yi|Ii,Ti,Si) where N represents the number of instances of training data, IV represents the first feature vector of the i-th instance of training data; TV represents the second feature vector of the i-th instance of training data; Siv represents the third feature vector of the i-th instance of training data (which can be omitted); y i The predictive indicator of the i-th instance of training data is represented. The function p() is any suitable probability or error function.

[0109] Some exemplary approaches for training a fusion algorithm use all the loss measures described above, for example, to create an overall loss measure for the fusion algorithm. In other words, some methods for training a fusion algorithm combine the loss measures described above (e.g., they perform a sum, weighted sum, average, or weighted average) of the described loss measure.

[0110] This total loss measure L insgesamt It consists of a classification loss, a contrastive loss, and a reconstruction loss, and provides a technique to utilize both interclass similarity and cross-modality information to address multimodal problems.

[0111] Equation (6) provides an example of an approach to generating a total loss measure L insgesamt dar: Total Lins = LClassification + α.LCombined + β.LReconstruction Factors α and β are used to regulate the influence of the intermodality task and the reconstruction task, respectively. The values ​​of factors α and β can be controlled, for example, based on the specific application scenario and / or user input.

[0112] Experiments were conducted to assess the effectiveness of using the proposed workflow (where at least the fusion model is trained using the total loss measure). The publicly available MIMIC CXR and MIMIC IV datasets were used to train and test three ARDS detection models. Both databases used the same patient IDs, stays, admissions, and the dates and times are consistent for each patient. The three ARDS detection models include medical imaging data (MID) classification using DenseNet121, medical non-imaging data (NIMD) classification using XGBoost, and using the proposed workflow to generate the predictive indicator.

[0113] Table 1 illustrates the effectiveness (AUROC) of each model along with the data types used by each model to generate the predictive indicator. TABLE 1 Model Data type AUROC XGBoost NIMD 0,68 DenseNet121 MID 0,84 Proposed approach MID + NIMD 0,88

[0114] The first machine learning algorithm, the second machine learning algorithm, and / or the third machine learning algorithm (and / or, if applicable, the fourth machine learning algorithm) can also be trained using the at least one loss measure generated by any loss function proposed herein. Thus, any loss function proposed herein can be integrated into a training procedure for the third, second, third, and / or (if applicable) fourth machine learning algorithm.

[0115] Fig. Figure 4 is a flowchart illustrating a computer-implemented procedure 400 that employs the approach proposed herein. Thus, the computer-implemented procedure is configured to generate a predictive indicator for acute respiratory distress syndrome for an individual.

[0116] The computer-implemented procedure 400 includes a step 410 of obtaining individual data of the individual. As previously explained, the individual data includes at least medical image data of the individual and medical non-image data of the individual.

[0117] The computer-implemented procedure 400 also includes a step 420 of processing the medical image data using an initial machine learning procedure to generate an initial feature vector.

[0118] The computer-implemented procedure also includes step 430 of processing the individual's non-image data using a second machine learning procedure to generate a second feature vector.

[0119] Optionally, the computer-implemented method includes a step 435 of processing the non-image data using a fourth machine learning method to generate a third feature vector. In such embodiments, the second feature vector can represent static information about the individual, information that does not change significantly during the course of medical treatment; and the third feature vector can represent dynamic information about the individual, information that is expected to change or vary during the course of medical treatment.

[0120] The computer-implemented procedure 400 also includes a step 440 of merging the first feature vector and the second feature vector using a fusion algorithm. The fusion algorithm forms a merged feature vector that represents the information of both the first and second feature vectors.

[0121] If step 435 is performed, step 440 may involve merging the first feature vector, the second feature vector, and the third feature vector using the fusion algorithm to form the merged feature vector to represent the information of the first feature vector, the second feature vector, and the third feature vector.

[0122] The computer-implemented procedure also includes step 450 of processing the merged feature vector using a third machine learning algorithm to generate a predictive indicator that specifies whether or not the individual has acute respiratory distress syndrome.

[0123] As previously explained, the fusion algorithm is a trainable algorithm and is trained using at least one loss measure generated by at least one loss function. Each loss function processes a variety of training-merged feature vectors, independently of the third machine learning algorithm, to generate its respective loss measure.

[0124] Procedure 400 may further include a step 460 of outputting the predictive indicator. Step 460 may, for example, include outputting the predictive indicator to another processing system or storage.

[0125] In some examples, step 460 involves controlling a user interface to provide a user-perceived output of the predictive indicator. This provides a user of the interface with an indicator of likely ARDS, thereby helping them to make a clinical prediction and decision about the individual (whose individual data were obtained in step 410).

[0126] Steps 410 to 450 can be repeated iteratively, for example, to iteratively update the individual data over time based on any changes. In such embodiments, step 460 may involve controlling a user interface to provide a user-perceived output of the predictive indicator along with historical indicators (e.g., predictive indicators generated in previous iterations of steps 410 to 450). This can advantageously allow a user (e.g., a physician) to view any changes in the predicted probability of ARDS in order to identify a potential deterioration in the individual's condition, for example, to initiate an intervention.

[0127] Other uses for a predictive indicator for ARDS will be apparent to a suitably qualified person and are not disclosed in detail herefor the sake of brevity. For example, the predictive indicator for ARDS can be provided as input for a further processing procedure to perform a further automated clinical analysis process.

[0128] Procedure 400 may further include a step 470 of controlling a user interface to provide a user-perceived output of a portion of the individual's data. This provides a user of the interface with information useful for confirming or verifying the precision of the indicator for probable ARDS. This assists the user in making a clinical decision about the individual (whose individual data were obtained in step 410).

[0129] A computer-implemented method for training a fusion algorithm for use in Procedure 400 is also proposed. The computer-implemented method involves training the fusion algorithm using at least one loss measure generated by at least one loss function that processes a variety of training-merged feature vectors independently of the third machine learning algorithm. Detailed approaches for training the fusion algorithm and examples of suitable loss measures have been previously described.

[0130] A person skilled in the art would readily be able to develop a processing system for carrying out the procedure described herein. Thus, each step of a flowchart can represent a different action performed by a processing system and can be carried out by a respective module of the processing system.

[0131] Implementations can therefore employ a processing system. The processing system can be implemented in numerous ways, using software and / or hardware to perform the various required functions. A processor is an example of a processing system that employs one or more microprocessors, which can be programmed using software (e.g., microcode) to perform the required functions. However, a processing system can be implemented with or without a processor and can also be implemented as a combination of dedicated hardware for performing some functions and a processor (e.g., one or more programmed microprocessors and associated switching logic) for performing other functions.

[0132] Examples of processing system components that may be used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs).

[0133] In various implementations, a processor or processing system can be associated with one or more storage media, such as volatile and non-volatile computer memory like RAM, PROM, EPROM, and EEPROM. The storage medium can be encoded with one or more programs that can perform the necessary functions when executed on one or more processors and / or processing systems. Various storage media can be fixed within a processor or processing system or be portable, allowing the one or more programs stored on them to be loaded into a processor or processing system.

[0134] A user interface system is also proposed, comprising any such processing system and a user interface communicatively coupled to the processing system. The processing system can be configured to control the operation of the user interface to provide one or more user-perceived outputs, such as the predictive indicator and / or historical predictive indicators.

[0135] It is understood that the disclosed methods are preferably computer-implemented methods. Therefore, the concept of a computer program is also proposed, which includes a code facility for implementing any described method when the program is executed on a processing system, such as a computer. Accordingly, different sections, lines, or blocks of code from a computer program according to one embodiment can be executed by a processing system or a computer to perform each method described herein.

[0136] A non-transient storage medium is also proposed, which stores or carries a computer program or computer code which, when executed by a processing system, causes the processing system to execute each procedure described herein.

[0137] In some alternative implementations, the functions shown in the block diagram(s) or flowchart(s) may be executed in a different order than the one shown in the figures. For example, two blocks shown consecutively may actually be executed essentially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the functionality involved.

[0138] Variations of the disclosed embodiments can be understood and implemented by those skilled in the art who implement the claimed invention by studying the drawings, the disclosure, and the accompanying claims. The mere fact that certain dimensions are mentioned in different dependent claims does not indicate that a combination of these dimensions cannot be used advantageously.

[0139] In the claims, the word "comprise" does not exclude further elements or steps, and the indefinite article "a," "an," or "a" does not exclude a plurality. When the term "adapted to" is used in the claims or description, it is indicated that the term "adapted to" is equivalent to the term "configured to." When the term "arrangement" is used in the claims or description, it is indicated that the term "arrangement" is equivalent to the term "system," and vice versa.

[0140] A single processor or other unit can perform the functions of several elements specified in the claims. Where a computer program has been discussed above, it can be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied with or as part of other hardware, but it can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0141] Any reference numerals in the claims shall not be interpreted as limitations of the scope of protection.

Claims

A computer-implemented method for generating a predictive indicator for acute respiratory distress syndrome (ARDS) for an individual, wherein the predictive indicator is used in predicting the course of ARDS, the computer-implemented method comprising: obtaining individual data, comprising: medical image data of the individual and medical non-image data of the individual; processing the medical image data using a first machine learning method to generate a first feature vector; processing the individual's non-image data using a second machine learning method to generate a second feature vector, wherein the second feature vector comprises static information of the individual, which is information that does not change significantly during the course of medical treatment;Merging the first feature vector and the second feature vector using a fusion algorithm to form a merged feature vector that represents the information of both the first feature vector and at least the static information of the second feature vector;and processing the merged feature vector using a third machine learning algorithm to generate a predictive indicator that specifies whether or not the individual has acute respiratory distress syndrome, wherein the fusion algorithm is a trainable algorithm and is trained using at least one loss measure generated by at least one loss function, wherein each loss function processes a plurality of training-merged feature vectors, independently of the third machine learning algorithm, to generate a respective loss measure, and wherein the respective loss measure represents a difference between a predetermined set of merged feature vectors by the loss function. Computer-implemented method according to claim 1, wherein the fusion algorithm comprises a neural autoencoder network. Computer-implemented method according to claim 1 or 2, wherein each training-related merged feature vector is associated with a respective individual and a respective predictive indicator that indicates whether the respective individual has acute respiratory distress syndrome or not. Computer-implemented method according to claim 3, wherein the at least one loss function comprises a first loss function which: for each training-merged feature vector in a first subset of the plurality of training-merged feature vectors, determines a first measure of the difference between the training-merged feature vector and another training-merged feature vector associated with an identical or similar predictive indicator compared to the predictive indicator of the merged feature vector, and sums and averages the determined first measures of difference to generate a first loss measure. Computer-implemented method according to claim 4, wherein the fusion algorithm is trained to reduce the first loss measure. A computer-implemented method according to any one of claims 3 to 5, wherein the at least one loss function comprises a second loss function which: for each training-merged feature vector in a second subset of the plurality of training-merged feature vectors, determines a second measure of the difference between the training-merged feature vector and another training-merged feature vector associated with a different predictive indicator compared to the predictive indicator of the merged feature vector, and sums and averages the determined second measures of difference to generate a second loss measure. Computer-implemented method according to claim 6, wherein the fusion algorithm is trained to increase the second loss measure. A computer-implemented method according to any one of claims 1 to 7, wherein the plurality of training-merged feature vectors is generated by, for each of a first plurality of instances of training individual data comprising medical training image data and medical non-training image data: processing the medical training image data using the first machine learning method to generate a first training feature vector; processing the medical non-training image data using the second machine learning method to generate a second training feature vector; and merging the first training feature vector and the second training feature vector using the fusion algorithm to generate the training-merged feature vector. A computer-implemented method according to claim 8, wherein the at least one loss function comprises a third loss function which: for each training-merged feature vector in a third subset of the plurality of training-merged feature vectors: processes the training-merged feature vector using a reconstruction algorithm to generate a reconstructed first feature vector and a reconstructed second feature vector; determines a third measure of the difference between the reconstructed first feature vector and the first training feature vector that was merged to generate the training-merged feature vector;a fourth measure of the difference between the reconstructed second feature vector and the second training feature vector, which was merged to produce the training-merged feature vector, is determined, and every third measure of difference and / or fourth measure of difference is summed and / or averaged to produce a third loss measure. Computer-implemented method according to any one of claims 1 to 9, wherein: the fusion algorithm is trained using at least one further loss measure generated by a further loss function;The further loss function is configured to process a plurality of instances of training data, each instance of training data comprising: training individual data, comprising medical training image data and medical non-training image data of a training subject, and a predictive training indicator indicating whether or not the training individual has acute respiratory distress syndrome. The further loss function comprises: for each instance of training data: processing the medical training image data and the medical non-training image data using the first machine learning procedure, the second machine learning procedure, the fusion algorithm, and the third machine learning procedure to generate a predictive intermediate indicator;Determining a fifth measure of the difference between the predictive intermediate indicator and the predictive training indicator, and summing and / or averaging each fifth measure of the difference to generate a further loss measure. A computer-implemented method according to any one of claims 1 to 10, further comprising processing the non-image data of the individual using a fourth machine learning method to generate a third feature vector, wherein the merging of the first feature vector and the second feature vector comprises merging the first feature vector, the second feature vector and the third feature vector using a fusion algorithm to form a merged feature vector to represent the information of the first feature vector, the second feature vector and the third feature vector, and the third feature vector represents dynamic information of the individual, which is information that is expected to change during the course of a medical treatment. A computer-implemented method according to any one of claims 1 to 11, wherein the first machine learning algorithm, the second machine learning algorithm and / or the third machine learning algorithm is / are trained using the at least one loss measure generated by the at least one loss function. A computer-implemented method for training a fusion algorithm for use in the method according to any one of claims 1 to 12, wherein the computer-implemented method comprises training the fusion algorithm using at least one loss measure generated by at least one loss function that processes a plurality of training-related merged feature vectors independently of the third machine learning algorithm. Computer program product comprising computer program code means which, when executed on a computer device with a processing system, cause the processing system to perform all steps of the method according to any one of claims 1 to 13. A processing system for generating a predictive indicator for acute respiratory distress syndrome (ARDS) for an individual, wherein the predictive indicator is used in predicting the course of ARDS, and wherein the processing system is configured to: receive individual data, comprising: medical image data and non-medical data; process the medical image data using a first machine learning procedure to generate a first feature vector; process the individual's non-medical data using a second machine learning procedure to generate a second feature vector, wherein the second feature vector comprises static information about the individual that does not change significantly during the course of medical treatment.Merging the first feature vector and the second feature vector using a fusion algorithm to form a merged feature vector that represents the information of the first feature vector and at least the static information of the second feature vector;and processing the merged feature vector using a third machine learning algorithm to generate a predictive indicator that specifies whether or not the individual has acute respiratory distress syndrome, wherein the fusion algorithm is a trainable algorithm and is trained using at least one loss measure generated by at least one loss function, wherein each loss function processes a plurality of training-merged feature vectors, independently of the third machine learning algorithm, to generate a respective loss measure, and wherein the respective loss measure represents a difference between a predetermined set of merged feature vectors by the loss function.

Images