Method for processing an image obtained according to a first imaging modality to generate a second image belonging to a second imaging modality

Abstract

The invention relates to a method for processing a first image (1) of dimension N obtained according to a first imaging modality, comprising a step of generating a second synthetic image (3) of dimension M belonging to a second imaging modality, this second synthetic image being obtained from the first image (1) of dimension N using an artificial intelligence model, this artificial intelligence model comprising: - at least one extractor (9) capable of extracting latent variables from the first image (1) of dimension N, and - at least one generator (11) for generating the second synthetic image of dimension M from the latent variables. Figure for the abstract: Fig. 1

Description

Title of the invention: Method for processing an image obtained according to a first imaging modality to generate a second image belonging to a second imaging modality Technical field

[0001] The present invention relates to an image processing method for the detection of discriminating characteristics, particularly for the purpose of diagnosis.

[0002] The present invention applies in particular in the medical field for the detection of pathologies such as knee osteoarthritis. But the invention has a broader scope since it can be applied outside the medical field, in particular when it involves super resolution of images. State of the prior art

[0003] Knee osteoarthritis, gonarthrosis or KOA for "Knee OsteoArthritis" in English, is a degenerative disease that can lead to disability, caused by the progressive wear and loss of articular cartilage. It is most commonly seen in the elderly population, with clinical symptoms and rates of progression varying between individuals. Symptoms generally become more severe and more frequent over time, resulting in pain, stiffness, swelling and discomfort in the knee after prolonged sitting or rest.

[0004] Unfortunately, the exact cause of KOA is still unknown and no cure exists. Therefore, it is crucial to predict the early onset of KOA and implement timely interventions, such as weight loss, to delay disease progression. Medical imaging technology has advanced significantly, and magnetic resonance imaging (MRI) has become a popular diagnostic tool for knee OA. MRI uses strong magnetic fields, magnetic field gradients, and radio waves to produce detailed images of body organs and structures. In addition to the always necessary routine clinical examination of the symptomatic joint, the standard diagnosis of knee OA relies on radiographic imaging (plain X-rays), which is safe, cost-effective, and widely available.Despite these advantages, plain radiographs are insensitive when it comes to detecting early changes in knee osteoarthritis, making early diagnosis of knee osteoarthritis difficult in clinical practice.

[0005] However, MRI is very rarely available and much more expensive than an X-ray and takes longer to perform. It is used as a last resort to confirm a diagnosis of gonarthrosis, for example. In addition, it cannot be performed on people with heart devices and ferromagnetic metal implants due to the strong magnetic field.

[0006] We know the document Z. Wang, A. Chetouani, D. Hans, E. Lespessailles, and R. Jennane, “Siamese-gap network for early detection of knee osteoarthritis” 2022, IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1-4, 2022, describing an approach, called Siamese-GAP network (“Global Average Pooling”), for the early detection of knee osteoarthritis using a KL grade classification.

[0007] We know the document A. Tiulpin, J. Thevenot, E. Rahtu, P. Lehenkari, and S. Saarakkala, “Automatic knee osteoarthritis diagnosis from plain radio-graphs: A deep learning-based approach” Scientific Reports, 2018, describing a transparent computer-assisted diagnosis method based on the “Deep Siamese Convolutional Neural Network” to automatically assess the severity of knee osteoarthritis according to the Kellgren-Lawrence (KL) grading scale.

[0008] We know the document Y. Nasser, R. Jennane, A. Chetouani, E. Lespessailles, and ME Hassouni, “Discriminative regularized auto-encoder for early detection of knee osteoarthritis: Data from the Osteoarthritis Initiative” IEEE Transactions on Medical Imaging, vol. 39, no. 9, pp. 2976-2984, 2020, describing a discriminative regularized auto-encoder (DRAE) that allows learning both relevant and discriminative properties to improve classification performance.

[0009] Also known is the document J. Masci, U. Meier, D. Ciresan, and J. Schmidhuber, “Stacked convolutional auto-encoders for hierarchical feature extraction” in International conference on artificial neural networks, pp. 52-59, Springer, 2011, describing the use of a convolutional auto-encoder.

[0010] Also known is the document Sanuwani Dayarathna et al., “Deep learning based synthesis of MRI, CT and PET: Review and analysis” in Medical Image Analysis 92 (December 1, 2023) 103046, describing different image synthesis techniques.

[0011] The aim of the present invention is the detection of pathologies such as gonarthrosis for example, at low cost.

[0012] The invention also aims to accelerate the detection of these pathologies. Statement of the invention

[0013] At least one of the objectives is achieved with a method for processing a first image of dimension N obtained according to a first imaging modality. This method comprises a step of generating a second synthetic image of dimension M belonging to a second imaging modality, this second synthetic image being obtained from the first image of dimension N using an artificial intelligence model. This artificial intelligence model comprises:

[0014] - at least one extractor capable of extracting from the first image of dimension N latent variables, and

[0015] - at least one generator for generating the second dimension synthetic image M from the latent variables.

[0016]

[0017] With the method according to the invention, it has been realized that starting from an image according to a first modality, for example a standard X-ray image, it is possible to generate a synthetic image of another modality, for example a pseudo 3D MRI image (Magnetic Resonance Imaging). The 3D image is made up of several two-dimensional images / two-dimensional planes. This method of generating a 3D MRI synthetic image from a 2D image is not intuitive for those skilled in the art. Indeed, the invention makes it possible in particular to generate several two-dimensional images at different depths from a single two-dimensional image. This is made possible by using an extractor whose purpose is to generate latent variables, these variables representing significant attributes of the input image.

[0018] With the invention, these latent variables feed a generator intended to generate images of a modality different or not from that of the input image.

[0019] In other words, the resulting latent variables of the extractor constitute the input to the generator, which uses the extracted representational features to generate images, such as MRI computer images.

[0020] The first image is advantageously a real X-ray image, but it can also be a synthetic image which is used as input to obtain a second synthetic image of a second modality.

[0021]

[0022] The association of an extractor with a generator according to the invention makes it possible to obtain output images which appear enriched compared to the input image. This sequence seems counterintuitive for those skilled in the art.

[0023] The extractor can for example be configured to reduce the size of the first image.

[0024] With the invention, when the second synthetic image of dimension M is a 3D image of MRI type, synthetic MRI data are thus obtained at low cost and quickly compared to a real MRI.

[0025] According to an advantageous characteristic of the invention, the generator can be a deconvolutional neural network, dCNN for “deConvolutional Neural Network” in English.

[0026] Other generator techniques can be used such as a 3D convolutional autoencoder (the decoder acting as the generator), techniques based on UNet models ("fully convolutional network"), VGG for “Visual Geometry Group” in English, ResNet (“Residual neural Network” in English), etc., configured to perform deconvolution.

[0027] According to the invention, the extractor can advantageously be an encoder. Other extractors can be used such as for example UNet (“fully convolutional network”), VGG for “Visual Geometry Group” in English, ResNet (“Residual neural Network”), etc., capable of also generating a latent vector to feed the generator.

[0028] According to an advantageous characteristic of the invention, the artificial intelligence model can further comprise a decoder which, associated with the encoder, forms an auto-encoder capable of generating a third synthetic image belonging to the first imaging modality.

[0029] In other words, we use a classic auto-encoder comprising an encoder and a decoder, but we modify it by integrating a generator which captures the characteristics extracted by the encoder.

[0030] The complete auto-encoder function, i.e. the reconstruction of a synthetic image corresponding to the input image, can be preserved. The resulting synthetic image, the third image, can also be used for classification as mentioned later.

[0031] With the invention, from a first image of a first modality, it is possible to generate both a second image which is a synthetic image of another modality, and a third image which is a synthetic image of the same modality as the first image.

[0032] Preferably, the auto-encoder may be of the convolutional type.

[0033] Such an auto-encoder provides excellent performance in X-ray reconstruction for example and the features extracted from the encoder have a remarkable ability to generate MRI images.

[0034] The technique according to the invention makes it possible to use numerous CNN blocks for "Convolutional Neuronal Network" in English, in the encoder, the generator and the decoder. This makes it possible in particular to preserve as much as possible the spatial information of the input images.

[0035] Other types of encoders can be used, such as the FCN for “Fully Convolutional Network”, the variational auto-encoder VAE for “variational Auto-encoder”, the contractive auto-encoder or “stacked Auto-encoder”, etc.

[0036] According to one embodiment of the invention, during a learning phase of the artificial intelligence model, for each given first image N, third synthetic images and second synthetic images can be generated iteratively, and at each iteration, a quadratic error can be minimized. average according to the following formula:

[0037] J hybrid - l^MSE ( % b 1 ) + M SE ( ^2' ^2

[0038] With Xi and X2 two hyperparameters,

[0039] Xt a first image of dimension N obtained according to the first modality imaging,

[0040]

[0041]

[0042] a third synthetic image belonging to the first imaging modality, X2 a real image of dimension M obtained according to the second imaging modality, a second computer-generated image of dimension M belonging to the second imaging modality.

[0043] Thus, during the training phase, both the difference between the third synthetic image and the first image and the difference between the second synthetic image and real images corresponding to the first image are minimized. Said real images here are, for example, the MRI images of the same object.

[0044] It is thus understood that only the first images are used as training data feeding the model. The real images corresponding to the synthetic images are simply used to minimize the squared error. average.

[0045] According to the invention, in the context of MRI images for example, different MRI sequences (IW, Tl, MPR, etc.) and different views (sagittal, coronal or axial) can be used.

[0046]

[0047] According to an advantageous characteristic of the invention, the hyperparameters can be defined in the following manner:

[0048] Xi + X2 = 1, with Xi and X2 included in [le-1,1].

[0049] Preferably, Xi = 0.6 and X2 = 0.4. Such values ​​allow good results to be obtained.

[0050]

[0051] According to an advantageous embodiment of the invention, the method can further comprise a classification step during which a classification model is fed with the first image of dimension N and the second synthetic image of dimension M so as to extract discriminating characteristics in each image and classify the first image of dimension N according to predetermined groups.

[0052] In this embodiment, a complementary operation of classification of the first image is carried out. Advantageously, the classification process is enriched by injecting into the classification model the second synthetic image which was generated from this same first image.

[0053] The classification model therefore receives as input, in parallel, on the one hand the first image of dimension N, and on the other hand the second synthetic image of dimension M.

[0054] According to another advantageous embodiment of the invention, the method may further comprise a classification step during which a classification model is fed with the third synthetic image and the second synthetic image of dimension M so as to extract discriminating characteristics in each image and classify the first image of dimension N according to predetermined groups.

[0055] In this embodiment, the first image is not used but its synthetic image, which is the third image. However, it is possible to provide in a suitable classification model the parallel use of the first image, the second image and the third image. Other inputs may be provided, such as data relating to the imaged object, such as a patient, when it comes to medical imaging.

[0056] For example, the use of synthetic MRI images considerably improves the performance of the classification model compared to the use of the standard radiographic image alone. The method according to the invention makes it possible to envisage a high-precision tool for diagnosis.

[0057] According to one embodiment of the invention, the classification model may comprise:

[0058] - a first branch of the model which is a neural network to generate independently a first vector of discriminating characteristics from the first image of dimension N,

[0059] - a second branch of the model which is a neural network to generate independently a second vector of discriminating characteristics from the second synthetic image of dimension M,

[0060] - a function for concatenating the first vector and the second vector,

[0061] - a probability distribution function allowing the first to be classified N-dimensional image based on predetermined groups.

[0062] For the first branch of the model, other techniques can be used such as DenseNet, ResNet, VGG, etc.

[0063] For the second branch of the model, other techniques can also be used such as 3D Inception V3, 3D DenseNet, ResNet 3D, 3D VGG, etc.

[0064] Preferably, the first vector and the second vector may be of the same size. Depending on the merging technique used, the first vector and the second vector may be of different sizes.

[0065] According to an advantageous embodiment of the invention:

[0066] - the first image may be a two-dimensional image,

[0067] - the second computer-generated image may be a three-dimensional computer-generated image,

[0068] - the extractor can be an encoder of a two-di convolutional neural network mensions, and

[0069] - the generator can be a three-dimensional deconvolutional neural network.

[0070] Such an arrangement marries perfectly with a first modality being a ra X-ray imaging and a second modality being MRI imaging. The second synthetic image obtained is a 3D image made up of several two-dimensional images commonly called planes, sections or "slices" in English, that is to say a sequence of images (planes or sections) according to a given view.

[0071] Advantageously, the classification model may be a fusion classification model in which:

[0072] - the first branch of the model is a two-di convolutional neural network statements,

[0073] - the second branch of the model is a three-di convolutional neural network measurements.

[0074] Advantageously, the first modality and the second modality can be constituted by one of the following pairs:

[0075] - Standard X-ray / Tomography (CT)

[0076] - Dual Energy X-ray Absorption, DEXA for “Dual Energy X-ray Absorp- tiometry » in English) / Standard X-ray,

[0077] - Standard X-ray / Dual-photon X-ray absorption, DEXA for "Dual Energy X-ray Absorptiometry » in English),

[0078] - Tomography (CT) / MRI

[0079] - Standard X-ray / EOS X-ray; EOS being a system allowing take full body images in functional position.

[0080] Many combinations are possible. In a preferred example, the dimension M is greater than or equal to the dimension N. We can also consider the opposite case where N is greater than M.

[0081] According to another aspect of the invention, there is provided a computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the method as described above.

[0082] Also provided is a computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to implement the method as described above.

[0083] Description of the figures and embodiments.

[0084] Other advantages and particularities of the invention will appear on reading the detailed description of implementations and embodiments which are in no way limiting, and of the following attached drawings:

[0085] [Fig.l] [Fig.l] is a schematic view of the overall method according to the invention,

[0086] [Fig.2] [Fig.2] is a schematic view of a conventional autoencoder according to the art prior,

[0087] [Fig.3] [Fig.3] is a schematic view of a convolutional autoencoder according to the invention integrating a generator,

[0088] [Fig.4] [Fig.4] is a schematic view of a classification model according to the invention.

[0089] The embodiments which will be described below are in no way limiting; it will be possible in particular to implement variants of the invention comprising only a selection of characteristics described below isolated from the other characteristics described, if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection comprises at least one preferably functional characteristic without structural details, or with only a part of the structural details if this part alone is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art.

[0090] In the figures, the elements common to several figures retain the same reference.

[0091] Although the invention is not limited thereto, a method according to the invention will now be described for the classification of a 2D standard X-ray image by generating a 3D MRI synthesis image for the early detection of knee osteoarthritis (KOA).

[0092] [Fig.l] is a schematic view of the overall method according to the invention.

[0093] A first image 1, 2D, of an X-ray of a knee is distinguished. This first image 1 feeds a generation model 2 according to the invention to generate a second synthetic 3D image 3 of the MRI type.

[0094] The first image 1 and the second image 3 then feed a fusion classification model 4 allowing the prediction of a radiological KL score. The KL score is a composite index created by Kellgren and Lawrence and allowing a classification into 5 classes: absence of osteoarthritis, doubtful osteoarthritis, minimal, certain, advanced.

[0095] The generation 2 model, with or without the fusion classification model 4, constitutes the artificial intelligence model according to the invention.

[0096] The method according to the invention relates to a new approach based on deep learning for, for example, the early detection of knee osteoarthritis. In the example described, two models are distinguished: the MRI generation model and the fusion classification model. The MRI generated from an X-ray is combined with this X-ray to obtain additional information. to predict the KL score.

[0097] The MRI generation model uses an autoencoder as seen in [Fig.3],

[0098] A conventional autoencoder (AE) is depicted in Figure 2. It is an unsupervised learning model composed of an encoder 5 and a decoder 6. It is based on a back-propagation algorithm and optimization methods (such as the gradient descent method), and uses the input data X as supervisory data to make the neural network learn a mapping relationship, thus obtaining a reconstructed output X- The encoder 5 aims to encode the input X, which is high-dimensional, into a low-dimensional hidden variable h so that the neural network learns the most informative features. The decoder 6 then processes h so as to reconstruct the input data. The differences between the reconstructed data and the initial data make it possible to measure the error made by the autoencoder.Training consists of modifying the autoencoder parameters in order to reduce the reconstruction error measured on different training data, i.e. minimize(dist(x,X)). .

[0099] The process of encoding the original data X from the input layer to the hidden layer is as follows:

[0100] h = g01 (X) = o(WiX + bû

[0101] where goi is the encoder function, o is an activation function, W) and bi are the encoder weight and bias matrix respectively.

[0102] The process of decoding the hidden variable h from the hidden layer to the output layer is as follows:

[0103] x = go2 W = o(W2 / z + b2)

[0104] where gü2 is the decoder function, W2 and b2 are respectively the weight and bias matrix of the decoder.

[0105] Usually, the AE model minimizes the reconstruction error using the Mean Square Error or MSE cost function, which is defined as:

[0106] T _} x2 JMSE - 77 A " A / ' v AG 1

[0107] where N is the number of samples in each training batch, and T represents the training set.

[0108] [Fig. 3] is a schematic view of the generation 2 model according to the invention, in this case it is MRI type synthetic images which are generated. The auto-encoder chosen for the invention is a convolutional auto-encoder (CAE) 7 capable of constructing a synthetic image 8 from the first 2D radiography image 1.

[0109] According to the invention, the latent variables h extracted between the encoder 9 and the decoder 10 are used as input data of the generator 11, which is based in this example on a 3D deconvolutional neural network. A deconvolutional neural network architecture is notably described in the document by Feilong Cao et al., “Deconvolutional neural network for image super-resolution”, Neural Networks Volume 132, December 2020, Pages 394-404.

[0110] In the example according to the invention, the output of the generator 11 are 3D synthetic images equivalent to a 3D MRI sequence. These are 3D synthetic images resembling MRI images that would have been obtained by performing an MRI of the same imaged object to obtain the first 2D dimension image 1.

[0111] Within the framework of the invention, in the operating phase, it is possible to envisage a generation 2 model consisting solely of the encoder 9 and the generator 11.

[0112] The autoencoder's encoder is cleverly used to extract informative features and feed the generator.

[0113] Encoder 9 in [Fig.3] consists of six 2D CNN (Convolutional Neural Network) blocks. These blocks consist of a set of 2D convolutional layers of the same kernel size of 3x3 with different depths (i.e. 32, 64, 128, 256, 512 and 1024) - a Batch Normalization (BN) layer - a LeakyReLU (Leaky Rectified Linear Unit) layer with a predefined slope of 0.2.

[0114] Decoder 10 in [Fig.3] has an almost mirror structure of encoder 9. It consists of seven 2D deconvolutional neural network blocks. These blocks consist of a set of 2D deconvolutional layers of the same 4x4 kernel size with different depths (i.e. 1024, 512, 256, 128, 64, 32 and 1) - a BN layer - a LeakyReLU layer with a predefined slope of 0.2.

[0115] The generator 11 is based on a 3D deconvolutional neural network (dCNN) architecture. This architecture consists of seven 3D dCNN blocks and two 2D CNN blocks. Each 3D dCNN block consists of a set of 3D deconvolutional layers of the same kernel size of 4x4x4 with different depths (i.e., 1024, 512, 256, 128, 64, 32, and 16) - a 3D BN layer - a ReLU layer. Then, for the next first 2D CNN block, it consists of a set of 2D convolutional layers of kernel size 1x1 with depths of 8 - a BN layer - a ReLU layer. Finally, for the last 2D CNN block, it consists of a 2D convolutional layer of kernel size 1x1 with depths of 128 with an output channel of 20.The resulting latent variables between encoder 9 and decoder 10 constitute the input to generator 11, which uses the extracted representation features to generate the corresponding MRI images 3. laying eggs.

[0116] During the training phase, outputs 8 and 3 are used to minimize on the one hand the difference between output 8 and input image 1 and on the other hand the difference between output 3 and real MRI images 12. However, these real images 12 do not feed the generation model.

[0117] This hybrid minimization of the MSE error is expressed according to a loss function Jhybride •

[0118] jhybrid = 2sJMSE (X], XJ + À2JMSE (-^2' ^2)

[0119] with Xi and X2 hyperparameters e [le-1, 1] requiring XI + X2 = 1. Preferably, XI = 0.6 and XI = 0.4.

[0120] In [Fig.4] a fusion classification model 4 is shown. It is made up of two branches (i.e. a 2D CNN and a 3D CNN), its input is composed of the first 2D radiographic image 1 and a synthetic image 3 which is of the 3D MRI sequence type. Instead of the first image 1, or in parallel if we add a new branch to the network, we can use the synthetic image 8.

[0121] Input images 1 and 3 are combined as an input pair of the fusion classification model to assess the severity of knee osteoarthritis (KOA) in the described example. This combination of the X-ray and MRI images significantly improves the performance of the classification model compared to using the X-ray image alone or the actual MRI images alone.

[0122] This makes it possible to simplify and reduce the cost of systems, for example for the detection of knee osteoarthritis.

[0123] The fusion classification model according to the invention makes it possible to combine 2D radiographic images and 3D MRI-type synthetic images of a patient and to extract the discriminating characteristics via two independent models based on the CNN, respectively.

[0124] The first branch of the fusion classification model 4 uses a 2D convolutional neural network 14 which generates a vector 15 of size 2048 from the 2D radiographic image 1.

[0125] The second branch of the fusion classification model 4 uses a 3D convolutional neural network 16 which generates a vector 17 of size 2048 from the 3D MRI type synthetic image 3.

[0126] The 3D convolutional neural network 16 has thirteen layers, namely nine CNN layers and four pooling layers in four blocks (i.e., 2, 3, 4, and 4). Each of these four blocks is composed of a combination of 3D convolution with filters of different sizes (i.e., 32, 64, 128, and 256) for layers having the same kernel size (3x3x3), rectified linear unit (ReLU), and a layer 3D max-pooling of size (2x2x2). Then, two fully connected layers with node lengths of 4096 and 2048 are used to obtain the 17 feature vector of length 2048.

[0127] Vectors 15 and 17 are of the same size so as to take into account equally the characteristics of the two inputs (x-ray images and pseudo MRI). Other combination techniques can be used with the same size or different sizes between the two vectors.

[0128] These resulting vectors are then concatenated to obtain a final vector 18 of features of length 4096. Finally, the probability distribution of grades KL-0 and KL-2 corresponding to the early detection of osteoarthritis is obtained via a Softmax layer 13.

[0129] As a classification loss function, we use the cross-entropy (CE) loss defined by: [OBO] JCE = lLs€T-yJog(ps), V s ŒT

[0131] hf=T lo y

[0132] V k &K

[0133] with ys a ground-truth label converted into vectors of zeros and ones of sample s; y and T are the predicted and actual labels respectively; k represents the KL score of sample s, is the conditional probability distribution; K is a set of KL scores (KL-0 and KL-2).

[0134] The generation model can be implemented as follows. Initialization of the model weights can be done using the Kaiming initialization method and training of the model can be done using the Adam optimizer for 1000 epochs. A batch-size can be set to 32 and learning rates of 0.001 and 0.0001 are used for the encoder and decoder on the one hand, and the generator on the other hand, respectively.

[0135] The classification model can be implemented as follows. Initialization of model weights can be performed using the Kaiming initialization method and training of the model can be performed using the Adam optimizer for 500 epochs. The batch-size can be set to 8 and a learning rate to 0.001. To deal with the imbalanced dataset, the bootstrap-based oversampling method is used. Then, the balanced dataset was randomly divided into training, validation, and test sets in a ratio of 7:1:2 for each KL score, respectively. Data augmentations, including random rotation, brightness, contrast, jitter, and gamma correction, are performed randomly. In addition, to avoid overfitting, a weight decay with a coefficient of 0.0003 was applied.

[0136] The present invention is a novel approach to improve the detection performance of early grades of knee osteoarthritis (KOA), i.e., KL-0 and KL-2. The overall model includes a pseudo MRI generation model and a fusion classification model. The generation model is based on a CAE and a 3D dCNN to generate pseudo MRI sequence images of the knee from an X-ray. These pseudo MRIs are combined with the X-ray to assist in the diagnosis of KOA for patients who only have X-rays.

[0137] The convolutional autoencoder (CAE) based generation model is capable of holistically extracting the features embedded in 2D X-ray images. These features are advantageously used to generate MRI sequence images via a generator.

[0138] Knee osteoarthritis (KOA) is a common musculoskeletal disorder and radiographs are commonly used for its diagnosis due to their cost-effectiveness. On the other hand, magnetic resonance imaging (MRI) is a more advanced imaging technique that provides detailed soft tissue information and has become an option in additional diagnosis for the assessment of KOA. Unfortunately, the high cost of MRI means that most patients with KOA only have radiographic images available. The present invention thus addresses this problem by providing a new approach to improve the early diagnosis of KOA, including specifically distinguishing between KL-0 and KL-2 stages.

[0139] Of course, the invention is not limited to the examples which have just been described. Numerous modifications can be made to these examples without departing from the scope of the present invention as described.

Claims

Claims

1. 1. Method for processing a first image (1) of dimension N obtained according to a first imaging modality, comprising a step of generating a second synthetic image (3) of dimension M belonging to a second imaging modality, this second synthetic image being obtained from the first image (1) of dimension N using an artificial intelligence model, this artificial intelligence model comprising: - at least one extractor (9) capable of extracting latent variables from the first image (1) of dimension N, and - at least one generator (11) for generating the second synthetic image of dimension M from the latent variables.

2. 2. Method according to claim 1, characterized in that the generator (11) is a deconvolutional neural network (dCNN).

3. 3. Method according to claim 1 or 2, characterized in that the extractor is an encoder (9).

4. 4. Method according to claim 3, characterized in that the artificial intelligence model further comprises a decoder (10) which, associated with the encoder, forms an auto-encoder (7) capable of generating a third synthetic image (8) belonging to the first imaging modality.

5. 5. Method according to claim 4, characterized in that the auto-encoder is of the convolutional type.

6. 6. Method according to claim 5, characterized in that during a learning phase of the artificial intelligence model, for each given first image N, third synthetic images and second synthetic images are iteratively generated, and at each iteration, a mean square error is minimized according to the following formula: j, , — il ( YY 1 -u JT ( YY '[With Xi and X2 two hy- Jhybrid ~ MSE\A2JMSE\A2'^2) perparameters, a first image of dimension N obtained according to the first imaging modality, j a third synthetic image belonging to the first imaging modality, X2 a real image of dimension M obtained according to the second imaging modality, and a second synthetic image of dimension M belonging to the second imaging modality.

7. 7. Method according to claim 6, characterized in that Xi + X2 = 1, with Xi and X2 included in [le-1,1].

8. 8. Method according to claim 7, characterized in that Xi = 0.6 and X2 = 0.

4.

9. 9. Method according to any one of the preceding claims, characterized in that it further comprises a classification step during which a classification model (4) is fed with the first image (1) of dimension N and the second synthetic image (3) of dimension M so as to extract discriminating characteristics in each image and classify the first image of dimension N according to predetermined groups.

10. 10. Method according to any one of claims 4 to 8, characterized in that it further comprises a classification step during which a classification model (4) is fed with the third synthetic image (8) and the second synthetic image (3) of dimension M so as to extract discriminating characteristics in each image and classify the first image of dimension N according to predetermined groups.

11. 11. Method according to claim 9 or 10, characterized in that the classification model comprises: - a first branch of the model which is a neural network for independently generating a first vector (15) of discriminating characteristics from the first image (1) of dimension N, - a second branch of the model which is a neural network for independently generating a second vector (17) of discriminating characteristics from the second synthetic image (3) of dimension M, - a function for concatenating the first vector and the second vector, - a probability distribution function (13) making it possible to classify the first image of dimension N according to predetermined groups.

12. 12. Method according to claim 11, characterized in that the first vector and the second vector are the same size.

13. 13. Method according to any one of the preceding claims, characterized in that: - the first image is a two-dimensional image, - the second synthetic image is a three-dimensional synthetic image, - the extractor is an encoder of a two-dimensional convolutional neural network, and - the generator is a three-dimensional deconvolutional neural network.

14. 14. Method according to any one of claims 11 to 13, characterized in that the classification model is a fusion classification model in which: - the first branch of the model is a two-dimensional convolutional neural network, - the second branch of the model is a three-dimensional convolutional neural network.

15. 15. Method according to any one of the preceding claims, characterized in that the first modality and the second modality are constituted by one of the following pairs: - Standard X-ray radiography / Tomography (CT) - Dual Energy X-ray Absorptiometry (DEXA) / Standard X-ray radiography, - Radiography / Dual Energy X-ray Absorptiometry (DEXA) - Tomography (CT) / MRI - Standard X-ray radiography / EOS radiography.

16. 16. Computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the method according to any one of the preceding claims.

17. 17. A computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of claims 1 to 15.

Images