Unsupervised feature analysis method and system based on 18F-AV45 PET image

Abstract

The invention discloses an unsupervised feature analysis method based on an 18F-AV45 PET image. The unsupervised feature analysis method comprises the following steps: acquiring medical image data; constructing a prediction network based on a generative network framework, wherein the prediction network comprises an image registration module, a feature extraction module and a reconstruction module; performing unsupervised training on the prediction network by using the training set to obtain an unsupervised generation model for reconstructing an initial image of the healthy person, and designing a residual module for calculating a feature residual between the two images; the output end of the unsupervised generation model is connected with the input end of the residual module, and a feature analysis model used for analyzing image differences is formed; and inputting a brain PET image to be analyzed into the feature analysis model to output an image feature difference distribution result. The invention also provides an unsupervised feature analysis system. According to the method provided by the invention, the information of the precipitated brain region in the PET image can be accurately identified, and a more comprehensive reference is provided for subsequent clinical diagnosis.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to an unsupervised feature analysis method and system based on 18F-AV45 PET images. Background Art

[0002] The unsupervised residual strategy is a method for detecting abnormalities, which identifies abnormal areas by comparing the differences between the original image and the reconstructed image. This strategy has been applied in the fields of tumor detection and brain atrophy detection in MRI imaging. In recent years, PET, as an emerging nuclear medicine functional imaging technology, can non-invasively and dynamically conduct specific quantitative research on AD biomarkers, thereby exploring pathology and protein distribution in the brain. The current deep learning model focuses on learning the distribution of labeled data, ignoring the individual heterogeneity of the subjects and the different Aβ deposition patterns, making the Aβ imaging individual deposition detection model based on deep learning unavailable.

[0003] Patent document CN114999629A discloses an early prediction method, system, and device for AD based on multi-feature fusion. This method is based on the complex problem of predicting AD from Brain 18F-AV45 PET slice images, and redesigns a new neural imaging prediction model that includes a dual convolution backbone network and a classification network; and introduces a multi-attention module into the network model. During the data set acquisition process, PET slices are amplified by filtering, cropping, etc. instead of traditional amplification methods to improve the generalization characteristics of the network model. In the training stage of the network model, fuzzy labels are applied to supervised contrastive learning loss, and in the prediction stage, the classification results and clinical psychiatric assessment results are combined for joint analysis.

[0004] Patent document CN117711579A discloses a method and device for visualizing heterogeneous pathology of glioma based on multimodal images, the method comprising: S1, obtaining a set of glioma images; S2, obtaining tissue pathology detection result labels of several points of the glioma shown in the image sample, and obtaining the point area corresponding to the point in the image sample based on the coordinates of the point; S3, fusing the multimodal images of the image sample to obtain fusion feature embedding; S4, inputting the fusion feature embedding into a multi-task prediction model for prediction analysis, and training the multi-task prediction model based on the loss value; S5, predicting the multi-task prediction model obtained by training; S6, reconstructing the model based on the molecular subtype and tissue-level pathological grade of the glioma border of each image in the image sequence and each point area in the glioma border to obtain a molecular visual model of the glioma. Summary of the invention

[0005] The purpose of the present invention is to provide an unsupervised feature analysis method and system based on 18F-AV45 PET images, which can accurately identify the deposited brain area information in PET images and provide a more comprehensive reference for subsequent clinical diagnosis.

[0006] In order to achieve the first object of the present invention, the following technical solution is provided: an unsupervised feature analysis method based on 18F-AV45 PET images, comprising the following steps: Acquire medical image data, including brain PET image data of healthy subjects, and register the brain PET image data to a standard meningeal plate to obtain an initial image, and form all the initial images of healthy subjects into a training set; Building a prediction network based on a generative network framework, wherein the prediction network includes an image registration module, a feature extraction module, and a reconstruction module; The image registration module is used to register the input brain PET image data to a preset standard meningeal plate to output a corresponding initial image; The feature extraction module is used to divide the initial image into image blocks, and extract features from the image blocks obtained by the division to output corresponding image features; The reconstruction module reconstructs the image using the input image features to obtain a corresponding reconstructed image; The prediction network is trained unsupervised using the training set to obtain an unsupervised generative model for reconstructing the initial image of a healthy person, and a residual module is designed to calculate the feature residual between the two images; Connecting the output end of the unsupervised generation model to the input end of the residual module to form a feature analysis model for analyzing image differences; The brain PET image to be analyzed is input into the feature analysis model to output the image feature difference distribution result.

[0007] The present invention maps the patient image onto a cognitively normal reference image and accurately locates the abnormal deposition area by comparing it with the original image, so that the distribution characteristics of Aβ PET images of cognitively normal individuals can be learned through the potential feature enhancement generative adversarial network, and the abnormal distribution area of ​​the patient can be output.

[0008] Specifically, the process of registering the brain PET image data to the standard meningeal plate includes origin correction, spatial standardization and smoothing.

[0009] Specifically, the spatial standardization includes linear registration and nonlinear registration. The linear registration includes linear coordinate transformation and radial transformation. The nonlinear registration refers to performing nonlinear transformation on a local part of the image.

[0010] Specifically, the generative network framework is built based on the GAN network and the VAE network.

[0011] Specifically, in the unsupervised training process, multiple loss functions are used to train the prediction network to update the weights of the network parameters, thereby solving the problem of distinguishing individual heterogeneity and non-pathological PET tracer signals in PET images.

[0012] Specifically, the expression of the multiple loss function is as follows: Among them, L1 represents the constraint function of the GAN network; P represents the intermediate distribution of the two distributions, λ represents the constant term; z represents the latent variable; L2 represents the pixel difference between the reconstructed image and the corresponding original image; x represents the pixel of the original image; x' represents the pixel of the generated image; L3 represents the auxiliary constraint function, which is used to ensure the latent vector of the original image and the latent layer distribution of the generated image; z represents the latent layer distribution of the original image; z' represents the latent layer distribution of the generated image.

[0013] Specifically, the residual module uses the ROI algorithm to calculate the feature residual between the reconstructed image and the corresponding input initial image, and labels the corresponding positions in the initial image based on the feature residual, and outputs all the labeling results as image feature difference distribution results.

[0014] Specifically, the labels include whole cerebellum, meta-ROI, and Aβ stages 1 to 4.

[0015] In order to achieve the second object of the present invention, the following technical solutions are provided: an unsupervised feature analysis system, implemented by the above-mentioned unsupervised feature analysis method based on 18F-AV45 PET images, comprising an image preprocessing unit, an image reconstruction unit and an image residual analysis unit; The image preprocessing unit is used to register the input brain PET image data to the standard meningeal plate to output an initial image; The image reconstruction unit predicts a reconstructed image corresponding to a healthy person based on the initial image; The image residual analysis unit is used to analyze the residual between the reconstructed image and the initial image to output the image feature difference distribution result for guiding doctors to design medical treatment plans.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The unsupervised region of interest extraction method based on 18F-AV45 PET images overcomes the problem of unclear distinction between individual heterogeneity and normal Aβ signals in traditional methods, thereby accurately identifying abnormal Aβ deposition areas in a single image and extracting relevant imaging markers. The combination of GAN and VAE and the addition of special multiple loss functions significantly improve the diagnostic prediction performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of the unsupervised feature analysis method provided in this embodiment; Figure 2 A flow chart of unsupervised residual strategy detection provided for this embodiment; Figure 3 A schematic diagram of the original image, the reconstructed image, and the corresponding image feature difference distribution results provided in this embodiment. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, an unsupervised feature analysis method provided in this embodiment includes the following steps: Acquire medical image data, including brain PET image data of healthy subjects, and register the brain PET image data to a standard meningeal plate to obtain an initial image, and form all the initial images of healthy subjects into a training set; Building a prediction network based on a generative network framework, wherein the prediction network includes an image registration module, a feature extraction module, and a reconstruction module; The image registration module is used to register the input brain PET image data to a preset standard meningeal plate to output a corresponding initial image; The feature extraction module is used to divide the initial image into image blocks, and extract features from the image blocks obtained by the division to output corresponding image features; The reconstruction module reconstructs the image using the input image features to obtain a corresponding reconstructed image; The prediction network is trained unsupervisedly using the training set to obtain an unsupervised generative model for reconstructing the initial image of a healthy person. The unsupervised generative model is as follows: Figure 1 As shown in (A), the update process of model parameters during training is as follows Figure 1 As shown in (B) in the figure, a residual module is designed to calculate the feature residual between two images; in, Figure 1 (A) in the figure shows the structure of LFGAN4Aβ. Figure 1 (B) in Figure 1 is the construction of the loss function. To solve the problem of identifying individual heterogeneity and non-pathological PET tracer signals in Aβ PET images, refer to Figure 1 (B) The latent constraint term is effectively conditioned by combining the encoding of the normal distribution in the model with a special multiple loss function. This regularization preserves the individual characteristics of the subjects and ultimately facilitates the detection of individual heterogeneity and pathological Aβ deposition.

[0020] Connecting the output end of the unsupervised generation model to the input end of the residual module to form a feature analysis model for analyzing image differences; The brain PET image to be analyzed is input into the feature analysis model to output the image feature difference distribution result.

[0021] More specifically, brain 18F-florbetapir (AV45) PET images were obtained from the public data of the Alzheimer's Disease Neuroimaging Initiative (ADNI) database and the Huashan Hospital PET Center to construct 18F-florbetapir (AV45) PET image data and corresponding clinical scale information for 1,200 healthy and diseased individuals. PET refers to positron emission tomography imaging; 18F-florbetapir (AV45) refers to a compound containing the radioactive isotope fluorine-18, which is mainly used for positron emission tomography (PET) imaging to detect beta-amyloid plaques in the brain.

[0022] Origin correction This step mainly performs anterior commissure (AC)-posterior commissure (PC) correction of the brain, unifies the image space origin of different sample data, and sets the image origin position at AC.

[0023] Spatial standardization: Due to the different shapes and sizes of the brains of different subjects, the images cannot be well overlapped in space. In this step, the images corrected in the previous step are registered to the standard brain template space of the Montreal Neurological Institute (MNI) in order to unify the coordinate space of all images. The main registration methods include linear registration and nonlinear registration: linear registration includes linear coordinate transformation and affine transformation, and nonlinear registration performs nonlinear transformation on the local area.

[0024] Smoothing This step uses a Gaussian filter to smooth the image to reduce the noise of the image, thereby further improving the signal-to-noise ratio of the image.

[0025] like Figure 2 The prediction network provided by this embodiment is shown, which includes two parts: an unsupervised residual strategy and a generative network framework. The generative network is the core of the unsupervised residual strategy, including two encoders, a decoder and a discriminator, wherein the decoder and the discriminator correspond to Figure 1 (A) and Figure 1 (B) in.

[0026] In order to optimize the problem of individual heterogeneity and non-pathological deposition being difficult to distinguish in imaging, a special multiple loss function is used to adjust the potential constraints of the network, which retains the individual characteristics of the subjects and thus improves the detection performance of individual heterogeneity and pathological Aβ deposition. The multiple loss function is expressed as follows: Among them, L1 is the main loss function and also the constraint term of the traditional GAN ​​model. P (penalty) represents the intermediate distribution of the two distributions; L2 represents the pixel difference between the generated image and the original image, where x represents the pixel of the original image and x' represents the pixel of the generated image; L3 represents the consistency between the latent vector of the original image and the latent distribution of the generated image. Using it as an auxiliary constraint can ensure higher quality image restoration and feature retention, where z represents the latent distribution of the original image and z' represents the latent distribution of the generated image. Based on the evaluation system of the generative network, the similarity between the reconstructed image and the target image is one of the criteria for evaluating the performance of the model. Three indicators are used to evaluate the robustness of the reconstructed image of the model to detect the training effect of the model.

[0027] The reconstructed images are compared with the original images in the training, validation and test groups using three metrics: structural similarity (SSIM), peak signal-to-noise ratio (PSNR) and mean square error (MSE). In addition, to demonstrate the superiority of multiple loss functions, we conduct ablation experiments in the same training, validation and test groups without training the model loss function.

[0028] The SSIM calculation process is shown in the formula: in, are the pixel means of the two images respectively; are the variances of the two images respectively; is the covariance of the two images; c1 and c2 are two constants determined by the pixel value range.

[0029] Calculating MSE is one of the methods to measure the difference between two images. It represents the square mean of the difference between each pixel value in the two images. The smaller the index is, the smaller the difference between the pixel values ​​of the two images is, and the more similar the images are. Its calculation is shown in the formula: Among them, I(i,j) and K(i,j) represent the pixel value at position (i,j) on image I and image K respectively; m and n are the length and width of the image respectively.

[0030] PSNR is a common indicator for measuring image quality. It calculates the peak signal-to-noise ratio between the original image and the reconstructed image. The higher the indicator, the higher the similarity between the two images and the better the image quality. Its calculation is shown in the formula: Among them, MAX1 is the maximum pixel value of the reconstructed image.

[0031] The unsupervised residual strategy is a method for detecting anomalies, which identifies abnormal regions by comparing the differences between the original image and the reconstructed image.

[0032] The model is trained using normal data, which enables the model to learn the distribution characteristics and patterns of normal images; then unlabeled data is input in the test phase, and the model will output a reconstructed image based on the distribution characteristics and patterns of normal images; finally, the residual image of the original image and the reconstructed image is output.

[0033] In summary, the residual strategy simulates the process of humans quickly identifying various anomalies with the assistance of a large amount of prior knowledge.

[0034] like Figure 3, which is a schematic diagram of the original image, the reconstructed image, and the corresponding image feature difference distribution results provided in this embodiment.

[0035] This embodiment also provides an unsupervised feature analysis system, which is implemented by the unsupervised feature analysis method provided by the above embodiment, and includes an image preprocessing unit, an image reconstruction unit and an image residual analysis unit; The image preprocessing unit is used to register the input brain PET image data to the standard meningeal plate to output an initial image; The image reconstruction unit predicts a reconstructed image corresponding to a healthy person based on the initial image; The image residual analysis unit is used to analyze the residual between the reconstructed image and the initial image to output the image feature difference distribution result for guiding doctors to design medical treatment plans.

[0036] In addition, the terms "upper", "lower", "inner", "outer", "front", "rear" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps described in these embodiments do not limit the scope of the present invention.

[0037] Of course, the above description is only a specific embodiment of the present invention and is not intended to limit the scope of implementation of the present invention. All equivalent changes or modifications made according to the structure, characteristics and principles described in the patent application scope of the present invention should be included in the patent application scope of the present invention.

[0038] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An unsupervised feature analysis method based on 18F-AV45 PET images, characterized in that: The following steps are involved: Acquire medical image data, including brain PET image data of healthy subjects, and register the brain PET image data to a standard meningeal plate to obtain an initial image, and form all the initial images of healthy subjects into a training set; Building a prediction network based on a generative network framework, wherein the prediction network includes an image registration module, a feature extraction module, and a reconstruction module; The image registration module is used to register the input brain PET image data to a preset standard meningeal plate to output a corresponding initial image; The feature extraction module is used to divide the initial image into image blocks, and extract features from the image blocks obtained by the division to output corresponding image features; The reconstruction module reconstructs the image using the input image features to obtain a corresponding reconstructed image; The prediction network is trained unsupervised using the training set to obtain an unsupervised generative model for reconstructing the initial image of a healthy person, and a residual module is designed to calculate the feature residual between the two images; Connecting the output end of the unsupervised generation model to the input end of the residual module to form a feature analysis model for analyzing image differences; The brain PET image to be analyzed is input into the feature analysis model to output the image feature difference distribution result.

2. The unsupervised feature analysis method based on 18F-AV45 PET images according to claim 1, characterized in that: The process of registering the brain PET image data to the standard meningeal plate includes origin correction, spatial standardization and smoothing.

3. The unsupervised feature analysis method based on 18F-AV45 PET images according to claim 2, characterized in that: The spatial standardization includes linear registration and nonlinear registration. The linear registration includes linear coordinate transformation and radial transformation. The nonlinear registration refers to performing nonlinear transformation on a local part of the image.

4. The unsupervised feature analysis method based on 18F-AV45 PET images according to claim 1, characterized in that: The generative network framework is built based on the GAN network and the VAE network.

5. The unsupervised feature analysis method based on 18F-AV45 PET images according to claim 1, characterized in that: During the unsupervised training process, multiple loss functions are used to train the prediction network to update the weights of the network parameters.

6. The unsupervised feature analysis method based on 18F-AV45 PET images according to claim 5, characterized in that: The expression of the multiple loss function is as follows: Among them, L1 represents the constraint function of the GAN network; P represents the intermediate distribution of the two distributions, λ represents the constant term; z represents the latent variable; L2 represents the pixel difference between the reconstructed image and the corresponding original image; x represents the pixel of the original image; x' represents the pixel of the generated image; L3 represents the auxiliary constraint function, which is used to ensure the latent vector of the original image and the latent layer distribution of the generated image; z represents the latent layer distribution of the original image; z' represents the latent layer distribution of the generated image.

7. The unsupervised feature analysis method based on 18F-AV45 PET images according to claim 1, characterized in that: The residual module uses the ROI algorithm to calculate the feature residual between the reconstructed image and the corresponding input initial image, and labels the corresponding positions in the initial image based on the feature residual, and outputs all the labeling results as image feature difference distribution results.

8. The unsupervised feature analysis method based on 18F-AV45 PET images according to claim 7, characterized in that: The labels include whole cerebellum, meta-ROI, and Aβ stages 1 to 4.

9. An unsupervised feature analysis system, characterized in that: The unsupervised feature analysis method based on 18F-AV45 PET images as described in any one of claims 1 to 8 is implemented, which includes an image preprocessing unit, an image reconstruction unit and an image residual analysis unit; The image preprocessing unit is used to register the input brain PET image data to the standard meningeal plate to output an initial image; The image reconstruction unit predicts a reconstructed image corresponding to a healthy person based on the initial image; The image residual analysis unit is used to analyze the residual between the reconstructed image and the initial image to output the image feature difference distribution result for guiding doctors to design medical treatment plans.

Images