PET / MRI (positron emission tomography / magnetic resonance imaging) image data governance model training method and device and data governance method

Abstract

The invention discloses a PET / MRI (positron emission tomography / magnetic resonance imaging) image data governance model training method and device and a data governance method. The PET / MRI image data governance model training method comprises the following steps: acquiring a tissue component label, a tissue component prediction result and a tissue component identification model through a medical image control system and an enhanced deep residual network model; obtaining a diagnostic book label, a diagnosis result and a diagnostic book identification model according to a preset language model and a preset questioning mode; performing cross task optimization on the tissue component identification model and the diagnostic book identification model according to the clearness of the MRI / PET image of the same body position of the same person in the sample medical image, the MRI / PET image of the same person in the preset time period, the text type digital medical record of the same person in the preset time period and the sample medical image; and associating the tissue component identification model with the diagnostic book identification model through a graph neural network model to obtain a data treatment model of the PET / MRI image. According to the invention, by means of the compensation function between the AI tool and the body, more modal evaluation results can be obtained, so that the human health can be evaluated more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image data processing, and in particular, to a data management model training method, device and data management method for PET / MRI images. Background Art

[0002] MRI (Magnetic resonance imaging) is a medical imaging technology used in radiology to form pictures of the human body's anatomical structure and physiological processes. PET (Positronemission Tomography) is a nuclear imaging technology that can display metabolic processes in the body. In the medical field, PET / MRI images are used to scan specific parts of a patient to show whether they have a disease. This is a widely used technology. During the scanning process, the doctor will diagnose whether the scanned part has a disease based on experience, and write the diagnosis results on the diagnosis book corresponding to the PET / MRI image. The patient can preliminarily determine his or her health status based on the diagnosis book and PET / MRI image, and further consult the outpatient doctor. However, the content that a PET / MRI image can reflect is often relatively single. If the patient wants to obtain more information about human body functions and determine whether there is a warning for his or her health status, he or she needs to scan more parts of the body, which increases the diagnosis time, or relies heavily on the doctor's experience, and the accuracy of the diagnosis result is not stable enough. In addition, if the previous PET / MRI image needs to be called and it has been damaged, the corresponding diagnostic information cannot be obtained to assist the current diagnosis and treatment.

[0003] Therefore, those skilled in the art are in urgent need of developing a new technical solution to solve the above problems. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present invention discloses a data management model training method, device and data management method for PET / MRI images.

[0005] According to a first aspect of the disclosed embodiments of the present invention, a method for training a data governance model for PET / MRI images is provided, the method comprising:

[0006] Obtaining each tissue component label and tissue component prediction result of a sample medical image through a medical image control system and an enhanced deep residual network model, and obtaining a trained tissue component recognition model according to the tissue component label and tissue component prediction result, wherein the sample medical image includes: an MRI image and a PET image;

[0007] Obtaining the diagnosis book label and diagnosis result corresponding to the text-based digital medical record according to the preset language model and the preset questioning method, and obtaining the trained diagnosis book recognition model according to the diagnosis book label and diagnosis result;

[0008] The tissue component recognition model and the diagnosis book recognition model are cross-task optimized by using the MRI images and PET images of the same person and the same body position in the sample medical images, the MRI images and PET images of the same person within a preset time period, the text-based digital medical records of the same person within a preset time period, and the clarity of the sample medical images to obtain the optimized tissue component recognition model and the diagnosis book recognition model;

[0009] The optimized tissue component identification model and the diagnosis certificate identification model are associated through a graph neural network model to obtain a data management model for PET / MRI images, and the data management model for PET / MRI images is used to generate a diagnosis certificate for the entire project.

[0010] Optionally, the obtaining of each tissue component label and tissue component prediction result of the sample medical image through the medical image control system and the enhanced deep residual network model, and obtaining a trained tissue component recognition model according to the tissue component label and tissue component prediction result, includes:

[0011] Generate bone tissue labels and muscle tissue labels of the sample medical image through a medical image control system;

[0012] Obtaining bone tissue feature vectors, bone tissue prediction results, muscle tissue feature vectors, and muscle tissue prediction results of the sample medical image by enhancing the deep residual network model;

[0013] The preset neural network model is trained according to the bone tissue label, the muscle tissue label, the bone tissue feature vector, the bone tissue prediction result, the muscle tissue feature vector and the muscle tissue prediction result to obtain a trained first recognition model nnUnet backbone1;

[0014] Acquire the heart tissue label and the brain tissue label of the sample medical image through the SAM2 segmentation model;

[0015] Obtaining a heart tissue feature vector, a heart tissue prediction result, a brain tissue feature vector, and a brain tissue prediction result of the sample medical image by enhancing a deep residual network model;

[0016] Training a preset neural network model according to the heart tissue label, brain tissue label, heart tissue feature vector, heart tissue prediction result, brain tissue feature vector and brain tissue prediction result to obtain a trained second recognition model nnUnet backbone2;

[0017] The sample medical image is reconstructed according to the first recognition model and the second recognition model, and a tissue component recognition model QLCnet is obtained according to the reconstructed image and the sample medical image.

[0018] Optionally, the step of obtaining a diagnosis book label and a diagnosis result corresponding to a text-based digital medical record according to a preset language model and a preset questioning method, and obtaining a trained diagnosis book recognition model according to the diagnosis book label and the diagnosis result, includes:

[0019] The GPT4 model is used to obtain the diagnosis label corresponding to the preset question format and text-based digital medical records;

[0020] Obtain the diagnosis results corresponding to the preset questioning method and text-based digital medical records through the Llama3 model;

[0021] The preset neural network model is trained according to the diagnosis book label and the diagnosis result to obtain the diagnosis book recognition model QIIT.

[0022] Optionally, the method further includes:

[0023] Changing the expression of the text-based digital medical record to obtain a changed expression medical record;

[0024] According to the similarity between the replaced expression medical record and the text-based digital medical record, a similarity judgment label is obtained;

[0025] The GPT4 model is used to obtain the replacement feature parameters of the replacement expression medical records, the original feature parameters of the text-based digital medical records, and the similarity judgment results;

[0026] The diagnosis book recognition model QIIT is optimized according to the similarity judgment mark, the replaced feature parameters, the original feature parameters and the similarity judgment result.

[0027] Optionally, the tissue component identification model QLCnet includes: an MRI image tissue component identification model QLCnet1 and a PET image tissue component identification model QLCnet2, wherein the tissue component identification model and the diagnosis book identification model are cross-task optimized by using the MRI images and PET images of the same person and body position in the sample medical images, the MRI images and PET images of the same person within a preset time period, the text-based digital medical records of the same person within a preset time period, and the clarity of the sample medical images to obtain the optimized tissue component identification model and the diagnosis book identification model, including:

[0028] Obtaining an MRI image Ormr at time t1 and a PET image Orpet at time t2 of the same person and body position in the sample medical image;

[0029] Generating a fan identification label according to whether the sample medical image is fan-made;

[0030] Generate special vector Vt1, special vector Vt2, and same-person judgment result Omt according to MRI image Ormr and PET image Orpet;

[0031] The MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized by using the same person judgment label, the special vector Vt1, the special vector Vt2, and the same person judgment result Omt.

[0032] Optionally, the method further includes:

[0033] Obtaining text-based digital medical records within a preset short period of time, MRI images Ormr within a preset short period of time, and PET images Orpet within a preset short period of time from the sample medical images;

[0034] Generating a fan identification label according to whether the sample medical image is fan-made;

[0035] Generate a feature value Vlm2 and a judgment result Om2 within a preset short period of time according to the text-based digital medical record within a preset short period of time;

[0036] Generate special vectors Vt1, Vt2, and a judgment result Omt1 within a short period of time preset by the same person according to the MRI image Ormr and the PET image Orpet;

[0037] The diagnosis book recognition model QIIT, MRI image tissue component recognition model QLCnet1 and PET image tissue component recognition model QLCnet2 are optimized by using the same person judgment label, characteristic value Vlm2, judgment result Om2, special vector Vt1, special vector Vt2 and judgment result Omt1.

[0038] Optionally, the method further includes:

[0039] Obtaining an MRI image Ormr1 of the same person within a preset time period and an MRI image Ormr2 of the same person within another preset time period from the sample medical images;

[0040] Generating a fan identification label according to whether the sample medical image is fan-made;

[0041] Generate a special vector Vsmr1, a special vector Vsmr2 and a judgment result Osmr in different preset time periods of the same person according to the MRI image Ormr1 in the preset time period of the same person and the MRI image Ormr2 in another preset time period of the same person;

[0042] The MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized by using the same person judgment label, the special vector Vsmr1, the special vector Vsmr2 and the same person judgment result Osmr in different preset time periods.

[0043] Optionally, the method further includes:

[0044] Obtain an MRI image Tmrx with a clarity level x and a PET image Tpetx with a clarity level x from the sample medical images, where x=0, 1, 2, 3, 4, 5, 6, 7, 8, 9;

[0045] Performing Gaussian-like processing and smoothing and blurring processing on the sample medical image to produce a clarity judgment label;

[0046] Generate a special vector Vtr1, a special vector Vtr2, a clarity judgment result Otmr, and a clarity judgment result Otpet according to the MRI image Tmrx with a clarity level x and the PET image Tpetx with a clarity level x;

[0047] The MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized by using the clarity judgment label, the special vector Vtr1, the special vector Vtr2, the clarity judgment result Otmr and the clarity judgment result Otpet.

[0048] According to a second aspect of the disclosed embodiment of the present invention, a method for data management of PET / MRI images is provided, the method comprising:

[0049] Acquire image data to be identified;

[0050] The image data to be identified is used as the input of the data management model of PET / MRI images, and a full diagnostic project book is obtained according to the output result of the data management model of PET / MR images.

[0051] According to a third aspect of the disclosed embodiment of the present invention, a data governance model training device for PET / MRI images is provided, the device comprising:

[0052] The first training module obtains the tissue component labels and tissue component prediction results of sample medical images through the medical image control system and the enhanced deep residual network model, and obtains the trained tissue component recognition model according to the tissue component labels and tissue component prediction results, wherein the sample medical images include: MRI images and PET images;

[0053] A second training module, connected to the first training module, obtains the diagnosis book label and diagnosis result corresponding to the text-based digital medical record according to a preset language model and a preset questioning method, and obtains a trained diagnosis book recognition model according to the diagnosis book label and diagnosis result;

[0054] an optimization module, connected to the second training module, for performing cross-task optimization on the tissue component recognition model and the diagnosis certificate recognition model through the MRI images and PET images of the same person and body position in the sample medical images, the MRI images and PET images of the same person within a preset time period, the text-based digital medical records of the same person within a preset time period, and the clarity of the sample medical images, to obtain optimized tissue component recognition models and diagnosis certificate recognition models;

[0055] The data management model training module is connected to the optimization module, and the optimized tissue component recognition model and the diagnosis book recognition model are associated through the graph neural network model to obtain the data management model of the PET / MRI image, and the data management model of the PET / MRI image is used to generate the diagnosis book of the whole project. In summary, the technical solution in the disclosed embodiment of the present invention can bring the following beneficial effects:

[0056] (1) AI tools can be used to reduce the gaps in professional doctors’ marking, ensure stable accuracy of diagnostic results, and reduce expenditure costs;

[0057] (2) Through the recognition of PET / MRI images and the use of the body's compensatory functions, more modality evaluation results and more comprehensive diagnostic maps can be obtained to more accurately evaluate human health;

[0058] (3) It can also repair damaged images. Previous PET / MRI images can be stored in the patient's personal medical records to establish a complete patient medical record information, provide a reference for doctors in the diagnosis and treatment process, and further improve the accuracy of the diagnosis results.

[0059] Other features and advantages disclosed in the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0061] Figure 1 is a flow chart of a data governance model training method for PET / MRI images according to an exemplary embodiment;

[0062] Figure 2is based on Figure 1 A flowchart of a nnUnet backbone1 model training method is shown;

[0063] Figure 3 is based on Figure 1 A flowchart of a nnUnet backbone2 model training method is shown;

[0064] Figure 4 is based on Figure 1 A schematic diagram of the process of reconstructing an image to train a QLCnet model is shown;

[0065] Figure 5 is based on Figure 1 A flow chart of a QIIT model training method is shown;

[0066] Figure 6 is based on Figure 1 A schematic diagram of a process of optimizing the QIIT model by changing the expression mode is shown;

[0067] Figure 7 is based on Figure 1 A schematic diagram of a process for optimizing a QLCnet model is shown;

[0068] Figure 8 is based on Figure 1 A schematic diagram of a process of optimizing the QIIT model and the QLCnet model is shown;

[0069] Fig. 9 is based on Figure 1 Another schematic diagram of the process of optimizing the QLCnet model is shown;

[0070] Fig.10 is based on Figure 1 A schematic diagram of another process of optimizing the QLCnet model is shown;

[0071] Fig.11 is based on Figure 1 A schematic diagram of the association relationship of the graph neural network shown;

[0072] Fig.12 is a flow chart of a method for data management of PET / MRI images according to an exemplary embodiment;

[0073] Fig.13 It is a structural block diagram of a data management model training device for PET / MRI images according to an exemplary embodiment. DETAILED DESCRIPTION

[0074] The specific embodiments disclosed in the present invention are described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure.

[0075] Figure 1 FIG. 1 is a flow chart of a method for training a data governance model for PET / MRI images according to an exemplary embodiment. Figure 1 As shown, the method includes:

[0076] In step 101, the tissue component labels and tissue component prediction results of the sample medical image are obtained through the medical image control system and the enhanced deep residual network model, and the trained tissue component recognition model is obtained according to the tissue component labels and tissue component prediction results.

[0077] Among them, the sample medical images include: MRI images and PET images.

[0078] For example, the tissue component recognition model for identifying each tissue component in the sample medical image is trained according to the feature vector extracted from the sample medical image, the prediction result and the produced label. Specifically, a label is made using medical-specific software (preferably a medical image control system MIMICS), and pictures of each tissue component of the sample medical image are generated and the pictures are made into separate labels (for example, PET / MRI image data in DICOM format is interactively read through the basic module in MIMICS, and the data is operated using segmentation and editing tools to select the brain, organs, bones, soft tissues, skin, etc. If a bone is selected, the point is marked as 1 for bone tissue, and 0 for non-bone tissue, and the same is true for other tissue components). Then, each eigenvalue and predicted value (i.e., tissue component prediction result) are obtained respectively through the decode part of EDSRnet (Enhanced Deep Residual Networks). The self-trained QLCnet neural network is used to segment the multi-organ static image of multimodal images using the nnU-Net segmentation backbone. Among them, nnU-Net is a robust and adaptive medical image segmentation framework based on U-Net, which can automatically adjust the network structure, preprocessing, training and inference steps according to different datasets and tasks. The backbone is the main component of the model, usually a convolutional neural network (CNN) or residual neural network (ResNet), etc. The backbone is responsible for extracting the features of the input image for subsequent processing and analysis.

[0079] Specifically, the tissue component labels and tissue component prediction results of the sample medical image are obtained through the medical image control system and the enhanced deep residual network model, and the trained tissue component recognition model is obtained according to the tissue component labels and tissue component prediction results, including: generating the bone tissue label and muscle tissue label of the sample medical image through the medical image control system; obtaining the bone tissue feature vector, bone tissue prediction result, muscle tissue feature vector and muscle tissue prediction result of the sample medical image through the enhanced deep residual network model; training the preset neural network model according to the bone tissue label, muscle tissue label, bone tissue feature vector, bone tissue prediction result, muscle tissue feature vector and muscle tissue prediction result to obtain the trained first recognition model nnUnet backbone1 (the flowchart of the above process is shown in FIG. 1 ). Figure 2 As shown); obtain the heart tissue label and brain tissue label of the sample medical image through the SAM2 segmentation model; obtain the heart tissue feature vector, heart tissue prediction result, brain tissue feature vector and brain tissue prediction result of the sample medical image through the enhanced deep residual network model; train the preset neural network model according to the heart tissue label, brain tissue label, heart tissue feature vector, heart tissue prediction result, brain tissue feature vector and brain tissue prediction result to obtain the trained second recognition model nnUnet backbone2 (the flowchart of the above process is shown in Figure 3 According to the first recognition model and the second recognition model, the sample medical image is reconstructed, and the tissue component recognition model QLCnet is obtained according to the reconstructed image and the sample medical image (the flowchart is shown in Figure 4 shown).

[0080] Sample medical images include MRI images and PET images. In the embodiments disclosed in the present invention, MRI images are used as an example to explain the process of training the nnUnet backbone. The processing of PET images is similar. Figure 2 As shown, labels are made through the medical image control system MIMICS to generate bone tissue labels Y1mr and muscle tissue labels Ynmr; the bone tissue feature vector (bone feature layer V1mr), bone tissue prediction results (bone tissue identification Op1mr), muscle tissue feature vector (muscle feature layer Vnmr) and muscle tissue prediction results (muscle tissue identification Opnmr) of the sample medical image are obtained through the enhanced deep residual network model EDSRnet; according to the bone tissue label, muscle tissue label, bone tissue feature vector, bone tissue prediction results, muscle tissue feature vector and muscle tissue prediction results, the preset neural network model is trained to obtain the trained first recognition model nnUnet backbone1.

[0081] like Figure 3As shown, for heart tissue and brain tissue, the SAM2 segmentation model is used to obtain images and create corresponding heart labels Z1mr and brain labels Zmmr. It should be noted that it is difficult and error-prone to directly use SAM2 to segment medical 3D images. Usually, the 3D image is first converted into a 2D video, and then the SAM2 segmentation model is used to segment the image and create labels. The method of converting 3D to 2D can be understood as: assuming that the lens moves forward at a fixed speed (such as 60mm / s, and the video is recorded at 60 frames / s) to generate a video picture. The enhanced deep residual network model EDSRnet is used to obtain the heart tissue feature vector (heart feature layer Vc1mr), heart tissue prediction result (heart recognition Ot1mr), brain tissue feature vector (brain feature layer Vcmmr) and brain tissue prediction result (brain recognition Otmmr) of the sample medical image; the preset neural network model is trained according to the heart tissue label, brain tissue label, heart tissue feature vector, heart tissue prediction result, brain tissue feature vector and brain tissue prediction result to obtain the trained first recognition model nnUnetbackbone2.

[0082] like Figure 4 As shown in the figure, in order to prevent the neural network from only paying attention to the regular organs and ignoring the small number of very important organ lesions (such as tumors, stones, etc.) or abnormal conditions (such as internal bleeding, mirror people, etc.), the above Figure 2 and Figure 3 The feature parameters trained in the training are added to the EDSRnet model, and the original data is reconstructed to obtain the reconstructed image VOmr. The special vector Vmr1 in the first recognition model nnUnet backbone1 and the special vector Vmr2 in the second recognition model nnUnet backbone2 are input into the EDSRnet backbone to obtain the reconstructed image Vomr, and the optimization training is performed based on the reconstructed image Vomr and the original MRI image (same for PET image) Ormr to obtain the tissue component recognition model QLCnet.

[0083] In step 102, a diagnosis book label and a diagnosis result corresponding to the text-based digital medical record are obtained according to a preset language model and a preset questioning method, and a trained diagnosis book recognition model is obtained according to the diagnosis book label and the diagnosis result.

[0084] For example, since medical data is private and does not need to be universal in other fields, factors such as saving time and computing power can be given priority when selecting a specific model. Usually, an open source language model is used to train a localized private model QllT, taking Llama3 as the backbone. In order to reduce the training cost, the currently more common large models (for example: GPT4, Wenxin Yiyan, etc.) can be used to participate in providing the original training set, and then a small amount of annotations by doctors are converted into the model they need. The preset language model in the disclosed embodiment of the present invention is preferably a GPT4 model.

[0085] Specifically, the GPT4 model is used to obtain the diagnosis book label corresponding to the preset questioning method and the text-type digital medical record; the Llama3 model is used to obtain the diagnosis result corresponding to the preset questioning method and the text-type digital medical record; the preset neural network model is trained according to the diagnosis book label and the diagnosis result to obtain the diagnosis book recognition model QllT.

[0086] For example, Figure 5 As shown, fixed-format questions and text-based digital medical records are input into the GPT4 model to produce diagnosis labels, and obtain labels Ygpt for learning. Among them, fixed-format questions are questions asked to text-based digital medical records through preset questioning methods in the database to obtain corresponding results. For example, "There is no obvious abnormality in brain imaging. No abnormal radioactive concentration is found in the nasopharynx. The muscle morphology of the bilateral posterior neck is acceptable, and radioactive concentration can be seen in PET imaging, with an SUV value of 2.7; multiple small nodules can be seen in the right carotid vascular space, deep neck, lower neck and lock area, with a CT value of 38Hu, some of the boundaries are unclear, and radioactive concentration can be seen in some PET imaging, with an SUV value of 1.8, and the largest is about 1.9x1.1cm. Multiple small nodules are found in the left carotid vascular space, deep neck and lock area, with clear boundaries, and the short diameter does not exceed 1cm. No obvious radioactive concentration is found in PET imaging." What parts of the human body are the above diagnosis books about? Are there obvious lesions? The size of the lesions? Which position is the most serious? Is there an SUV value? Which part has the largest SUV value? And so on.

[0087] In addition, fixed-format questions (preset questioning method) and text-based digital medical records are input into the Llama3 model to obtain the eigenvalue Vlm and the diagnosis result Olm, so as to train the preset neural network model according to the learnable labels Ygpt, eigenvalue Vlm and diagnosis result Olm to obtain the diagnosis book recognition model QllT.

[0088] During training, the loss function is based on cross entropy loss and can be expressed as:

[0089]

[0090] Among them, Wt represents the t-th word, w<t represents the previous (t - 1) words, T represents the sequence length, and P(wt|w<t) represents the probability that the t-th word appears given the previous (t - 1) words.

[0091] It should also be noted that if the digital medical record is not in text form but in picture format, it is necessary to use an OCR recognition model to recognize the text in the picture-based digital medical record and convert it into a text-based digital medical record.

[0092] In addition, the method further includes: changing the expression mode of the text-based digital medical record to obtain a medical record with a changed expression; obtaining a similarity judgment label according to the similarity between the medical record with a changed expression and the text-based digital medical record; obtaining the changed feature parameters of the medical record with a changed expression, the original feature parameters of the text-based digital medical record, and a similarity judgment result through the GPT4 model; and optimizing the diagnosis book recognition model QIIT according to the similarity judgment label, the changed feature parameters, the original feature parameters, and the similarity judgment result.

[0093] Exemplarily, as Figure 6 shown, use GPT4 to change the expression mode of the text-based digital medical record (for example: concise expression, polished expression, exaggerated / downsized condition expression, misdiagnosis expression, etc.), make various labels for the medical record with a changed expression mode, and then submit them to QllT respectively together with the original text-based digital medical record to judge whether they are consistent. If the expression mode is "concise expression" or "polished expression", it is considered consistent, and the label is recorded as 1. If it is "exaggerated / downsized condition expression", it is considered partially consistent, and the label is recorded as 0. If it is "misdiagnosis expression", it is also considered partially consistent, and the label is recorded as -1. In this way, obtain the similarity judgment label Ysim. Obtain the changed feature parameters (feature value Vlm1) of the medical record with a changed expression, the original feature parameters (feature value Vlm2) of the text-based digital medical record, and the similarity judgment result (diagnosis result Osim); optimize the diagnosis book recognition model QIIT according to the similarity judgment label, the changed feature parameters, the original feature parameters, and the similarity judgment result. During the training process, the loss function is the absolute difference between Ysim and Osim.

[0094] In step 103, cross-task optimization is performed on the tissue component recognition model and the diagnosis book recognition model through the MRI image and PET image of the same person in the same position in the sample medical image, the MRI image and PET image of the same person within a preset time period, the text-based digital medical record of the same person within a preset time period, and the clarity of the sample medical image, to obtain an optimized tissue component recognition model and diagnosis book recognition model.

[0095] For example, cross-tasks are used to further improve the feature recognition effect of Qlcn. Since this process involves the comparison and analysis between MRI images and PET images, it will also involve the MRI image tissue component recognition model that has been explained in detail in the above content, as well as the PET image tissue component recognition model omitted in the above content. It can be understood that the tissue component recognition model QLCnet includes the MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2. The above process omits the description of the PET image for the sake of simplicity, so the tissue component recognition model QLCnet is not divided in detail.

[0096] Specifically, Figure 7 As shown, the MRI image Ormr of the same person and the same body position at time t1 and the PET image Orpet at time t2 in the sample medical image are obtained; a same-person judgment label is made according to whether the sample medical image is of the same person; a special vector Vt1, a special vector Vt2, and a same-person judgment result Omt are generated according to the MRI image Ormr and the PET image Orpet; and the MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized through the same-person judgment label, the special vector Vt1, the special vector Vt2, and the same-person judgment result Omt.

[0097] For example, the task of matching different types of images of the same person in the same position is to learn the reasonable range of mapping of different types of images. If the same person has different positions, the PET image will inevitably not match the MRI image, for example, the PET signal appears outside the human body marked by the MRI. Make a same-person judgment label, the label of the same person at the same time is 1, otherwise it is 0. In other words, even if the same person is at different times, the heart and breathing (unconscious micro-tremors, etc.) will make the PET image and the MRI image at other times different. In this case, the label is recorded as 0. Obtain the same-person judgment label, special vector Vt1, special vector Vt2, and the same-person simultaneous judgment result Omt, and optimize the MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2. During the optimization process, the loss function directly takes the absolute difference between the actual label and Omt.

[0098] And, if Figure 8As shown, obtain the text-based digital medical records within a preset short period of time of the same person in the sample medical image, the MRI image Ormr within a preset short period of time, and the PET image Orpet within a preset short period of time; make a same person judgment label according to whether the sample medical image is a same person; generate the characteristic value Vlm2 and the judgment result Om2 within the same person preset short period of time according to the text-based digital medical records within the preset short period of time; generate the special vector Vt1, the special vector Vt2, and the judgment result Omt1 within the same person preset short period of time according to the MRI image Ormr and the PET image Orpet; optimize the diagnosis book recognition model QIIT, the MRI image tissue component recognition model QLCnet1, and the PET image tissue component recognition model QLCnet2 through the same person judgment label, the characteristic value Vlm2, the judgment result Om2, the special vector Vt1, the special vector Vt2, and the judgment result Omt1.

[0099] For example, the same person is matched with different types of images / diagnoses in a short period of time. This task is to learn the relative muscle / joint changes and diagnosis matching after various human actions. When performing optimization training, the loss function takes the absolute difference between the actual label and Om1 / Om2.

[0100] like Fig. 9 As shown, the MRI image Ormr1 within the preset time period of the same person and the MRI image Ormr2 within another preset time period of the same person in the sample medical image are obtained; a same person judgment label is generated according to whether the sample medical image is of the same person; a special vector Vsmr1, a special vector Vsmr2 and a judgment result Osmr within different preset time periods of the same person are generated according to the MRI image Ormr1 within the preset time period of the same person and the MRI image Ormr2 within another preset time period of the same person; the MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized by the same person judgment label, the special vector Vsmr1, the special vector Vsmr2 and the judgment result Osmr within different preset time periods of the same person.

[0101] For example, matching the same type of images of the same person over a long period of time is a task that aims to find fingerprint-like features of the human body that are not greatly affected by time, such as some parameters of bones. When performing optimization training, the loss function takes the absolute difference between the actual label and Osmr.

[0102] like Fig.10As shown, an MRI image Tmrx with a clarity level x and a PET image Tpetx with a clarity level x are obtained from the sample medical images, where x = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9; the sample medical images are subjected to quasi-Gaussian processing and smoothing and blurring processing to produce clarity judgment labels; a special vector Vtr1, a special vector Vtr2, a clarity judgment result Otmr and a clarity judgment result Otpet are generated according to the MRI image Tmrx with a clarity level x and the PET image Tpetx with a clarity level x; the MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized through the clarity judgment labels, the special vector Vtr1, the special vector Vtr2, the clarity judgment result Otmr and the clarity judgment result Otpet.

[0103] For example, MRI images and PET images are sampled. In the sampled samples, the image with the highest clarity is set to 9, and the image with the lowest clarity is set to 0. A fuzzy formula is used to process part of the sampled images to obtain ten levels of clarity: 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9.

[0104] The fuzzy formula is expressed as:

[0105]

[0106] Where: a, b are coefficients (a / b is between 0.5 and 2, b is less than 0.5, and the two values ​​are obtained by actual measurement), σ is the standard deviation, S xyz represents the coordinate group of a rectangular sub-image window with a size of 3*3*3 and a center at point (x, y, z). g(r, s, t) is the value of the pixel in the window, and f 0 (x, y, z) is the value before blurring, and f(x, y, z) is the value after blurring.

[0107] The sampled image with the highest definition is slightly blurred several times until it is judged as the lowest definition, and the intermediate process is divided into ten equal parts, namely, definition 9, 8, ..., 1, 0. After multiple blurs, the step parameters of a, b, and σ are manually determined to automatically generate image values ​​of each definition from the image with the highest definition for optimizing the MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2. During the optimization training process, the loss function is the absolute difference between the actual label and Otmr / Otpet.

[0108] In step 104, the optimized tissue component identification model and the diagnosis certificate identification model are associated through the graph neural network model to obtain the data management model of the PET / MRI image, and the data management model of the PET / MRI image is used to generate the diagnosis certificate for the entire project.

[0109] For example, Fig.11 As shown in the figure, a graph neural network is used to associate various related data to obtain a data governance model for PET / MRI images.

[0110] Fig.12 FIG. 1 is a flow chart of a method for data management of PET / MRI images according to an exemplary embodiment. Fig.12 As shown, the method includes:

[0111] In step 1201, image data to be recognized is obtained.

[0112] In step 1202, the image data to be identified is used as the input of the data management model of the PET / MRI image, and a full diagnosis project book is obtained according to the output result of the data management model of the PET / MRI image.

[0113] Fig.13 is a structural block diagram of a data management model training device for PET / MRI images according to an exemplary embodiment. Fig.13 As shown, the device 1300 includes:

[0114] The first training module 1310 obtains each tissue component label and tissue component prediction result of a sample medical image through a medical image control system and an enhanced deep residual network model, and obtains a trained tissue component recognition model according to the tissue component label and tissue component prediction result, wherein the sample medical image includes: an MRI image and a PET image;

[0115] The second training module 1320 is connected to the first training module 1310, and obtains the diagnosis book label and diagnosis result corresponding to the text-based digital medical record according to the preset language model and the preset questioning method, and obtains the trained diagnosis book recognition model according to the diagnosis book label and the diagnosis result;

[0116] The optimization module 1330 is connected to the second training module 1320, and performs cross-task optimization on the tissue component recognition model and the diagnosis book recognition model through the MRI images and PET images of the same person and the same body position in the sample medical images, the MRI images and PET images of the same person within a preset time period, the text-based digital medical records of the same person within a preset time period, and the clarity of the sample medical images to obtain the optimized tissue component recognition model and the diagnosis book recognition model;

[0117] The data management model training module 1340 is connected to the optimization module 1330, and associates the optimized tissue component identification model and the diagnosis certificate identification model through the graph neural network model to obtain the data management model of the PET / MRI image. The data management model of the PET / MRI image is used to generate the diagnosis certificate for the entire project.

[0118] In summary, the present invention discloses a data management model training method, device and data management method related to PET / MRI images, including: obtaining tissue component labels, tissue component prediction results and tissue component recognition models through a medical image control system and an enhanced deep residual network model; obtaining diagnosis book labels, diagnosis results and diagnosis book recognition models according to a preset language model and a preset questioning method; cross-task optimization of tissue component recognition models and diagnosis book recognition models through the clarity of MRI / PET images of the same person and body position in sample medical images, MRI / PET images of the same person within a preset time period, text-based digital medical records of the same person within a preset time period and sample medical images; associating tissue component recognition models and diagnosis book recognition models through a graph neural network model to obtain a data management model for PET / MRI images. It is possible to obtain more modal evaluation results with the help of the compensatory function between AI tools and the body to more accurately evaluate human health.

[0119] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0120] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0121] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A data management model training method for PET / MRI images, characterized in that: The method comprises: Obtaining each tissue component label and tissue component prediction result of a sample medical image through a medical image control system and an enhanced deep residual network model, and obtaining a trained tissue component recognition model according to the tissue component label and tissue component prediction result, wherein the sample medical image includes: an MRI image and a PET image; Obtaining the diagnosis book label and diagnosis result corresponding to the text-based digital medical record according to the preset language model and the preset questioning method, and obtaining the trained diagnosis book recognition model according to the diagnosis book label and diagnosis result; The tissue component recognition model and the diagnosis book recognition model are cross-task optimized by using the MRI images and PET images of the same person and the same body position in the sample medical images, the MRI images and PET images of the same person within a preset time period, the text-based digital medical records of the same person within a preset time period, and the clarity of the sample medical images to obtain the optimized tissue component recognition model and the diagnosis book recognition model; The optimized tissue component identification model and diagnosis certificate identification model are associated through a graph neural network model to obtain a data management model for PET / MRI images. The data management model for PET / MRI images is used to generate a diagnosis certificate for the entire project.

2. The data management model training method for PET / MRI images according to claim 1, characterized in that: The method of obtaining each tissue component label and tissue component prediction result of the sample medical image through the medical image control system and the enhanced deep residual network model, and obtaining the trained tissue component recognition model according to the tissue component label and tissue component prediction result, includes: Generate bone tissue labels and muscle tissue labels of the sample medical image through a medical image control system; Obtaining bone tissue feature vectors, bone tissue prediction results, muscle tissue feature vectors, and muscle tissue prediction results of the sample medical image by enhancing the deep residual network model; The preset neural network model is trained according to the bone tissue label, the muscle tissue label, the bone tissue feature vector, the bone tissue prediction result, the muscle tissue feature vector and the muscle tissue prediction result to obtain a trained first recognition model nnUnet backbone1; Acquire the heart tissue label and the brain tissue label of the sample medical image through the SAM2 segmentation model; Obtaining a heart tissue feature vector, a heart tissue prediction result, a brain tissue feature vector, and a brain tissue prediction result of the sample medical image by enhancing a deep residual network model; Training a preset neural network model according to the heart tissue label, brain tissue label, heart tissue feature vector, heart tissue prediction result, brain tissue feature vector and brain tissue prediction result to obtain a trained second recognition model nnUnet backbone2; The sample medical image is reconstructed according to the first recognition model and the second recognition model, and a tissue component recognition model QLCnet is obtained according to the reconstructed image and the sample medical image.

3. The data management model training method for PET / MRI images according to claim 2, characterized in that: The method of obtaining a diagnosis book label and a diagnosis result corresponding to a text-based digital medical record according to a preset language model and a preset questioning method, and obtaining a trained diagnosis book recognition model according to the diagnosis book label and the diagnosis result, includes: The GPT4 model is used to obtain the diagnosis label corresponding to the preset question format and text-based digital medical records; Obtain the diagnosis results corresponding to the preset questioning method and text-based digital medical records through the Llama3 model; The preset neural network model is trained according to the diagnosis book label and the diagnosis result to obtain the diagnosis book recognition model QIIT.

4. The data management model training method for PET / MRI images according to claim 3, characterized in that: The method further comprises: Changing the expression of the text-based digital medical record to obtain a changed expression medical record; According to the similarity between the replaced expression medical record and the text-based digital medical record, a similarity judgment label is obtained; The GPT4 model is used to obtain the replacement feature parameters of the replacement expression medical records, the original feature parameters of the text-based digital medical records, and the similarity judgment results; The diagnosis book recognition model QIIT is optimized according to the similarity judgment mark, the replaced feature parameters, the original feature parameters and the similarity judgment result.

5. The data management model training method for PET / MRI images according to claim 4, characterized in that: The tissue component recognition model QLCnet includes: an MRI image tissue component recognition model QLCnet1 and a PET image tissue component recognition model QLCnet2. The cross-task optimization of the tissue component recognition model and the diagnosis book recognition model is performed through the MRI images and PET images of the same person and body position in the sample medical images, the MRI images and PET images of the same person within a preset time period, the text-based digital medical records of the same person within a preset time period, and the clarity of the sample medical images to obtain the optimized tissue component recognition model and the diagnosis book recognition model, including: Obtaining an MRI image Ormr at time t1 and a PET image Orpet at time t2 of the same person and body position in the sample medical image; Generating a fan identification label according to whether the sample medical image is fan-made; Generate special vector Vt1, special vector Vt2, and same-person judgment result Omt according to MRI image Ormr and PET image Orpet; The MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized by using the same person judgment label, the special vector Vt1, the special vector Vt2, and the same person judgment result Omt.

6. The data governance model training method for PET / MRI images according to claim 5, further comprising: Obtaining text-based digital medical records within a preset short period of time, MRI images Ormr within a preset short period of time, and PET images Orpet within a preset short period of time from the sample medical images; Generating a fan identification label according to whether the sample medical image is fan-made; Generate a feature value Vlm2 and a judgment result Om2 within a preset short period of time according to the text-based digital medical record within a preset short period of time; Generate special vectors Vt1, Vt2, and a judgment result Omt1 within a short period of time preset by the same person according to the MRI image Ormr and the PET image Orpet; The diagnosis book recognition model QIIT, MRI image tissue component recognition model QLCnet1 and PET image tissue component recognition model QLCnet2 are optimized by using the same person judgment label, characteristic value Vlm2, judgment result Om2, special vector Vt1, special vector Vt2 and judgment result Omt1.

7. The method for training a data governance model for PET / MRI images according to claim 1, further comprising: Obtaining an MRI image Ormr1 of the same person within a preset time period and an MRI image Ormr2 of the same person within another preset time period from the sample medical images; Generating a fan identification label according to whether the sample medical image is fan-made; Generate a special vector Vsmr1, a special vector Vsmr2 and a judgment result Osmr in different preset time periods of the same person according to the MRI image Ormr1 in the preset time period of the same person and the MRI image Ormr2 in another preset time period of the same person; The MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized by using the same person judgment label, the special vector Vsmr1, the special vector Vsmr2 and the same person judgment result Osmr in different preset time periods.

8. The method for training a data governance model for PET / MRI images according to claim 7, further comprising: Obtain an MRI image Tmrx with a clarity level x and a PET image Tpetx with a clarity level x from the sample medical images, where x=0, 1, 2, 3, 4, 5, 6, 7, 8, 9; Performing Gaussian-like processing and smoothing and blurring processing on the sample medical image to produce a clarity judgment label; Generate a special vector Vtr1, a special vector Vtr2, a clarity judgment result Otmr, and a clarity judgment result Otpet according to the MRI image Tmrx with a clarity level x and the PET image Tpetx with a clarity level x; The MRI image tissue component recognition model QLCnet1 and the PET image tissue component recognition model QLCnet2 are optimized by using the clarity judgment label, the special vector Vtr1, the special vector Vtr2, the clarity judgment result Otmr and the clarity judgment result Otpet.

9. A method for data management of PET / MRI images, characterized in that: The method comprises: Acquire image data to be identified; The image data to be identified is used as the input of the data management model of the PET / MRI image, and a full diagnostic project book is obtained according to the output result of the data management model of the PET / MRI image.

10. A data management model training device for PET / MRI images, characterized in that: The device comprises: The first training module obtains the tissue component labels and tissue component prediction results of sample medical images through the medical image control system and the enhanced deep residual network model, and obtains the trained tissue component recognition model according to the tissue component labels and tissue component prediction results, wherein the sample medical images include: MRI images and PET images; A second training module, connected to the first training module, obtains the diagnosis book label and diagnosis result corresponding to the text-based digital medical record according to a preset language model and a preset questioning method, and obtains a trained diagnosis book recognition model according to the diagnosis book label and diagnosis result; an optimization module, connected to the second training module, for performing cross-task optimization on the tissue component recognition model and the diagnosis certificate recognition model through the MRI images and PET images of the same person and body position in the sample medical images, the MRI images and PET images of the same person within a preset time period, the text-based digital medical records of the same person within a preset time period, and the clarity of the sample medical images, to obtain optimized tissue component recognition models and diagnosis certificate recognition models; The data management model training module is connected to the optimization module, and associates the optimized tissue component identification model and the diagnosis certificate identification model through a graph neural network model to obtain a data management model for PET / MRI images. The data management model for PET / MRI images is used to generate a diagnosis certificate for the entire project.

Images