Medical Image Processing Method, Computer Device, and Storage Medium

Abstract

The present application relates to a medical image processing method, a computer device, and a storage medium. The method includes: inputting a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image of a lesion area in an initial medical image; inputting the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model. Using this method can improve the accuracy of detecting whether a lung nodule has infiltrated.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular, to a medical image processing method, a computer device, and a storage medium. Background Art

[0002] Early detection and early diagnosis of lung cancer are the keys to the prevention and treatment of lung cancer and the improvement of survival rate. At present, low-dose CT (LDCT) screening is the only medical examination method that can effectively reduce the overall mortality rate of lung cancer. Lung nodules in the image can be detected through low-dose CT images, and lung cancer can be detected in a timely manner by detecting whether the lung nodules in the image are infiltrated. However, the detection of whether the lung nodules are infiltrated has high requirements for the quality of medical images.

[0003] In traditional technologies, mainly through image reconstruction algorithms, the conventional CT chest scan images are reconstructed to obtain medical images with higher resolution, so as to use the medical images with higher resolution to detect whether the lung nodules are infiltrated.

[0004] However, the traditional method for detecting whether the lung nodules are infiltrated has the problem of low accuracy. Summary of the Invention

[0005] Based on this, it is necessary to provide a medical image processing method, a computer device, and a storage medium that can improve the accuracy of detecting whether the lung nodules are infiltrated for the above technical problems.

[0006] In a first aspect, the present application provides a medical image processing method, and the method includes:

[0007] Input a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image of a lesion area in the initial medical image;

[0008] Input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0009] In one of the embodiments, the method further includes:

[0010] Obtain the radiomics features and medical clinical features of the second medical image;

[0011] Input the radiomics features, the medical clinical features, and the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image.

[0012] In one embodiment, the classification model includes a feature extraction layer, a fusion layer, and a classification layer; the step of inputting the radiomics features, the medical clinical features, and the second medical image into the preset classification model to obtain a classification result corresponding to the first medical image includes:

[0013] Input the second medical image into the feature extraction layer to obtain features of the second medical image;

[0014] Input the features of the second medical image, the radiomics features, and the medical clinical features into the fusion layer to perform feature fusion on the features of the second medical image, the radiomics features, and the medical clinical features, and obtain fused features;

[0015] Input the fused features into the classification layer to obtain the classification result.

[0016] In one embodiment, the step of inputting the features of the second medical image, the radiomics features, and the medical clinical features into the fusion layer to perform feature fusion on the features of the second medical image, the radiomics features, and the medical clinical features, and obtain fused features includes:

[0017] Input the features of the second medical image, the radiomics features, and the medical clinical features into the fusion layer to perform structured processing on the features of the second medical image, the radiomics features, and the medical clinical features to obtain structured features, and

[0018] Perform dimensionality reduction processing on the structured features to obtain the fused features.

[0019] In one embodiment, the training processes of the generation model and the classification model include:

[0020] Obtain a first sample medical image, a gold standard medical image corresponding to the first sample medical image, and a gold standard classification result corresponding to the first sample medical image; wherein, the first sample medical image is an image of an interested region of the sample medical image; the resolution of the gold standard medical image is higher than that of the first sample medical image;

[0021] Input the first sample medical image into a preset initial generation model to obtain a second sample medical image;

[0022] Based on the second sample medical image and the gold standard medical image, obtain the value of the first loss function of the initial generation model;

[0023] Input the second sample medical image into a preset initial classification model to obtain a sample classification result corresponding to the first sample medical image;

[0024] Based on the sample classification result and the gold standard classification result, obtain the value of the second loss function of the initial classification model;

[0025] Determine the weighted sum of the value of the first loss function and the value of the second loss function as the value of the target loss function;

[0026] Train the initial generation model and the initial classification model according to the value of the target loss function to obtain the classification model and the generation model.

[0027] In one embodiment, the method further includes:

[0028] Segment the lesion area in the initial medical image to obtain the first medical image.

[0029] In one embodiment, the obtaining the radiomics features and medical clinical features of the second medical image includes:

[0030] Use a preset feature extraction algorithm to extract features from the second medical image to obtain the radiomics features;

[0031] Obtain the medical clinical features from the inspection report corresponding to the second medical image.

[0032] In one embodiment, the method further includes:

[0033] Generate a third medical image for the image in the initial medical image except the region of interest by using the nearest neighbor algorithm; the resolution of the third medical image is the same as the resolution of the second medical image;

[0034] Perform a splicing process on the second medical image and the third medical image to generate a fourth medical image corresponding to the initial medical image.

[0035] In a second aspect, the present application further provides a medical image processing device, and the device includes:

[0036] A first acquisition module, configured to input a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than the resolution of the first medical image; the first medical image is a lesion area image in the initial medical image;

[0037] A second acquisition module, configured to input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by performing cascaded training on the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0038] In a third aspect, the present application further provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0039] Input a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image of a lesion region in an initial medical image;

[0040] Input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by performing cascaded training on the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0042] Input a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image of a lesion region in an initial medical image;

[0043] Input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by performing cascaded training on the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0044] In a fifth aspect, the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0045] Input a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image of a lesion area in an initial medical image.

[0046] Input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0047] In the above medical image processing method, computer device and storage medium, inputting a first medical image into a preset generation model can obtain a second medical image with a resolution higher than that of the first medical image, and inputting the second medical image into a preset classification model can obtain a classification result corresponding to the first medical image. Since the first medical image is an image of a lesion area in an initial medical image and the resolution of the second medical image input into the preset classification model is relatively high, therefore, inputting the second medical image into the classification model can accurately obtain the classification result corresponding to the first medical image, thereby improving the accuracy of the obtained classification result corresponding to the first medical image. Description of the Drawings

[0048] Figure 1 It is an application environment diagram of the medical image processing method in an embodiment.

[0049] Figure 2 It is a schematic flowchart of the medical image processing method in an embodiment.

[0050] Figure 3 It is a schematic flowchart of the medical image processing method in another embodiment.

[0051] Figure 4 It is a schematic flowchart of the medical image processing method in another embodiment.

[0052] Figure 5 It is a schematic flowchart of the medical image processing method in another embodiment.

[0053] Figure 6 It is a schematic flowchart of the medical image processing method in another embodiment.

[0054] Figure 7 It is a schematic flowchart of the medical image processing method in an embodiment.

[0055] Figure 8Schematic flowchart of a medical image processing method in an embodiment;

[0056] Figure 9 Block diagram of the structure of a medical image processing device in an embodiment;

[0057] Figure 10 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0058] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] The medical image processing method provided by the embodiments of the present application can be applicable to a computer device as shown in Figure 1 The computer device includes a processor and a memory connected through a system bus. A computer program is stored in the memory. When the processor executes the computer program, it can execute the steps of the following method embodiments. Optionally, the computer device may further include a network interface, a display screen, and an input device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. Optionally, the computer device may be a server, a personal computer, a personal digital assistant, or other terminal devices, such as a tablet computer, a mobile phone, etc., or may be a cloud or a remote server. The specific form of the computer device is not limited in the embodiments of the present application.

[0060] In one embodiment, as shown in Figure 2 A medical image processing method is provided. Taking the method applied to the computer device in Figure 1 as an example, the method includes the following steps:

[0061] S201, input a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image of a lesion area in the initial medical image.

[0062] Among them, the first medical image is the image of the lesion area in the initial medical image. Optionally, the initial medical image can be a chest scan image, an abdominal scan image, or a scan image of other parts. Optionally, the initial medical image can be a Computed Tomography (CT) image, a Magnetic Resonance Imaging (MRI) image, etc. Optionally, the above-mentioned lesion area can be the lungs, etc. The resolution of a medical image refers to the amount of information stored in the medical image, that is, how many pixel points are there per inch in the medical image. The resolution of the second medical image being higher than that of the first medical image means that there are more pixel points per inch in the second medical image than in the first medical image per inch of the medical image.

[0063] Optionally, the preset generation model can be obtained by training the initial generation model with the conventional CT image of the chest as the sample image and the target scan CT image of the chest as the gold standard image. Optionally, the generation model can be any one of the V-Net network, the DenseNet network, and the Generative Adversarial Networks (GAN).

[0064] Optionally, in this embodiment, the first medical image can be a two-dimensional image or a three-dimensional image. Optionally, the first medical image can be directly obtained from a PACS (Picture Archiving and Communication Systems) server, or it can be the first medical image obtained by segmenting the lesion area in the initial medical image retrieved from the PACS server.

[0065] S202: Input the second medical image into the preset classification model to obtain the classification result corresponding to the first medical image; among them, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0066] Among them, the preset classification model can be any one of the V-Net network, the DenseNet network, and the GAN network. Optionally, the lesion type of the region of interest in the first medical image can be any one of infiltration, non-infiltration, micro-infiltration, etc.

[0067] Among them, the above-mentioned generation model and classification model are obtained by jointly performing cascaded training on the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model. It can be understood that since the resolution of the second medical image generated by the generation model is relatively high, when using the second medical image generated by the generation model for classification, the cascaded generation model and classification model will simultaneously reach the optimal for the second medical image and the classification result. In addition, since the resolution of the second medical image input into the preset classification model is relatively high, the classification model can more accurately obtain the features in the second medical image, so that the classification result corresponding to the first medical image can be accurately obtained.

[0068] In the above-mentioned medical image processing method, when the first medical image is input into the preset generation model, a second medical image with a resolution higher than that of the first medical image can be obtained. When the second medical image is input into the preset classification model, the classification result corresponding to the first medical image can be obtained. Since the first medical image is the lesion area image in the initial medical image and the resolution of the second medical image input into the preset classification model is relatively high, therefore, inputting the second medical image into the classification model can accurately obtain the classification result corresponding to the first medical image, thereby improving the accuracy of the classification result corresponding to the obtained first medical image.

[0069] In some scenarios, the computer device can also obtain the radiomics features and medical clinical features of the second medical image, and obtain the classification result corresponding to the first medical image by combining the radiomics features and medical clinical features of the second medical image. In one embodiment, as Figure 3 shown, the above method further includes:

[0070] S301, obtaining the radiomics features and medical clinical features of the second medical image.

[0071] Among them, the radiomics features include first-order statistical features, shape features, texture features, etc.; the medical clinical features include blood test features, urine test features, pathological features, gene features, and so on. Optionally, the computer device can use a preset feature extraction algorithm to extract the features of the second medical image to obtain the radiomics features of the second medical image. It can be understood that the medical image report includes various feature information corresponding to the medical image. Therefore, as an optional implementation manner, the computer device can read the medical clinical features of the second medical image from the inspection report corresponding to the second medical image.

[0072] S302, inputting the radiomics features, medical clinical features, and the second medical image into the preset classification model to obtain the classification result corresponding to the first medical image.

[0073] Optionally, after the computer device inputs the radiomics features, medical clinical features, and the second medical image of the second medical image into a preset classification model, the classification model can extract the features of the second medical image, and then perform feature fusion on the extracted features of the second medical image, the radiomics features of the second medical image, and the medical clinical features, and obtain the classification result corresponding to the first medical image by using the fused features. Optionally, the classifier in the classification model can be used to classify the fused features to obtain the classification result corresponding to the first medical image.

[0074] In this embodiment, by obtaining the radiomics features and medical clinical features of the second medical image, the computer device can input the radiomics features, medical clinical features, and the second medical image of the second medical image into a preset classification model. Since what is input into the preset classification model are the radiomics features, medical clinical features, and the second medical image, the classification model can combine the radiomics features, medical clinical features, and the second medical image to classify the lesion type of the region of interest in the first medical image. Since the feature information used for classification is relatively rich, the classification result corresponding to the first medical image can be accurately obtained, improving the accuracy of the classification result corresponding to the first medical image.

[0075] In the above scenario where the radiomics features, medical clinical features, and the second medical image of the second medical image are input into a preset classification model to obtain the classification result corresponding to the first medical image, the above classification model includes a feature extraction layer, a fusion layer, and a classification layer. In one embodiment, as Figure 4 shown, the above S302 includes:

[0076] S401: Input the second medical image into the feature extraction layer to obtain the features of the second medical image.

[0077] Specifically, the feature extraction layer in the classification model extracts the features of the input second medical image to obtain the features of the second medical image. Optionally, the feature extraction layer can extract the features of the second medical image by using the principal component analysis method, or the feature extraction layer can extract the features of the second medical image by using the gray-level co-occurrence matrix method.

[0078] S402: Input the features of the second medical image, the radiomics features, and the medical clinical features into the fusion layer to perform feature fusion on the features of the second medical image, the radiomics features, and the medical clinical features, and obtain the fused features.

[0079] Optionally, after inputting the features of the second medical image, the radiomics features of the second medical image, and the medical clinical features of the second medical image into the above fusion layer, the fusion layer can perform structured processing on the features, radiomics features, and medical clinical features of the second medical image to obtain structured features, and perform dimensionality reduction processing on the obtained structured features to obtain the fused features.

[0080] S403. Input the fused features into the classification layer to obtain a classification result.

[0081] Specifically, after obtaining the fused features through the above fusion layer, the obtained fused features can be input into the classification layer of the classification model to obtain the classification result corresponding to the first medical image. Optionally, the classification layer of the classification model can be a classifier, and the classification result corresponding to the first medical image can be obtained through this classifier.

[0082] In this embodiment, by inputting the second medical image into the feature extraction layer of the classification model, the features of the second medical image can be extracted. By inputting the features of the second medical image, the radiomics features of the second medical image, and the medical clinical features into the fusion layer of the classification model, the feature fusion of the features of the second medical image, the radiomics features of the second medical image, and the medical clinical features can be performed to obtain fused features. Since the information included in the fused features is richer than that of the single features, inputting the fused features into the classification layer of the classification model can accurately classify the fused features through the classification layer, accurately obtain the classification result corresponding to the first medical image, and improve the accuracy of the obtained classification result corresponding to the first medical image.

[0083] Further, as Figure 5 shown, in one embodiment, the training processes of the above generation model and classification model include:

[0084] S501. Obtain the first sample medical image, the gold standard medical image corresponding to the first sample medical image, and the gold standard classification result corresponding to the first sample medical image; wherein, the first sample medical image is the image of the region of interest of the sample medical image; the resolution of the gold standard medical image is higher than that of the first sample medical image.

[0085] Optionally, in this embodiment, the computer device may obtain the sample medical image, the first sample medical image, the gold standard medical image corresponding to the first sample medical image, and the gold standard classification result corresponding to the first sample medical image from the PACS server. Alternatively, the computer device may obtain the sample medical image, the gold standard medical image corresponding to the sample medical image, and the gold standard classification result corresponding to the sample medical image from the PACS server, divide the region of interest of the obtained sample medical image to obtain the above-mentioned first sample medical image, and use the obtained gold standard classification result corresponding to the sample medical image as the gold standard classification result corresponding to the first sample medical image. Optionally, the first sample medical image may be a medical image obtained by conventional scanning, and the gold standard medical image corresponding to the first medical image may be a medical image obtained by target scanning.

[0086] S502. Input the first sample medical image into a preset initial generation model to obtain a second sample medical image.

[0087] Specifically, after the computer device inputs the first sample medical image into the preset initial generation model, it may use the initial generation model to reconstruct the first sample medical image to obtain a second sample medical image with a resolution higher than that of the first sample medical image.

[0088] S503. Obtain the value of the first loss function of the initial generation model according to the second sample medical image and the gold standard medical image.

[0089] Optionally, the computer device may obtain the value of the first loss function of the initial generation model by calculating the similarity between the second sample medical image and the above-mentioned gold standard medical image. Alternatively, the computer device may obtain the value of the first loss function of the initial generation model through the difference value between the pixel values of each pixel of the second sample medical image and the pixel values of each pixel of the gold standard medical image.

[0090] S504. Input the second sample medical image into a preset initial classification model to obtain the sample classification result corresponding to the first sample medical image.

[0091] Optionally, the computer device may also extract features from the second sample medical image to obtain the radiomics features of the second sample medical image, and obtain the medical clinical features of the second sample medical image from the inspection report corresponding to the second sample medical image. Then, the second sample medical image, the radiomics features of the second sample medical image, and the medical clinical features of the second sample medical image are input into the preset initial classification model to obtain the sample classification result corresponding to the first sample medical image.

[0092] S505. Obtain the value of the second loss function of the initial classification model according to the sample classification result and the gold standard classification result.

[0093] Optionally, the computer device can obtain the value of the second loss function of the initial classification model by using the difference value between the sample classification result obtained through calculation and the obtained gold standard classification result.

[0094] S506. Determine the value of the target loss function as the weighted sum of the value of the first loss function and the value of the second loss function.

[0095] Optionally, the computer device can calculate the weighted sum of the value of the first loss function and the value of the second loss function according to the weight corresponding to the first loss function and the weight corresponding to the second loss function, and determine the weighted sum of the value of the first loss function and the value of the second loss function as the value of the target loss function.

[0096] S507. Train the initial generation model and the initial classification model according to the value of the target loss function to obtain the classification model and the generation model.

[0097] Optionally, the computer device can use the value of the target loss function to adjust the parameters of the initial generation model and the parameters of the initial classification model, and train the initial generation model and the initial classification model to obtain the above-mentioned classification model and generation model.

[0098] In this embodiment, when the computer device inputs the first sample medical image into the preset initial generation model, it can obtain the second sample medical image. According to the second sample medical image and the gold standard medical image corresponding to the first sample medical image, it can obtain the value of the first loss function of the initial generation model. Inputting the second sample medical image into the preset initial classification model can obtain the sample classification result corresponding to the first sample medical image. Thus, the value of the second loss function of the initial classification model can be obtained according to the sample classification result corresponding to the first sample medical image and the gold standard classification result corresponding to the first sample medical image. Furthermore, the weighted sum of the value of the first loss function and the value of the second loss function can be determined as the value of the target loss function. Since the value of the target loss function is the weighted sum of the value of the first loss function and the value of the second loss function, it can achieve the effect of jointly training the initial generation model and the initial classification model, improve the accuracy of training the initial generation model and the initial classification model according to the value of the target loss function, and thus accurately obtain the classification model and the generation model.

[0099] In the scenario of inputting the first medical image into the preset generation model as described above, it is necessary to first obtain the first medical image. In one embodiment, the above method further includes: segmenting the lesion area in the initial medical image to obtain the first medical image.

[0100] Optionally, the computer device may first locate the lesion area in the initial medical image, and use a preset segmentation algorithm to segment the lesion area in the initial medical image to obtain the first medical image. For example, the computer device may segment the lesion area in the initial medical image according to the size of the lesion area in the initial medical image, using a sliding window of the same size as the size, to obtain the first medical image.

[0101] In this embodiment, the process of the computer device segmenting the lesion area in the initial medical image is very simple. Therefore, the computer device can quickly obtain the first medical image, improving the efficiency of obtaining the first medical image.

[0102] In some scenarios, when doctors view films, they may need complete medical images with a relatively high resolution. In one embodiment, as Figure 6 shown, the above method further includes:

[0103] S601, generating a third medical image for the image in the initial medical image except for the region of interest by using the nearest neighbor algorithm; the resolution of the third medical image is the same as the resolution of the second medical image.

[0104] Specifically, the computer device may determine the image in the initial medical image except for the region of interest, and generate a third medical image for the image in the initial medical image except for the region of interest by using the nearest neighbor algorithm. Among them, the resolution of the generated third medical image is the same as the resolution of the second medical image above, that is, it can adjust the resolution of the image in the initial medical image except for the region of interest, so that the resolution of the image in the initial medical image except for the region of interest is the same as the resolution of the region of interest in the initial medical image.

[0105] S602, performing a splicing process on the second medical image and the third medical image to generate a fourth medical image corresponding to the initial medical image.

[0106] Specifically, the computer device may perform a splicing process on the obtained third medical image and the second medical image above to generate a fourth medical image corresponding to the initial medical image. It can be understood that since the resolution of the third medical image is the same as the resolution of the second medical image, and the resolution of the second medical image is higher than the resolution of the first medical image, the resolution of the obtained fourth medical image is higher than the resolution of the initial medical image.

[0107] In this embodiment, for the images in the initial medical image except the region of interest, the nearest neighbor algorithm can be used to generate a third medical image, so that the second medical image and the third medical image can be stitched together to generate a fourth medical image with a resolution higher than that of the initial medical image. This avoids the need for doctors to perform a target scan on the patient again when using a medical image with a higher resolution, thus exposing the patient to radiation again. At the same time, this processing process is relatively simple, enabling the computer device to quickly generate the fourth medical image corresponding to the initial medical image.

[0108] For the convenience of those skilled in the art, the following provides a detailed introduction to the medical image processing method provided by this application. Please refer to Figure 7 and Figure 8 , and this method may include:

[0109] S1. Obtain a first sample medical image, a gold standard medical image corresponding to the first sample medical image, and a gold standard classification result corresponding to the first sample medical image; wherein, the first sample medical image is the image of the region of interest of the sample medical image; the resolution of the gold standard medical image is higher than that of the first sample medical image.

[0110] S2. Input the first sample medical image into a preset initial generation model to obtain a second sample medical image.

[0111] S3. Obtain the value of the first loss function of the initial generation model according to the second sample medical image and the gold standard medical image.

[0112] S4. Input the second sample medical image into a preset initial classification model to obtain a sample classification result corresponding to the first sample medical image.

[0113] S5. Obtain the value of the second loss function of the initial classification model according to the sample classification result and the gold standard classification result.

[0114] S6. Determine the weighted sum of the value of the first loss function and the value of the second loss function as the value of the target loss function.

[0115] S7. Train the initial generation model and the initial classification model according to the value of the target loss function to obtain a classification model and a generation model.

[0116] S8. Segment the lesion area in the initial medical image to obtain a first medical image.

[0117] S9. Input the first medical image into a preset generation model to obtain a second medical image.

[0118] S10. Use a preset feature extraction algorithm to extract features from the second medical image to obtain radiomics features.

[0119] S11. Obtain medical clinical features from the inspection report corresponding to the second medical image.

[0120] S12. Input the second medical image into the feature extraction layer of the classification model to obtain the features of the second medical image.

[0121] S13. Input the features of the second medical image, the radiomics features, and the medical clinical features into the fusion layer of the classification model to perform structured processing on the features of the second medical image, the radiomics features, and the medical clinical features to obtain structured features, and perform dimensionality reduction processing on the structured features to obtain the fused features.

[0122] S14. Input the fused features into the classification layer of the classification model to obtain the classification result corresponding to the first medical image.

[0123] S15. For the image in the initial medical image except the region of interest, generate a third medical image using the nearest neighbor algorithm; the resolution of the third medical image is the same as that of the second medical image; splice the second medical image and the third medical image to generate a fourth medical image corresponding to the initial medical image.

[0124] It should be noted that for the descriptions in S1 - S15 above, reference can be made to the relevant descriptions in the above embodiments, and their effects are similar, so they will not be elaborated herein.

[0125] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0126] Based on the same inventive concept, an embodiment of the present application also provides a medical image processing device for implementing the medical image processing method involved above. The solution provided by this device to solve the problem is similar to the solution recorded in the above method. Therefore, the specific limitations in one or more embodiments of the medical image processing device provided below can refer to the limitations on the medical image processing method in the above text, and will not be elaborated herein.

[0127] In one embodiment, as Figure 9As shown in the figure, a medical image processing device is provided, including: a first acquisition module and a second acquisition module, where:

[0128] The first acquisition module is used to input the first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image of a lesion area in the initial medical image.

[0129] The second acquisition module is used to input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0130] The medical image processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0131] On the basis of the above embodiment, optionally, the above device further includes: a third acquisition module and a fourth acquisition module, where:

[0132] The third acquisition module is used to acquire the radiomics features and medical clinical features of the second medical image.

[0133] The fourth acquisition module is used to input the radiomics features, medical clinical features and the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image.

[0134] The medical image processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0135] On the basis of the above embodiment, optionally, the above classification model includes a feature extraction layer, a fusion layer and a classification layer; the above fourth acquisition module includes: a first acquisition unit, a fusion unit and a second acquisition unit, where:

[0136] The first acquisition unit is used to input the second medical image into the feature extraction layer to obtain the features of the second medical image.

[0137] The fusion unit is used to input the features of the second medical image, radiomics features and medical clinical features into the fusion layer to perform feature fusion on the features of the second medical image, radiomics features and medical clinical features to obtain the fused features.

[0138] The second acquisition unit is used to input the fused features into the classification layer to obtain the classification result.

[0139] The medical image processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0140] Based on the above embodiment, optionally, the above fusion unit is configured to input the features of the second medical image, the radiomics features, and the medical clinical features into the fusion layer, perform structured processing on the features of the second medical image, the radiomics features, and the medical clinical features to obtain structured features, and perform dimensionality reduction processing on the structured features to obtain the fused features.

[0141] The medical image processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0142] Based on the above embodiment, optionally, the above device further includes: a fifth acquisition module, a sixth acquisition module, a seventh acquisition module, an eighth acquisition module, a ninth acquisition module, a determination module, and a training module, where:

[0143] The fifth acquisition module is configured to acquire a first sample medical image, a gold standard medical image corresponding to the first sample medical image, and a gold standard classification result corresponding to the first sample medical image; wherein, the first sample medical image is a region of interest image of the sample medical image; the resolution of the gold standard medical image is higher than that of the first sample medical image.

[0144] The sixth acquisition module is configured to input the first sample medical image into a preset initial generation model to obtain a second sample medical image.

[0145] The seventh acquisition module is configured to obtain the value of the first loss function of the initial generation model according to the second sample medical image and the gold standard medical image.

[0146] The eighth acquisition module is configured to input the second sample medical image into a preset initial classification model to obtain a sample classification result corresponding to the first sample medical image.

[0147] The ninth acquisition module is configured to obtain the value of the second loss function of the initial classification model according to the sample classification result and the gold standard classification result.

[0148] The determination module is configured to determine the weighted sum of the value of the first loss function and the value of the second loss function as the value of the target loss function.

[0149] The training module is configured to train the initial generation model and the initial classification model according to the value of the target loss function to obtain a classification model and a generation model.

[0150] The medical image processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0151] Based on the above embodiments, optionally, the above device further includes: a segmentation module, where:

[0152] The segmentation module is configured to segment the lesion area in the initial medical image to obtain a first medical image.

[0153] The medical image processing device provided in this embodiment can execute the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0154] Based on the above embodiments, optionally, the above third acquisition module includes: a feature extraction module and a third acquisition unit, where:

[0155] The feature extraction module is configured to extract features from the second medical image using a preset feature extraction algorithm to obtain radiomics features.

[0156] The third acquisition unit is configured to obtain medical clinical features from the inspection report corresponding to the second medical image.

[0157] The medical image processing device provided in this embodiment can execute the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0158] Based on the above embodiments, optionally, the above device further includes: a generation module and a splicing module, where:

[0159] The generation module is configured to generate a third medical image for the image in the initial medical image except for the region of interest using the nearest neighbor algorithm; the resolution of the third medical image is the same as that of the second medical image.

[0160] The splicing module is configured to splice the second medical image and the third medical image to generate a fourth medical image corresponding to the initial medical image.

[0161] The medical image processing device provided in this embodiment can execute the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0162] Each module in the above medical image processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0163] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 10As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements a medical image processing method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0164] Those skilled in the art can understand that Figure 10 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0165] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0166] Input a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image of the lesion area in the initial medical image;

[0167] Input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0168] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0169] Input the first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is the lesion area image in the initial medical image;

[0170] Input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0171] In one embodiment, a computer program product is provided, including a computer program, which when executed by a processor implements the following steps:

[0172] Input the first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is the lesion area image in the initial medical image;

[0173] Input the second medical image into a preset classification model to obtain a classification result corresponding to the first medical image; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the values of the loss functions of the initial generation model and the initial classification model.

[0174] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0175] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0176] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0177] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A medical image processing method, characterized in that, The method includes: Inputting a first medical image into a preset generation model to obtain a second medical image; the resolution of the second medical image is higher than that of the first medical image; the first medical image is an image obtained by segmenting a lesion area in an initial medical image; the generation model is trained based on a gold standard medical image corresponding to a first sample medical image; the gold standard medical image is an image obtained by target scanning and having a resolution higher than that of the first sample medical image; Inputting the second medical image into a feature extraction layer in a preset classification model to obtain features of the second medical image; Inputting the features of the second medical image, the radiomics features of the second medical image, and medical clinical features into a fusion layer in the classification model to perform structured processing on the features of the second medical image, the radiomics features, and the medical clinical features to obtain structured features, and performing dimensionality reduction processing on the structured features to obtain the fused features; Obtaining a classification result corresponding to the first medical image according to the fused features; wherein, the classification result is used to characterize the lesion type of the region of interest in the first medical image; the generation model and the classification model are obtained by cascaded training of the initial generation model and the initial classification model according to the weighted sum of the values of the loss function of the initial generation model and the value of the loss function of the initial classification model; Generating a third medical image for the image in the initial medical image except the lesion area by using the nearest neighbor algorithm; the resolution of the third medical image is the same as that of the second medical image; Performing stitching processing on the second medical image and the third medical image to generate a fourth medical image corresponding to the initial medical image; the resolution of the fourth medical image is higher than that of the initial medical image.

2. The method according to claim 1, characterized in that The classification model further includes a classification layer; the obtaining the classification result corresponding to the first medical image according to the fused features includes: Inputting the fused features into the classification layer to obtain the classification result.

3. The method according to claim 1 or 2, characterized in that, The training process of the generation model and the classification model includes: Obtaining a first sample medical image, the gold standard medical image corresponding to the first sample medical image, and the gold standard classification result corresponding to the first sample medical image; wherein, the first sample medical image is an image of the region of interest of the sample medical image; the resolution of the gold standard medical image is higher than that of the first sample medical image; Inputting the first sample medical image into a preset initial generation model to obtain a second sample medical image; Obtaining the value of the first loss function of the initial generation model according to the second sample medical image and the gold standard medical image; Inputting the second sample medical image into a preset initial classification model to obtain a sample classification result corresponding to the first sample medical image; Obtaining the value of the second loss function of the initial classification model according to the sample classification result and the gold standard classification result; Determine the weighted sum of the values of the first loss function and the second loss function as the value of the target loss function; Train the initial generation model and the initial classification model according to the value of the target loss function to obtain the classification model and the generation model.

4. The method according to claim 3, wherein The obtaining the value of the first loss function of the initial generation model according to the second sample medical image and the gold standard medical image includes: Obtain the value of the first loss function according to the similarity between the second sample medical image and the gold standard medical image.

5. The method according to claim 1, characterized in that The method further includes: Obtain the size of the lesion area in the initial medical image, and use a sliding window with the same size to segment the lesion area in the initial medical image to obtain the first medical image.

6. The method according to claim 1, characterized in that The method further includes: Use a preset feature extraction algorithm to extract features from the second medical image to obtain the radiomics features; Obtain the medical clinical features from the inspection report corresponding to the second medical image.

7. The method according to claim 6, characterized in that, The radiomics features include at least one of first-order statistical features, shape features, and texture features; the medical clinical features include at least one of blood test features, urine test features, pathological features, and gene features.

8. The method according to claim 1, wherein The feature extraction layer extracts features from the second medical image by principal component analysis or gray-level co-occurrence matrix method.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 8.

Images