Deep Learning-Based Image Processing Method and System for Ovarian Cancer

Abstract

The present invention provides an ovarian cancer image processing method and system based on deep learning. The ovarian cancer image processing method includes the following steps: obtaining a first test ovarian tumor image and forming a first mixed image through image mixing; inputting the first mixed image into four different classification neural network models respectively to determine the optimal classification neural network model; obtaining a second test ovarian tumor image and forming a second mixed image through lesion mixing; inputting the second mixed image into four different segmentation neural network models respectively to determine the optimal segmentation neural network model; integrating the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model; classifying the CT-PET ovarian cancer image based on the optimal classification neural network model and performing lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model. After adopting the above technical solution, image processing of ovarian cancer images can be performed under optimal conditions.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and particularly to an ovarian cancer image processing method and system based on deep learning. Background Art

[0002] With the rapid development of computer science and technology, big data and artificial intelligence have gradually come into people's view and become a popular research direction in scientific research in various fields. As an important field closely related to human health, the medical field has great significance and good prospects for interdisciplinary cross - cooperation with artificial intelligence research. Thus, intelligent diagnostic systems have emerged. Medical images, as the most important expression form of patients' pathological features, are not only the direct basis for doctors to diagnose diseases and judge the lesion area, but also the key object of image processing research. It is mentioned in the "White Paper on Artificial Intelligence in Healthcare" that it is necessary to use intelligent medical image analysis technology to study extremely rich medical image data. However, in the traditional diagnosis and treatment process, due to the variety of medical images and different analysis techniques, the cultivation of doctors' ability to analyze and read images requires a large amount of resources and time. In addition, the large amount of medical image data may also consume excessive physical and mental strength of doctors during the analysis and interpretation process, resulting in misdiagnosis and missed diagnosis. If artificial intelligence can be well applied in medical image processing, it can not only obtain deep features that are not easily detected by the naked eye in medical images, but also save doctors' time and energy and reduce the occurrence of misdiagnosis and missed diagnosis. This undoubtedly has great significance and research value for the research in the medical field and the artificial intelligence field.

[0003] When diagnosing ovarian cancer, doctors use CT - PET images to judge the lesion area and benign or malignant nature of patients. The application of deep learning algorithms in multi - modal medical image - assisted diagnosis systems has the following advantages: First, as a data - driven automatic feature learning algorithm, it can directly extract features from training data, thus greatly reducing the workload of feature extraction and the influence of artificial features; Second, it can represent the intersection of features through the deep structure of the neural network. Third, in the optimization of the same deep structure, the three core steps of feature extraction, feature selection, and feature classification can be achieved. Therefore, deep learning is expected to solve the problem of insufficient capabilities of traditional shallow machine learning, thus greatly improving the ability of assisted diagnosis. Existing research results show that compared with traditional feature extraction algorithms, deep learning algorithms are not affected by interference and have good robustness. If deep learning algorithms are used for intelligent diagnosis of CT - PET images to assist doctors in achieving lesion segmentation and benign or malignant determination, it will be of great significance in the diagnosis of ovarian cancer.

[0004] Therefore, a new type of ovarian cancer image processing method and system based on deep learning is needed, which can use the optimal deep learning algorithm for intelligent diagnosis of CT - PET images. Summary of the Invention

[0005] In order to overcome the above technical defects, the purpose of the present invention is to provide an ovarian cancer image processing method and system based on deep learning, which can perform image processing on ovarian cancer images under optimal conditions.

[0006] The present invention discloses an ovarian cancer image processing method based on deep learning, including the following steps:

[0007] Obtain at least 2 first test ovarian tumor images, and form a first mixed image after image mixing;

[0008] Input the first mixed image into four different classification neural network models respectively, perform five-fold cross-validation, and determine the optimal classification neural network model according to the precision rate of the classification evaluation criteria of each classification neural network model;

[0009] Obtain at least 2 second test ovarian tumor images, and form a second mixed image after lesion mixing;

[0010] Input the second mixed image into four different segmentation neural network models respectively, perform five-fold cross-validation, and determine the optimal segmentation neural network model according to the precision rate of the segmentation evaluation criteria of each segmentation neural network model;

[0011] Integrate the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model;

[0012] Classify CT-PET ovarian cancer images based on the optimal classification neural network model, and perform lesion segmentation on CT-PET ovarian cancer images based on the final segmentation neural network model.

[0013] Preferably, the step of inputting the first mixed image into four different classification neural network models respectively, performing five-fold cross-validation, and determining the optimal classification neural network model according to the precision rate of the classification evaluation criteria of each classification neural network model includes:

[0014] Input the first mixed image into the ConvNeXt classification neural network model, Swin-Transformer classification neural network model, DenseNet classification neural network model, and EfficientNet classification neural network model respectively;

[0015] The ConvNeXt classification neural network model, Swin-Transformer classification neural network model, DenseNet classification neural network model, and EfficientNet classification neural network model respectively randomly divide the first mixed image into 5 parts;

[0016] For the ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model, one of them is selected as the test set respectively, and the remaining four are used as the training set for model training. Then, the selection is repeated 4 times, where one of the unselected test sets is selected as the test set, and the remaining four are used as the training set for training;

[0017] In the ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model, after training on each training set, a first initial model is obtained, and the first initial model is used to test on the corresponding test set, and the classification evaluation criterion precision rate of the first initial model is calculated and saved;

[0018] The neural network model corresponding to the first initial model with the highest classification evaluation criterion precision rate is selected as the optimal classification neural network model.

[0019] Preferably, the steps of obtaining at least 2 second test ovarian tumor images and forming a second mixed image after lesion mixing include:

[0020] Obtain at least 2 second test ovarian tumor images, and respectively extract the lesion features in the second test ovarian tumor images;

[0021] Extract the regional image with lesion features from one of the second test ovarian tumor images, and superimpose the regional image on the corresponding region of the other second test ovarian tumor image to form a second mixed image.

[0022] Preferably, the steps of inputting the second mixed image into four different segmentation neural network models respectively for five-fold cross-validation and determining the optimal segmentation neural network model according to the segmentation evaluation criterion precision rate of each segmentation neural network model include:

[0023] Input the second mixed image into the U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively;

[0024] The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively randomly divide the second mixed image into 5 parts;

[0025] One copy is selected from each of the U-Net-VGG segmentation neural network model, U-Net-MobileNetv3 segmentation neural network model, U-Net segmentation neural network model, FCN segmentation neural network model, and Deeplabv3+ segmentation neural network model as the test set, and the remaining 4 copies are used as the training set for model training. Then, the selection process is repeated 4 times, each time selecting one copy from the unselected test sets as the test set and using the remaining 4 copies as the training set for training;

[0026] Within the U-Net-VGG segmentation neural network model, U-Net-MobileNetv3 segmentation neural network model, U-Net segmentation neural network model, FCN segmentation neural network model, and Deeplabv3+ segmentation neural network model, after training on each training set, a second initial model is obtained, and the second initial model is used to test on the corresponding test set, and the segmentation evaluation criterion precision rate of the initial model is calculated and saved;

[0027] The neural network model corresponding to the initial model with the highest segmentation evaluation criterion precision rate is selected as the optimal segmentation neural network model.

[0028] Preferably, the steps of integrating the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model include:

[0029] Determine the final loss function based on the cross-entropy and Dice loss functions;

[0030] Incorporate the final loss function into the final segmentation neural network model;

[0031] And the steps of performing lesion segmentation on CT-PET ovarian cancer images based on the final segmentation neural network model include:

[0032] Perform lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model to obtain a first output feature;

[0033] Perform lesion segmentation on the CT-PET ovarian cancer image after scaling and restore it to the original size to obtain a second output feature;

[0034] Fuse the first output feature and the second output feature to form a segmentation image.

[0035] The present invention also discloses an ovarian cancer image processing system based on deep learning, including:

[0036] A first mixing unit, which acquires at least 2 first test ovarian tumor images and forms a first mixed image after image mixing;

[0037] The first modeling unit inputs the first mixed image into four different classification neural network models respectively for five-fold cross-validation, and determines the optimal classification neural network model according to the precision of the classification evaluation criteria of each classification neural network model;

[0038] The second mixing unit obtains at least two second test ovarian tumor images, and forms a second mixed image after lesion mixing;

[0039] The second modeling unit inputs the second mixed image into four different segmentation neural network models respectively for five-fold cross-validation, determines the optimal segmentation neural network model according to the precision of the segmentation evaluation criteria of each segmentation neural network model, and integrates the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model;

[0040] The processing unit classifies the CT-PET ovarian cancer image based on the optimal classification neural network model, and performs lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model.

[0041] Preferably, the first modeling unit inputs the first mixed image into the ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model respectively;

[0042] The ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model randomly divide the first mixed image into 5 parts respectively;

[0043] The ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model respectively select 1 part as the test set, and the remaining 4 parts as the training set for model training, and repeat 4 times to select 1 part from the unselected test sets as the test set, and the remaining 4 parts as the training set for training;

[0044] In the ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model, a first initial model is obtained after training on each training set, and the first initial model is used to test on the corresponding test set, and the precision of the classification evaluation criteria of the first initial model is calculated and saved;

[0045] The first modeling unit selects the neural network model corresponding to the first initial model with the highest precision of the classification evaluation criterion as the optimal classification neural network model.

[0046] Preferably, the second mixing unit obtains at least two second test ovarian tumor images, and respectively extracts the lesion features in the second test ovarian tumor images.

[0047] The second mixing unit extracts the regional image with lesion features from one of the second test ovarian tumor images, and superimposes the regional image on the corresponding region of the other second test ovarian tumor image to form a second mixed image.

[0048] Preferably, the second modeling unit inputs the second mixed image into the U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively.

[0049] The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively randomly divide the second mixed image into five parts.

[0050] The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively select one of them as the test set, and the remaining four as the training set for model training, and repeat the selection of one of the unselected test sets as the test set four more times, and the remaining four as the training set for training.

[0051] In the U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model, after training on each training set, a second initial model is obtained, and the second initial model is used to test on the corresponding test set, and the precision of the segmentation evaluation criterion of the initial model is calculated and saved.

[0052] The second modeling unit selects the neural network model corresponding to the initial model with the highest precision of the segmentation evaluation criterion as the optimal segmentation neural network model.

[0053] Preferably, the processing unit determines the final loss function based on the cross-entropy and Dice loss functions, integrates the final loss function into the final segmentation neural network model, and performs lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model to obtain the first output feature;

[0054] The processing unit scales the CT-PET ovarian cancer image and then performs lesion segmentation, and restores it to the original size to obtain the second output feature, and fuses the first output feature and the second output feature to form a segmentation image.

[0055] After adopting the above technical solution, compared with the prior art, it has the following beneficial effects:

[0056] 1. It can simultaneously determine the optimal models for image classification and segmentation;

[0057] 2. After multiple optimizations of the performance of the segmentation network, it has excellent segmentation effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of forming a first mixed image in a preferred embodiment of the present invention;

[0059] Figure 2 It is a schematic diagram of a comparison table of the classification effects of the classification neural network in a preferred embodiment of the present invention;

[0060] Figure 3 It is a schematic diagram of forming a second mixed image in a preferred embodiment of the present invention;

[0061] Figure 4 It is a schematic diagram of a comparison table of the segmentation effects of the segmentation neural network in a preferred embodiment of the present invention;

[0062] Figure 5 It is a schematic flowchart of an ovarian cancer image processing method in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0063] The advantages of the present invention will be further elaborated below in conjunction with the accompanying drawings and specific embodiments.

[0064] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0065] The terms used in this disclosure are for the purpose of describing particular embodiments only and are not intended to limit the disclosure. The singular forms "a", "the", and "said" used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0066] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0067] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0068] In the description of the present invention, unless otherwise specified and defined, it should be noted that the terms "mounted", "connected", and "coupled" should be understood in a broad sense. For example, it may be a mechanical connection or an electrical connection, or it may be the communication inside two elements. It may be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms may be understood according to specific circumstances.

[0069] In the following description, the suffixes such as "module", "component", or "unit" used to represent elements are only for the convenience of explaining the present invention, and they do not have a specific meaning in themselves. Therefore, "module" and "component" may be used interchangeably.

[0070] Referring to Figure 5 , there is shown an image processing method for ovarian cancer based on deep learning in a preferred embodiment of the present invention, including the following steps:

[0071] S100: Obtain at least two first test ovarian tumor images, and form a first mixed image after image mixing;

[0072] Referring to Figure 1, the first test ovarian tumor images at different levels (or cross-sections) are obtained, and after image mixing (Mixup), a first mixed image is formed, and the region of the ovarian tumor is two-dimensionalized.

[0073] S200: Input the first mixed image into four different classification neural network models respectively for five-fold cross-validation, and determine the optimal classification neural network model according to the precision of the classification evaluation criteria of each classification neural network type;

[0074] To determine the best classification neural network model for classifying the benign and malignant of ovarian tumors in ovarian tumor images, the first mixed image is input into four different classification neural network models respectively for five-fold cross-validation. Cross-validation is also called cyclic estimation, which is a statistical method of cutting data samples into smaller sets. The application scenario of cross-validation is mainly in model training. In a given sample space, most samples are taken as the training set, and a small part of the samples are used to test the just-established model, and the prediction error or prediction accuracy of this small part of the samples is calculated, and the sum of their average values is recorded. This process is iterated K times, that is, K-fold cross-validation (K = 5 in the present invention). The sum of the squared prediction errors of each sample is called the prediction error. The purpose of cross-validation is to obtain as much effective information as possible from limited learning data; cross-validation can start learning samples from multiple directions, which can effectively avoid falling into local minima; it can avoid overfitting problems to a certain extent. Finally, the optimal classification neural network model is determined according to the precision (Accuracy) of the classification evaluation criteria of each classification neural network type

[0075] S300: Obtain at least 2 second test ovarian tumor images, and after mixing the lesions, form a second mixed image;

[0076] Refer to Figure 3 , further, obtain at least 2 second test ovarian tumor images (which can be the same as or different from the first test ovarian tumor images), target the lesion areas in the second test ovarian tumor images, and image mixing (CarveMix) is the second mixed image.

[0077] S400: Input the second mixed image into four different segmentation neural network models respectively for five-fold cross-validation, and determine the optimal segmentation neural network model according to the precision of the segmentation evaluation criteria of each segmentation neural network type;

[0078] To determine the best segmentation neural network model for extracting the lesion features in ovarian tumor images, the second mixed image is input into four different classification neural network models respectively for five-fold cross-validation.

[0079] S500: Integrate the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model;

[0080] Aiming at the problems of general CT-PET ovarian cancer image quality, blurred lesion features, and small lesion areas with unclear boundaries, a segmentation model based on the lesion edge, the KiteNet segmentation network, is introduced to fully learn the lesion edge features in the segmentation task. After the feature fusion and model integration of the optimal segmentation neural network model and the KiteNet segmentation network based on the lesion edge, the effects are improved compared with the commonly used segmentation networks.

[0081] S600: Classify the CT-PET ovarian cancer images based on the optimal classification neural network model, and segment the lesions of the CT-PET ovarian cancer images based on the final segmentation neural network model.

[0082] In a preferred embodiment, step S200 includes:

[0083] S210: Input the first mixed image into the ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model respectively;

[0084] From the Swin-T transformation of ResNet50, the ConvNeXt classification neural network model was finally obtained. Transformer is widely used in the NLP field and can establish dependencies between large ranges of data in the form of attention. Language has a good foundation with individual words as elements, but the basic elements of images vary greatly in scale. The token scales of existing vision models based on transformers are fixed and not suitable for vision tasks. On the other hand, vision tasks have higher resolution requirements, such as semantic segmentation, which is accurate to the pixel level and not suitable for directly using the self-attention mechanism of transformers (which would be on the order of the square of the image size). The Swin Transformer constructs hierarchical feature maps, and the computational complexity is a linear function of the image size. Overall, the Swin Transformer starts with small patches and then fuses neighbor patch information in the deeper layers. And self-attention is only used within non-adjacent large windows (that is, the shifted window, the full name of Swin), and there is no overlap between the windows. A core of DenseNet is the use of a large number of Dense Blocks in the network. It is a convolutional neural network with a tightly connected property. Any two layers in this neural network have a direct connection, that is, the input of each layer in the network is the union of the outputs of all previous layers, and the features learned by this layer will also be directly passed to all subsequent layers as input. This tight connection only exists within the same Dense Block, and there is no such tight connection between different Dense Blocks. EfficientNet is a fast and high-precision model that uses a co-regulation technique for depth, width, and input image resolution.

[0085] S220: The ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model respectively randomly divide the first mixed image into 5 parts;

[0086] S230: Select one of the ConvNeXt classification neural network model, Swin-Transformer classification neural network model, DenseNet classification neural network model, and EfficientNet classification neural network model as the test set, and use the remaining four as the training set for model training. Then repeat the selection process four more times, each time choosing one from the unselected test sets as the test set and using the remaining four as the training set for training;

[0087] S240: In the ConvNeXt classification neural network model, Swin-Transformer classification neural network model, DenseNet classification neural network model, and EfficientNet classification neural network model, after training on each training set, obtain a first initial model, and use the first initial model to test on the corresponding test set, calculate and save the precision rate of the classification evaluation criterion of the first initial model;

[0088] S250: Select the neural network model corresponding to the first initial model with the highest classification evaluation criterion precision rate as the optimal classification neural network model.

[0089] Refer to Figure 2 , after continuous testing, it is found that the ConvNeXt classification neural network model has the best accuracy rate and average accuracy rate, and it is used as the optimal classification neural network model.

[0090] Furthermore, step S300 includes:

[0091] S310: Obtain at least two second test ovarian tumor images, and respectively extract the lesion features in the second test ovarian tumor images;

[0092] S320: Extract the regional image with lesion features from one of the second test ovarian tumor images, and superimpose the regional image with the corresponding region of the other second test ovarian tumor image to form the second mixed image.

[0093] And step S400 includes:

[0094] S410: Input the second mixed image into the U-Net-VGG segmentation neural network model, U-Net-MobileNetv3 segmentation neural network model, U-Net segmentation neural network model, FCN segmentation neural network model, and Deeplabv3+ segmentation neural network model respectively;

[0095] U-Net is an encoder-decoder structure extended and modified based on a fully convolutional network, used to obtain the edges of an image. The encoder gradually reduces the spatial dimension of the pooling layer, and the decoder gradually restores the details and spatial dimension of the object. There are usually skip connections between the encoder and the decoder, which can help the decoder better restore the details of the target. VGGNet contains many levels of networks, with depths ranging from 11 layers to 19 layers. The commonly used ones are VGGNet-16 and VGGNet-19. VGGNet divides the network into 5 segments, each of which concatenates multiple 3*3 convolutional networks in series. After each segment of convolution, a max pooling layer is connected, and finally there are 3 fully connected layers and a softmax layer. MobileNetV3 utilizes two AutoML techniques, namely: MnasNet (an automatic mobile neural architecture search (MNAS) method), NetAdapt (a platform-aware algorithm for mobile applications). MobileNetV3 first uses MnasNet to search for a rough structure, and then uses reinforcement learning to select the optimal configuration from a set of discrete choices. After that, MobileNetV3 uses NetAdapt to fine-tune the architecture, which reflects the supplementary function of NetAdapt, and it can adjust the under-utilized activation channels with a small reduction. FCN is a representative work of deep learning applied to image segmentation, and it is an end-to-end image segmentation method that allows the network to directly obtain the label map by making pixel-level predictions. DeepLabv3+ is a very advanced deep learning-based image semantic segmentation method that can perform pixel-level segmentation of objects.

[0096] S420: The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively randomly divide the second mixed image into 5 parts;

[0097] S430: The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively select 1 part as the test set, and the remaining 4 parts as the training set for model training, and repeat 4 times to select 1 part from the unselected test sets as the test set, and the remaining 4 parts as the training set for training;

[0098] S440: In the U-Net-VGG segmentation neural network model, U-Net-MobileNetv3 segmentation neural network model, U-Net segmentation neural network model, FCN segmentation neural network model, and Deeplabv3+ segmentation neural network model, after training on each training set, a second initial model is obtained, and the second initial model is used to test on the corresponding test set, and the segmentation evaluation criterion precision rate of the initial model is calculated and saved.

[0099] S450: Select the neural network model corresponding to the initial model with the highest segmentation evaluation criterion precision rate as the optimal segmentation neural network model.

[0100] Refer to Figure 4 , after continuous testing, it is found that the U-Net-MobileNetv3 segmentation neural network model has the best accuracy and average accuracy, and it is used as the optimal segmentation neural network model.

[0101] Further, step S500 includes:

[0102] S510: Determine the final loss function based on the Cross-Entropy and Dice loss functions.

[0103] In information theory, the cross-entropy of two probability distributions and based on the same event measure refers to the average number of bits (bit) required to uniquely identify an event in the event set when encoding based on a "non-natural" (relative to the "true" distribution) probability distribution. Dice loss and Dice coefficient are the same thing, and their relationship is: DiceLoss = 1 - DiceCoefficient, DiceLoss = 1 - DiceCoefficient. The Dice coefficient is a set similarity metric function, usually used to calculate the similarity between two samples (value range is [0,1]). DiceCoefficient = 2|X∩Y| / |X|+|Y|, where |X|∩|Y| represents the intersection of sets X and Y, and |X| and |Y| represent the number of their elements. For the segmentation task, |X| and |Y| represent the segmentation ground truth and predict_mask. The formula for Dice Loss: DiceLoss = 1 - 2|X∩Y| / |X|+|Y|, and the final loss function is the Dice-CE loss function.

[0104] S520: Incorporate the final loss function into the final segmentation neural network model.

[0105] Furthermore, step S600 includes:

[0106] S610: Performing lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model to obtain a first output feature;

[0107] S620: Performing lesion segmentation on the scaled CT-PET ovarian cancer image and then restoring it to the original size to obtain a second output feature;

[0108] S630: Fusing the first output feature and the second output feature and restoring the size to form a segmentation image.

[0109] Fusing the first output feature and the second output feature to form a segmentation image.

[0110] The present invention also discloses an image processing system for ovarian cancer based on deep learning, including:

[0111] A first mixing unit, which acquires at least two first test ovarian tumor images and forms a first mixed image after image mixing;

[0112] A first modeling unit, which inputs the first mixed image into four different classification neural network models respectively for five-fold cross-validation, and determines the optimal classification neural network model according to the precision of the classification evaluation criteria of each classification neural network model;

[0113] A second mixing unit, which acquires at least two second test ovarian tumor images and forms a second mixed image after lesion mixing;

[0114] A second modeling unit, which inputs the second mixed image into four different segmentation neural network models respectively for five-fold cross-validation, and determines the optimal segmentation neural network model according to the precision of the segmentation evaluation criteria of each segmentation neural network model, and integrates the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model;

[0115] A processing unit, which classifies the CT-PET ovarian cancer image based on the optimal classification neural network model and performs lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model.

[0116] Preferably, the first modeling unit inputs the first mixed image into a ConvNeXt classification neural network model, a Swin-Transformer classification neural network model, a DenseNet classification neural network model, and an EfficientNet classification neural network model respectively;

[0117] The ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model respectively randomly divide the first mixed image into 5 parts;

[0118] The ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model respectively select 1 part of them as the test set, and the remaining 4 parts as the training set for model training, and repeat 4 times to select 1 part from the unselected test sets as the test set, and the remaining 4 parts as the training set for training;

[0119] In the ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model, after training on each training set, a first initial model is obtained, and the first initial model is used to test on the corresponding test set, and the classification evaluation criterion precision rate of the first initial model is calculated and saved;

[0120] The first modeling unit selects the neural network model corresponding to the first initial model with the highest classification evaluation criterion precision rate as the optimal classification neural network model.

[0121] Preferably, the second mixing unit obtains at least 2 second test ovarian tumor images, and respectively extracts the lesion features in the second test ovarian tumor images;

[0122] The second mixing unit extracts the regional image with lesion features of one of the second test ovarian tumor images, and superimposes the regional image with the corresponding region of the other second test ovarian tumor image to form a second mixed image.

[0123] Preferably, the second modeling unit inputs the second mixed image into the U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively;

[0124] The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively randomly divide the second mixed image into 5 parts;

[0125] For the U-Net-VGG segmentation neural network model, U-Net-MobileNetv3 segmentation neural network model, U-Net segmentation neural network model, FCN segmentation neural network model, and Deeplabv3+ segmentation neural network model, select 1 of them as the test set, and use the remaining 4 as the training set for model training. Then repeat the process 4 times, each time selecting 1 from the unselected test sets as the test set and using the remaining 4 as the training set for training;

[0126] In the U-Net-VGG segmentation neural network model, U-Net-MobileNetv3 segmentation neural network model, U-Net segmentation neural network model, FCN segmentation neural network model, and Deeplabv3+ segmentation neural network model, after training on each training set, obtain a second initial model, and use the second initial model to test on the corresponding test set, calculate and save the precision rate of the segmentation evaluation criterion of the initial model;

[0127] The second modeling unit selects the neural network model corresponding to the initial model with the highest segmentation evaluation criterion precision rate as the optimal segmentation neural network model.

[0128] Preferably, the processing unit determines the final loss function based on cross-entropy and Dice loss function, incorporates the final loss function into the final segmentation neural network model, and performs lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model to obtain the first output feature;

[0129] The processing unit performs lesion segmentation on the scaled CT-PET ovarian cancer image and then restores it to the original size to obtain the second output feature, and fuses the first output feature and the second output feature to form a segmentation image.

[0130] It should be noted that the embodiments of the present invention have good implementability and are not in any form a limitation to the present invention. Any person skilled in the art may use the disclosed technical content to modify or transform it into an equivalent effective embodiment. However, as long as it does not depart from the technical solution of the present invention, any modification, equivalent change, or modification made to the above embodiments based on the technical essence of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. An image processing method for ovarian cancer based on deep learning, characterized in that, It includes the following steps: Obtain at least two first test ovarian tumor images, and form a first mixed image after image mixing; Input the first mixed image into four different classification neural network models respectively for five-fold cross-validation, and determine the optimal classification neural network model according to the precision of the classification evaluation criteria of each classification neural network model; Obtain at least two second test ovarian tumor images, and form a second mixed image after lesion mixing; Input the second mixed image into four different segmentation neural network models respectively for five-fold cross-validation, and determine the optimal segmentation neural network model according to the precision of the segmentation evaluation criteria of each segmentation neural network model; Integrate the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model; Classify CT-PET ovarian cancer images based on the optimal classification neural network model, and segment the lesions of CT-PET ovarian cancer images based on the final segmentation neural network model.

2. The image processing method for ovarian cancer according to claim 1, characterized in that, The steps of inputting the first mixed image into four different classification neural network models respectively for five-fold cross-validation, and determining the optimal classification neural network model according to the precision of the classification evaluation criteria of each classification neural network model include: Input the first mixed image into the ConvNeXt classification neural network model, Swin-Transformer classification neural network model, DenseNet classification neural network model, and EfficientNet classification neural network model respectively; The ConvNeXt classification neural network model, Swin-Transformer classification neural network model, DenseNet classification neural network model, and EfficientNet classification neural network model respectively randomly divide the first mixed image into 5 parts; The ConvNeXt classification neural network model, Swin-Transformer classification neural network model, DenseNet classification neural network model, and EfficientNet classification neural network model respectively select 1 part as the test set, and the remaining 4 parts as the training set for model training, and repeat 4 times to select 1 part from the unselected test sets as the test set, and the remaining 4 parts as the training set for training; In the ConvNeXt classification neural network model, Swin-Transformer classification neural network model, DenseNet classification neural network model, and EfficientNet classification neural network model, after training on each training set, obtain a first initial model, and use the first initial model to test on the corresponding test set, calculate and save the precision of the classification evaluation criteria of the first initial model; Select the neural network model corresponding to the first initial model with the highest precision of the classification evaluation criteria as the optimal classification neural network model.

3. The image processing method for ovarian cancer according to claim 1, characterized in that, The steps of obtaining at least two second test ovarian tumor images, and forming a second mixed image after lesion mixing include: Obtain at least two second test ovarian tumor images, and respectively extract the lesion features in the second test ovarian tumor images; extract the regional image with lesion features in one of the second test ovarian tumor images, and superimpose the regional image with the corresponding region of the other second test ovarian tumor image to form the second mixed image.

4. The image processing method for ovarian cancer according to claim 3, characterized in that, The steps of inputting the second mixed image into four different segmentation neural network models respectively for five-fold cross-validation and determining the optimal segmentation neural network model according to the precision of the segmentation evaluation criterion of each segmentation neural network model include: Input the second mixed image into the U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively; The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively randomly divide the second mixed image into 5 parts; The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively select 1 part as the test set, and the remaining 4 parts as the training set for model training, and repeat 4 times to select 1 part from the unselected test sets as the test set, and the remaining 4 parts as the training set for training; In the U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model, after training on each training set, obtain a second initial model, and use the second initial model to test on the corresponding test set, calculate and save the precision of the segmentation evaluation criterion of the initial model; Select the neural network model corresponding to the initial model with the highest segmentation evaluation criterion precision as the optimal segmentation neural network model.

5. The image processing method for ovarian cancer according to claim 1, characterized in that, The steps of integrating the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model include: Determine the final loss function based on the cross-entropy and Dice loss functions; Integrate the final loss function into the final segmentation neural network model; And the steps of performing lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model include: Perform lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model to obtain the first output feature; Perform lesion segmentation on the CT-PET ovarian cancer image after scaling and restore it to the original size to obtain the second output feature; Fuse the first output feature and the second output feature to form a segmentation image.

6. An image processing system for ovarian cancer based on deep learning, characterized in that, Include: The first mixing unit obtains at least two first test ovarian tumor images, and forms a first mixed image after image mixing; The first modeling unit inputs the first mixed image into four different classification neural network models respectively for five-fold cross-validation, and determines the optimal classification neural network model according to the precision of the classification evaluation criteria of each classification neural network model; The second mixing unit obtains at least two second test ovarian tumor images, and forms a second mixed image after lesion mixing; The second modeling unit inputs the second mixed image into four different segmentation neural network models respectively for five-fold cross-validation, and determines the optimal segmentation neural network model according to the precision of the segmentation evaluation criteria of each segmentation neural network model, and integrates the optimal segmentation neural network model with the KiteNet segmentation network based on the lesion edge to determine the final segmentation neural network model; The processing unit classifies the CT-PET ovarian cancer image based on the optimal classification neural network model, and performs lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model.

7. The ovarian cancer image processing system according to claim 6, wherein, The first modeling unit inputs the first mixed image into the ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model respectively; The ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model randomly divide the first mixed image into 5 parts respectively; The ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model respectively select 1 part as the test set, and the remaining 4 parts as the training set for model training, and repeat 4 times to select 1 part from the unselected test sets as the test set, and the remaining 4 parts as the training set for training; In the ConvNeXt classification neural network model, the Swin-Transformer classification neural network model, the DenseNet classification neural network model, and the EfficientNet classification neural network model, a first initial model is obtained after training on each training set, and the first initial model is used to test on the corresponding test set, and the precision of the classification evaluation criteria of the first initial model is calculated and saved; The first modeling unit selects the neural network model corresponding to the first initial model with the highest classification evaluation criteria precision as the optimal classification neural network model.

8. The ovarian cancer image processing system according to claim 6, wherein, The second mixing unit obtains at least two second test ovarian tumor images, and extracts the lesion features in the second test ovarian tumor images respectively; The second mixing unit extracts the regional image with lesion features from one of the second test ovarian tumor images, and superimposes the regional image on the corresponding region of the other second test ovarian tumor image to form the second mixed image.

9. The ovarian cancer image processing system according to claim 8, wherein, The second modeling unit inputs the second mixed image into the U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively; The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model randomly divide the second mixed image into 5 parts respectively; The U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model respectively select 1 part as the test set, and the remaining 4 parts as the training set for model training, and repeat 4 times to select 1 part from the unselected test sets as the test set, and the remaining 4 parts as the training set for training; In the U-Net-VGG segmentation neural network model, the U-Net-MobileNetv3 segmentation neural network model, the U-Net segmentation neural network model, the FCN segmentation neural network model, and the Deeplabv3+ segmentation neural network model, after training on each training set, a second initial model is obtained, and the second initial model is used to test on the corresponding test set, and the segmentation evaluation criterion precision rate of the initial model is calculated and saved; The second modeling unit selects the neural network model corresponding to the initial model with the highest segmentation evaluation criterion precision rate as the optimal segmentation neural network model.

10. The ovarian cancer image processing system according to claim 6, wherein, The processing unit determines the final loss function based on the cross-entropy and Dice loss functions, integrates the final loss function into the final segmentation neural network model, and performs lesion segmentation on the CT-PET ovarian cancer image based on the final segmentation neural network model to obtain the first output feature; The processing unit performs lesion segmentation on the CT-PET ovarian cancer image after scaling and restores it to the original size to obtain the second output feature, and fuses the first output feature and the second output feature to form a segmentation image.

Images