Ovarian cancer intelligent diagnosis system based on deep learning

Abstract

The invention relates to an intelligent ovarian cancer diagnosis system based on deep learning. The intelligent ovarian cancer diagnosis system comprises a report acquisition module and an image processing module, the feature extraction module comprises parallel feature extraction networks which are respectively used for performing feature extraction on the ovarian ultrasound image before preprocessing to obtain a first feature and performing feature extraction on the ovarian ultrasound image after preprocessing to obtain a second feature; wherein the feature extraction network sequentially comprises a feature scale attention network, a space structure attention network and a channel weighting network. According to the method, the benign and malignant tumors in the image can be directly pre-classified through the paper ultrasonic report shot by the patient, so that the time from detection to diagnosis can be greatly shortened, the waiting anxiety of the patient is reduced, and meanwhile, the accuracy and efficiency of diagnosis can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image analysis, and more specifically to an intelligent diagnosis system for ovarian cancer based on deep learning. Background Art

[0002] As a non-invasive and efficient medical imaging technology, ultrasound examination plays an important role in early detection of tumors. Especially in the screening and diagnosis of ovarian cancer, ultrasound examination provides doctors with important information about ovarian tissue. However, the current ultrasound examination process has some limitations and shortcomings.

[0003] First, after completing the ultrasound examination and obtaining the paper report, patients usually need to wait for several days to get the diagnosis results from professional physicians. This waiting period is full of anxiety and uncertainty for patients, especially when they are worried about their health. In addition, too long a waiting time may lead to a delay in treatment. For patients with malignant tumors, this delay may affect the treatment effect and prognosis.

[0004] Second, because ultrasound images need to be interpreted by experienced radiologists or specialists, the number of professionals is limited, which limits the efficiency of the diagnostic process. This inefficiency is particularly evident in resource-constrained healthcare systems.

[0005] Furthermore, the diagnosis of ovarian cancer is somewhat complicated because benign and malignant ovarian tissues may show similar characteristics on ultrasound images. This requires physicians to have a high level of professional knowledge and rich clinical experience to make accurate judgments. For inexperienced doctors, this undoubtedly increases the risk of misdiagnosis and missed diagnosis.

[0006] Therefore, it is particularly important to develop an intelligent diagnostic system that can automatically, efficiently and accurately analyze ultrasound images. Summary of the invention

[0007] In order to overcome the above technical problems, the present invention provides an intelligent diagnosis system for ovarian cancer based on deep learning, so that patients can directly obtain tumor pre-classification based on paper ultrasound reports.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] An intelligent diagnosis system for ovarian cancer based on deep learning, comprising:

[0010] A report collection module, used to collect ovarian ultrasound images of patients;

[0011] An image processing module, used for preprocessing ovarian ultrasound images;

[0012] A feature extraction module, including parallel feature extraction networks, which are used to extract features from the ovarian ultrasound image before preprocessing to obtain a first feature, and to extract features from the ovarian ultrasound image after preprocessing to obtain a second feature; wherein the feature extraction network includes a feature scale attention network, a spatial structure attention network, and a channel weighted network in sequence;

[0013] The ovarian cancer prediction module is used to splice and fuse the first feature and the second feature, and classify and predict the nature of ovarian cancer based on the fused feature.

[0014] Furthermore, the preprocessing step includes:

[0015] Use the YOLOv5s model to segment the ovarian image based on the ultrasound report and resize the image;

[0016] Based on the detection and fitting of the adjusted image contour, the image is oriented and corrected;

[0017] Map the color of the rectified image to BGR and normalize it.

[0018] Furthermore, based on the detection and fitting of the image contour, the image is oriented and corrected, the steps include:

[0019] Convert the image into a grayscale image, and convert the grayscale image into a binary image based on a set threshold;

[0020] Use the Sobel operator to detect the outer edge of the binary image and obtain the outer contour of the image;

[0021] Obtaining a minimum bounding rectangle based on the image periphery contour fitting, and determining the rotation angle according to the minimum bounding rectangle, and

[0022] The image is oriented according to the rotation angle.

[0023] Furthermore, the feature scale attention network includes three parallel convolution branches, the output results of each convolution branch are concatenated based on the channel dimension and enter the adaptive global pooling layer, and then the weighted vectors at different scales are obtained through the first 1×1 convolution;

[0024] The weighted vector is dot-multiplied with the output result of the corresponding convolution branch and then merged to obtain a feature scale attention map.

[0025] Further, each convolution branch includes a second 1×1 convolution and a dilated convolution in series;

[0026] Among them, the size of the dilated convolution in the first branch is 2×2; the size of the dilated convolution in the second branch is 3×3; the size of the dilated convolution in the third branch is 4×4.

[0027] Furthermore, the spatial structure attention network includes a third 1×1 convolutional layer, and a horizontal pooling layer and a vertical pooling layer simultaneously connected to the third 1×1 convolutional layer;

[0028] The output results of the horizontal pooling layer and the vertical pooling layer are merged and concatenated, and then multiplied with the feature map output by the third 1×1 convolutional layer to obtain the spatial structure attention map.

[0029] Further, the channel weighted network includes a fourth 1×1 convolutional layer, and an adaptive average pooling layer, a first fully connected layer, and a second fully connected layer sequentially connected to the fourth 1×1 convolutional layer;

[0030] Among them, the result of multiplying the output result of the second fully connected layer and the feature map output by the fourth 1×1 convolutional layer is used as the channel weighted feature.

[0031] Furthermore, a convolution block is provided at the input end of the feature scale attention network and the channel weighted network, and the convolution block includes a stacked fifth 1×1 convolution and a 3×3 convolution.

[0032] Furthermore, the ovarian cancer prediction module sequentially includes a fully connected layer with 512 neurons and a Relu activation function, a batch normalization layer, a fully connected layer with 64 neurons, a Relu activation function and an L2 regularization function, a Dropout layer, a batch normalization layer, and one neuron and a sigmoid activation function.

[0033] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses an intelligent diagnosis system for ovarian cancer based on deep learning, which can directly pre-classify benign and malignant tumors in the image through the paper ultrasound report taken by the patient. It can not only greatly shorten the time from detection to diagnosis and reduce the anxiety of patients waiting, but also improve the accuracy and efficiency of diagnosis, assist doctors to quickly make more accurate diagnoses, thereby reducing the workload of doctors and optimizing the allocation of medical resources.

[0034] Specifically, in pre-classification, the feature scale attention network can improve the model's sensitivity to information of different scales and classification performance through effective fusion, adaptive processing, dynamic weighting and enhancement, and final merging of multi-scale features. This design idea helps solve the problem of large scale changes and high classification difficulty of ovarian cancer tissue pathology images;

[0035] Furthermore, the spatial structure attention network can finely process and integrate spatial information in different directions, and dynamically adjust the importance of features, thereby significantly improving the model's ability to understand and utilize the spatial structure of images;

[0036] Furthermore, the channel weighted network can achieve effective compression of the input feature map, dynamic weighting of channel information and deep fusion of global features, enabling the network to improve the model's understanding of image features and classification accuracy while maintaining a low computational cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 This is a structural block diagram of the intelligent diagnosis system for ovarian cancer provided by the present invention;

[0039] Figure 2 The schematic diagram of the intelligent diagnosis system for ovarian cancer provided by the present invention;

[0040] Figure 3 A schematic diagram of the feature extraction network structure provided by the present invention;

[0041] Figure 4 A schematic diagram of a characteristic scale attention network structure provided by the present invention;

[0042] Figure 5 A schematic diagram of the spatial structure attention network structure provided by the present invention;

[0043] Figure 6 A schematic diagram of the channel weighted network structure provided by the present invention;

[0044] Figure 7 This is a schematic diagram of the internal structure of the ovarian cancer prediction module provided by the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] In order to match the actual application scenario where patients can only obtain paper ultrasound reports, an embodiment of the present invention discloses an intelligent diagnosis system for ovarian cancer based on deep learning. The system can be implanted in the current medical self-service film dispensing machine or ultrasound self-service film dispensing machine; it can also be implanted in a mobile terminal, such as a mobile phone, iPad, etc.

[0047] The intelligent ovarian cancer diagnosis system disclosed in the present invention automatically identifies and extracts key images from paper ultrasound reports, and uses a deep learning model to analyze the extracted ultrasound images, thereby automatically distinguishing between benign and malignant tumors and providing preliminary diagnostic recommendations for patients.

[0048] Specifically, in this embodiment, the ovarian cancer intelligent diagnosis system based on deep learning is as follows: Figure 1 As shown, including:

[0049] 1. Report collection module, used to collect ovarian ultrasound images of patients;

[0050] In one embodiment, the ovarian ultrasound image can be obtained by taking a picture with a built-in camera, or by scanning with a system built-in scanner;

[0051] 2. An image processing module, used for preprocessing ovarian ultrasound images;

[0052] Since the ultrasound report contains multiple ovarian cancer ultrasound images, it is difficult for conventional image classification methods to classify multiple ovarian cancer ultrasound images. Secondly, the paper ovarian cancer ultrasound report taken by the camera often carries a lot of environmental noise, such as glare, rotation, deformation and other problems. The existence of glare can be attributed to the excessive light in the environment when the patient is photographed; rotation is the rotation noise generated when the patient uses the report acquisition module to collect ultrasound images, and the report is not in a vertical or horizontal direction; and deformation noise is often caused by the patient folding or improperly handling the ultrasound report, resulting in distortion of the ultrasound image.

[0053] The above factors will affect the accuracy of classification. Therefore, the present application first preprocesses the ovarian ultrasound image, including segmentation and denoising.

[0054] In one embodiment, the preprocessing step includes:

[0055] S1. Use the YOLOv5s model to segment the ovarian image based on the ultrasound report and adjust the image size;

[0056] In this application, the YOLOv5s model has been pre-trained to accurately identify and segment the ovarian image in the ultrasound report; at the same time, as a preferred method, the image is adjusted to 200×200 pixels;

[0057] S2. Based on the detection and fitting of the adjusted image contour, the image is oriented. The correction steps include:

[0058] S21, converting the color image into a grayscale image for subsequent processing; further converting the grayscale image into a binary image based on a set threshold to highlight the main features; in one embodiment, a morphological operation (such as an opening operation or a closing operation) is also used to remove small noise points;

[0059] S22, using the Sobel operator to detect the outer edge of the binary image to obtain the outer contour of the image;

[0060] S23. Obtain a minimum circumscribed rectangle based on the image peripheral contour fitting. The border of the rectangle is the smallest, completely surrounds the contour, and can have any angle. It should be noted that if there are multiple recognized peripheral contours, it is necessary to determine which contour or which contour angles best represent the tilt of the entire image. At this time, it can be determined by considering the size, position or other features of the contour; or take the average of all contour angles to reduce the influence of individual contours.

[0061] S24, determining a rotation angle according to the minimum circumscribed rectangle, and obtaining the rotation angle from the fitted rectangle, where the angle represents the rotation angle of the rectangle relative to the horizontal line of the image;

[0062] S25. Finally, the image is oriented according to the rotation angle, that is, a proper rotation matrix is ​​applied to the original image to correct the tilt.

[0063] S3, map the color of the rectified image to BGR and perform normalization.

[0064] After the image is rectified, its color is mapped to BGR to avoid color distortion or incorrect display of the image; it is then normalized to ensure that the learning process can be performed normally even when the input data value range is very wide.

[0065] In one embodiment, the method for normalizing the data is:

[0066] Subtract the mean from each data item and divide by the standard deviation. The mean and standard deviation can be calculated based on the input image or the dataset. In this application, the mean and standard deviation are calculated only from the training dataset, and then each input image x is normalized using the following formula.

[0067]

[0068] Where y is the output image, μ is the mean of the training dataset, and σ is the standard deviation.

[0069] 3. Feature extraction module. In the present invention, the feature extraction module includes a parallel feature extraction network, which is used to extract features from the ovarian ultrasound image before preprocessing to obtain a first feature, and to extract features from the ovarian ultrasound image after preprocessing to obtain a second feature;

[0070] Since the image is preprocessed to eliminate glare, deformation and rotation problems in the image, some original features may be lost; therefore, in order to retain the features of low-quality images, this application constructs a dual-input network model, such as Figure 2 As shown, the original low-quality image is taken as one of the inputs.

[0071] Furthermore, in this embodiment, the feature extraction network includes a feature scale attention network, a spatial structure attention network, and a channel weighted network in sequence; wherein, Figure 3 As shown, the input end of the feature scale attention network and the channel weighted network is provided with a convolution block, and the convolution block includes a stacked fifth 1×1 convolution and a 3×3 convolution. The 1×1 convolution kernel can linearly combine different channels of the input data to achieve information fusion and compression between channels. The present invention uses 1×1 convolution to reduce the number of parameters of the convolution kernel, thereby reducing the complexity of the model and reducing the risk of overfitting. The 3×3 convolution kernel expands the receptive field of the network, allowing the network to capture more global information while maintaining sensitivity to local features.

[0072] Setting the 3×3 convolution after the 1×1 convolution can achieve channel recombination while maintaining the feature space structure, thereby extracting more diverse features.

[0073] In one embodiment, since ovarian cancer tissue pathology images have multiple microscopic magnifications and the scale characteristics of cancerous nests in the tissue pathology images are different, it is necessary to optimize the network to pay attention to different scale characteristics. Figure 4 As shown, the feature scale attention network includes three parallel convolution branches. Preferably, each convolution branch includes a second 1×1 convolution and an expansion convolution in series; the second 1×1 convolution is used to compress the number of input feature channels, and the expansion convolution is used to further compress the number of feature map channels to increase the receptive field while reducing the increase in parameters and computing resource usage caused by multi-scale large convolution kernels.

[0074] Among them, the size of the dilated convolution in the first branch is 2×2; the size of the dilated convolution in the second branch is 3×3; the size of the dilated convolution in the third branch is 4×4.

[0075] Furthermore, the output results of each convolution branch are spliced ​​based on the channel dimension and then enter the adaptive global pooling layer. The present invention can achieve feature fusion by adding the feature maps generated by the three branches in the channel dimension, integrating information of different scales together to enhance the network's ability to process multi-scale information and improve classification accuracy. Adaptive pooling can automatically adjust the size of the pooling kernel according to the size of the input feature map, so that no matter what the size of the input feature map is, a fixed-size output can be obtained, thereby helping to maintain the spatial resolution of the feature map while reducing the number of parameters and computational complexity.

[0076] Then, the weighted vectors at different scales are obtained through the first 1×1 convolution to realize dynamic weighting of features of different scales. That is, the importance of features of different scales can be learned according to the specific situation of the input data, and their weights can be adjusted accordingly.

[0077] Finally, the weighted vector is multiplied with the output result of the corresponding convolution branch and then merged to obtain a feature scale attention map, thereby realizing dynamic enhancement of features. This operation allows the network to enhance or suppress certain features in a targeted manner according to the importance of features at different scales, thereby improving classification performance.

[0078] By merging the weighted feature maps of the three branches, the present invention can obtain a feature representation that integrates multi-scale information. This feature representation not only contains rich spatial information, but also incorporates the network's learning results on the importance of features at different scales, making subsequent classification decisions more accurate and reliable.

[0079] In one embodiment, in order to further enhance the network's attention to spatial structure information, the present invention introduces a spatial structure attention network, such as Figure 5 As shown, the spatial structure attention network includes a third 1×1 convolutional layer, and a horizontal pooling layer and a vertical pooling layer connected to the third 1×1 convolutional layer at the same time; wherein the third 1×1 convolutional layer is used to compress the number of feature map channels, to reduce the number of parameters and computational complexity, to accelerate the training and reasoning process, to avoid overfitting, and to maintain important spatial structure information; and the horizontal pooling layer and the vertical pooling layer are used to extract information in the horizontal and vertical directions of the feature map, respectively, to capture structural features in different directions, and to enhance the model's understanding of direction-sensitive image content.

[0080] Then, the output results of the horizontal pooling layer and the vertical pooling layer are merged and spliced ​​to integrate spatial information in different directions, and then multiplied with the feature map output by the third 1×1 convolutional layer to obtain the spatial structure attention map.

[0081] Preferably, the output result of the horizontal pooling layer is preferentially subjected to tensor dimension conversion through the convolution layer 1, so as to be spliced ​​with the output result of the vertical pooling layer; at the same time, the spliced ​​result is compressed through the convolution layer 2, and the compressed vertical pooling features and horizontal pooling features are subjected to tensor dimension conversion by the convolution layer 3 and the convolution layer 4 respectively. In this embodiment, tensor dimension conversion and compression can help adjust the dimension and scale of the features to adapt to subsequent multiplication operations, while reducing the consumption of computing resources;

[0082] In a preferred embodiment, convolutional layer 1, convolutional layer 2, convolutional layer 3 and convolutional layer 4 are all 1×1 convolutional layers.

[0083] Finally, based on the dimension-changed features, the feature map output by the third 1×1 convolutional layer is multiplied to obtain the spatial structure attention map. The spatial structure attention network of the present invention realizes dynamic weighting and enhancement of features, allowing the network to emphasize or suppress specific spatial structure information according to the needs of the current task, thereby improving model performance.

[0084] In one embodiment, if Figure 6 As shown, the channel weighted network includes a fourth 1×1 convolutional layer, and an adaptive average pooling layer, a first fully connected layer, and a second fully connected layer sequentially connected to the fourth 1×1 convolutional layer;

[0085] Among them, the fourth 1×1 convolution layer is used to compress the number of channels of the input feature map to the target number of channels. The fourth 1×1 convolution of the present invention can not only perform linear combination on different channels to achieve information fusion and compression between channels, but also enhance the network's ability to integrate channel information and improve the utilization efficiency and representation ability of features; the adaptive average pooling layer is used to automatically adjust the size of the pooling kernel according to the size of the input feature map to ensure that the output feature map has a fixed size; and the first fully connected layer and the second fully connected layer are used to learn channel weights. The fully connected layer can capture global information and perform complex abstract reasoning, which is crucial for understanding the high-level semantic features of the input image.

[0086] Furthermore, the result of multiplying the output of the second fully connected layer and the feature map output by the fourth 1×1 convolutional layer is used as the channel weighted feature to strengthen or suppress the information of certain channels.

[0087] The channel weighted network of the present invention not only improves the feature extraction and representation capabilities of the model, but also realizes the efficient use and screening of different channel information in the feature map by dynamically adjusting the channel weights. The combined use of adaptive pooling and fully connected layers provides the model with an adaptive feature compression and information integration mechanism, which not only optimizes the use of computing resources, but also improves the model's ability to identify and emphasize important features.

[0088] 4. An ovarian cancer prediction module is used to splice and fuse the first feature and the second feature, and classify and predict the nature of ovarian cancer based on the fused feature.

[0089] For deep learning models, a well-designed fully connected layer is crucial. To address the overfitting problem, this application implements dropout and regularization techniques. For the dropout technique, a dropout probability of 0.5 is used. For the regularization technique, L2 regularization is used, and the L2 regularization factor is 0.001. In addition, many batch normalization layers are applied in the fully connected layer to stabilize and speed up the training process.

[0090] Figure 7 Figure 2 shows the organizational structure of each layer in the ovarian cancer prediction module, which includes a fully connected layer with 512 neurons and a Relu activation function, a batch normalization layer, a fully connected layer with 64 neurons, a Relu activation function and an L2 regularization function, a Dropout layer, a batch normalization layer, and one neuron and a sigmoid activation function.

[0091] In an optional embodiment, the present application uses a binary cross entropy loss function in the case of binary classification.

[0092] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0093] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent diagnosis system for ovarian cancer based on deep learning, characterized in that: include: A report collection module, used to collect ovarian ultrasound images of patients; An image processing module, used for preprocessing ovarian ultrasound images; A feature extraction module, including parallel feature extraction networks, which are used to extract features from the ovarian ultrasound image before preprocessing to obtain a first feature, and to extract features from the ovarian ultrasound image after preprocessing to obtain a second feature; wherein the feature extraction network includes a feature scale attention network, a spatial structure attention network, and a channel weighted network in sequence; The ovarian cancer prediction module is used to splice and fuse the first feature and the second feature, and classify and predict the nature of ovarian cancer based on the fused feature.

2. The deep learning-based intelligent diagnosis system for ovarian cancer according to claim 1, characterized in that: The preprocessing steps include: Use the YOLOv5s model to segment the ovarian image based on the ultrasound report and resize the image; Based on the detection and fitting of the adjusted image contour, the image is oriented and corrected; Map the color of the rectified image to BGR and normalize it.

3. The deep learning-based intelligent diagnosis system for ovarian cancer according to claim 2, characterized in that: Based on the detection and fitting of the image contour, the image is oriented and the steps include: Convert the image into a grayscale image, and convert the grayscale image into a binary image based on a set threshold; Use the Sobel operator to detect the outer edge of the binary image and obtain the outer contour of the image; Obtaining a minimum bounding rectangle based on the image periphery contour fitting, and determining the rotation angle according to the minimum bounding rectangle, and The image is oriented according to the rotation angle.

4. The deep learning-based intelligent diagnosis system for ovarian cancer according to claim 1, characterized in that: The feature scale attention network consists of three parallel convolution branches. The output results of each convolution branch are concatenated based on the channel dimension and enter the adaptive global pooling layer. Then, the weighted vectors at different scales are obtained through the first 1×1 convolution. The weighted vector is dot-multiplied with the output result of the corresponding convolution branch and then merged to obtain a feature scale attention map.

5. The deep learning-based intelligent diagnosis system for ovarian cancer according to claim 4, characterized in that: Each convolution branch includes a second 1×1 convolution and a dilated convolution in series; Among them, the size of the dilated convolution in the first branch is 2×2; the size of the dilated convolution in the second branch is 3×3; the size of the dilated convolution in the third branch is 4×4.

6. The deep learning-based intelligent diagnosis system for ovarian cancer according to claim 1, characterized in that: The spatial structure attention network includes a third 1×1 convolutional layer, and a horizontal pooling layer and a vertical pooling layer simultaneously connected to the third 1×1 convolutional layer; The output results of the horizontal pooling layer and the vertical pooling layer are merged and concatenated, and then multiplied with the feature map output by the third 1×1 convolutional layer to obtain the spatial structure attention map.

7. The deep learning-based intelligent diagnosis system for ovarian cancer according to claim 1, characterized in that: The channel weighted network includes a fourth 1×1 convolutional layer, an adaptive average pooling layer, a first fully connected layer, and a second fully connected layer sequentially connected to the fourth 1×1 convolutional layer; Among them, the result of multiplying the output result of the second fully connected layer and the feature map output by the fourth 1×1 convolutional layer is used as the channel weighted feature.

8. The deep learning-based intelligent diagnosis system for ovarian cancer according to claim 1, characterized in that: The input of the feature scale attention network and the channel weighted network is provided with a convolution block including a stacked fifth 1×1 convolution and a 3×3 convolution.

9. The deep learning-based intelligent diagnosis system for ovarian cancer according to claim 1, characterized in that: The ovarian cancer prediction module includes a fully connected layer with 512 neurons and a Relu activation function, a batch normalization layer, a fully connected layer with 64 neurons, a Relu activation function and an L2 regularization function, a Dropout layer, a batch normalization layer, and one neuron and a sigmoid activation function.

Images