Automatic medical image classification and anomaly detection system based on deep learning

Abstract

The invention belongs to the technical field of medical image analysis and artificial intelligence, and relates to an automatic medical image classification and anomaly detection system based on deep learning, and the system employs an image preprocessing module to carry out the preprocessing of a collected HSG image; inputting the preprocessed HSG image into an image classification module, and outputting an HSG image with a classification label; inputting the HSG image with the classification label into a condition guidance generation module to generate a new sample image; inputting the preprocessed HSG image and the new sample image into an anomaly detection module for anomaly detection, and visually displaying the distribution condition and severity of an abnormal area through a thermodynamic diagram display module; the invention provides the automatic, efficient and accurate image anomaly detection and positioning method, the false detection and omission ratio can be effectively reduced, the efficiency and accuracy of clinical diagnosis are improved, and a more reliable auxiliary diagnosis tool is provided for doctors.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image analysis and artificial intelligence, and relates to an automated medical image classification and anomaly detection system based on deep learning. Background Art

[0002] The fast-paced life and work pressure in modern society have made female fertility problems receive increasing attention. Among them, tubal factors are one of the important causes of female infertility. Hysterosalpingography (HSG), as the main examination method for evaluating tubal patency and structural abnormalities, is widely used in clinical practice. However, the traditional HSG image analysis mainly relies on doctors' experience and visual judgment, which is not only time-consuming but also easily affected by subjective factors, resulting in poor diagnostic accuracy and consistency. With the increasing demand for fertility and the improvement of the requirements for precision medicine, it is obviously difficult to meet the clinical needs by manual analysis alone, especially when facing complex conditions and large amounts of data. The limitations of traditional analysis methods are gradually emerging.

[0003] At present, some automated methods based on image processing and analysis have been introduced into the medical field. However, in the application of HSG images, the existing technologies still have not achieved ideal results in dealing with the fine structures of the fallopian tubes and identifying abnormalities such as blockage or partial blockage. Most of these methods cannot accurately capture the individual differences and lesion characteristics of the fallopian tubes, resulting in misdiagnosis or missed diagnosis from time to time. In addition, with the aggravation of the aging population in China, the detection demand for tubal abnormalities in elderly women has increased, further highlighting the deficiencies of the existing technologies in efficient and accurate analysis. Therefore, there is an urgent need to develop a more intelligent HSG image analysis technology to improve the diagnosis efficiency and accuracy and provide more precise medical services for patients.

[0004] Automated medical image classification and anomaly detection algorithms based on deep learning, as an emerging image analysis method, show great potential in image recognition and anomaly detection. Through deep learning technology, it is possible to achieve precise analysis of complex structures in HSG images and automatically identify abnormalities such as tubal patency, stenosis, and obstruction. This technology uses deep learning models such as convolutional neural networks to extract features, classify, and detect anomalies in images, which can effectively reduce the deviation of human judgment and achieve automated and efficient processing of HSG images. However, most of the existing algorithms are based on fixed models and parameters and do not fully consider the differences of individual patients, resulting in unsatisfactory detection effects in some complex or special cases.

[0005] To achieve individualized and precise HSG image analysis, the following core issues must be addressed: First, the extraction and classification of individualized features. The algorithm needs to be able to flexibly adjust model parameters to adapt to the specific anatomical structures and lesion characteristics of the fallopian tubes of different patients. Common solutions include using pre-trained deep learning models for feature extraction and optimizing the classification model through transfer learning and fine-tuning methods to enable accurate identification of various situations such as bilateral patency, unilateral patency, bilateral partial obstruction, and bilateral obstruction. Second is the problem of precise localization of abnormal regions. The abnormal features in HSG images are usually very subtle. How to find the abnormal regions during the image analysis process. We propose a brand-new method. By using Gaussian filtering occlusion and a conditional generative adversarial network (CGAN) to generate new samples of relatively healthy image patches, calculate the differences by comparing the original image and the prediction scores after replacing with healthy image patches, and generate a heatmap matrix to mark the abnormal regions. In addition, the generated heatmap needs to have good interpretability, be able to intuitively display the degree of influence and location of the abnormal regions, and help doctors make diagnoses and decisions quickly. Therefore, it is necessary to further optimize the difference analysis algorithm and the heatmap generation method to enhance the accuracy of the results and the interpretability of clinical applications. These improvements will help more accurately locate the abnormal regions in the fallopian tubes and significantly improve the effectiveness and reliability of diagnosis. Summary of the Invention

[0006] In view of the problems that the current HSG image analysis technology relies on manual judgment during the diagnosis process, is easily affected by subjective factors, and is difficult to accurately identify subtle abnormalities, the present invention proposes an automated medical image classification and anomaly detection system and method based on deep learning. The system includes an image processing module, an image classification module, a conditional guidance generation module, an anomaly detection module, and a heatmap display module, forming an overall logic of "acquisition - preprocessing - classification and guidance generation - difference analysis - localization". Through the automated processing and individualized analysis of HSG images, the system can real-time identify and locate the abnormal regions of the fallopian tubes, overcoming the limitations of existing image analysis methods. The present invention not only improves the accuracy and efficiency of diagnosis, but also reduces the dependence on doctors' experience, provides an individualized and automated image analysis solution, and provides strong support for clinical diagnosis and treatment decisions. The core innovation of the present invention lies in using a deep learning model fine-tuned with individualized data to real-time analyze the patency and lesion characteristics of the fallopian tubes and accurately locate the abnormal regions, significantly enhancing the reliability and effectiveness of hysterosalpingography diagnosis.

[0007] To achieve the above object, the present invention provides the following solutions: An automated medical image classification and anomaly detection system based on deep learning, comprising: The image processing module is responsible for preprocessing the collected HSG images; The image classification module includes a first model and a second model, which are used to identify the category of the HSG image and output the HSG image with a classification label; where: The first model is obtained by training a convolutional neural network model with a large medical image dataset; The second model is obtained by training the first model with the preprocessed HSG image to obtain a pre-trained model, and then fine-tuning the pre-trained model; the fine-tuning is achieved by freezing the general feature layer of the pre-trained model and adjusting the classification layer; the classification labels include A, B, C, and D, where: A is bilateral patency, B is unilateral patency, C is bilateral partial patency, and D is bilateral obstruction; The conditional guidance generation module includes an occlusion experiment unit and a healthy image block generation unit, where: The occlusion experiment unit randomly generates Gaussian noise blocks from the HSG images input by the image classification module, and simulates image damage or occlusion by occluding different parts of the image block by block; The healthy image block generation unit generates healthy image blocks similar to the HSG images of class A by using an adversarial network guided by the classification label, replaces the Gaussian noise blocks, and generates new sample images; The anomaly detection module determines the anomaly region by calculating the difference score of each region by subtracting the prediction scores of the original image and the new sample image; The heatmap display module marks the anomaly region according to the difference score, and superimposes the heatmap on the original image to display the distribution and severity of the anomaly region.

[0008] As a preferred solution of the present invention, the preprocessing includes cropping, standardization, denoising, image enhancement, normalization, and image chunking, where: The cropping is performed on the original HSG image to retain the key parts of the fallopian tube and related anatomical structures, reduce background noise, and ensure that the image focuses on the analysis target region; The standardization is performed on the cropped HSG image to enhance the clarity of the fallopian tube region; The denoising is to remove random noise and speckles to enhance the contour and details of the fallopian tube; The image enhancement is to highlight the details of the HSG image to identify abnormal features; The normalization is to adjust the pixel values of the HSG image to the range of [0, 1] to reduce the influence of illumination and scale changes; The image chunking is to divide the image into several regions.

[0009] As a preferred solution of the present invention, the standardization is to unify the image resolution, brightness, contrast, and size.

[0010] As a preferred embodiment of the present invention, the denoising method is any one of Gaussian filtering, adaptive filtering, median filtering, bilateral filtering, and wavelet denoising.

[0011] As a preferred embodiment of the present invention, the image enhancement method is any one of histogram equalization, adaptive histogram equalization, Laplacian enhancement, and sharpening filter enhancement.

[0012] As a preferred embodiment of the present invention, the convolutional neural network model is any one of ResNet50, Vision Transformer, EfficientNet, VGG neural network model, and its derivative models.

[0013] An automated medical image classification and anomaly detection method based on deep learning, comprising the following steps: S1. Use an image processing module to perform cropping, standardization, denoising, normalization, and image block operations on the collected HSG images to ensure that the image quality meets the analysis requirements; S2. Input the HSG images preprocessed in step S1 into an image classification module, classify them through a second model, and output HSG images with classification labels, where the classification labels include A, B, C, and D, where: A is bilateral patency, B is unilateral patency, C is bilateral patency with obstruction, and D is bilateral obstruction; S3. Input the HSG images with classification labels output by the image classification module into a conditional guidance generation module, and generate new sample images through an occlusion experiment unit and a healthy image block generation unit in sequence; S4. Input the HSG images preprocessed in step S1 and the new sample images generated in step S3 into an anomaly detection module, perform anomaly detection on the new sample images and HSG images respectively through the anomaly detection module, obtain the prediction scores of the new sample images and HSG images, and perform subtraction calculation on the prediction scores of the new sample images and HSG images to quantify the difference scores of each region; S5. Input the difference scores obtained in step S4 into a heat map display module to generate a heat map matrix, mark the abnormal regions in the image, and superimpose the heat map on the original image to visually display the distribution and severity of the abnormal regions.

[0014] As a preferred embodiment of the present invention, the generation process of the new sample images includes the following steps: S31. The HSG images with classification labels pass through an occlusion experiment unit, randomly generate Gaussian noise blocks in the HSG images, and replace all regions in the HSG images with classification labels block by block to simulate different image damages or occlusions; S32. The HSG image with classification labels passes through the healthy image block generation unit, uses the classification labels to guide the generative adversarial network, generates healthy image blocks similar to the samples of classification label A, replaces the noise blocks in the occlusion experiment unit, and generates new sample images.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: An innovative deep learning-based automated medical image classification and anomaly detection system and method are provided. Through automated image processing, classification, and anomaly detection, accurate judgment of fallopian tube patency and anomalies can be achieved. It can analyze HSG images in real time, accurately judge the patency of the fallopian tubes, and locate abnormal areas. By generating healthy image blocks guided by classification and using difference calculation to effectively identify and locate subtle abnormal areas in the images, the problem that traditional manual analysis is easily affected by subjective factors and difficult to accurately identify is solved. Through heatmap visualization, efficient auxiliary diagnosis suggestions are provided for clinicians. This method is widely applicable to the detection requirements of fallopian tube anomalies and is expected to bring breakthrough progress to the field of HSG image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is the schematic diagram of the deep learning-based automated medical image classification and anomaly detection system; Figure 2 is the workflow diagram of the HSG image data processing module; Figure 3 is the network structure diagram of the four-classification of HSG images; Figure 4 is the workflow diagram of the image classification module; Figure 5 is the workflow diagram of generating healthy image samples based on the generative adversarial network (GAN); Figure 6 is the workflow diagram of the deep learning-based automated medical image classification and anomaly detection method; Figure 7 is the workflow diagram of difference calculation; Figure 8 is the heat matrix sample diagram. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] As an embodiment of the present invention, as Figure 1 shown, a deep learning-based automated medical image classification and anomaly detection system includes: An image processing module is responsible for preprocessing the collected HSG images; The image classification module includes a first model and a second model, which are used to identify the category of HSG images and output HSG images with classification labels; where: The first model is obtained by training a convolutional neural network model with a large medical image dataset; The second model is obtained by training the first model with preprocessed HSG images to obtain a pre-trained model, and then fine-tuning the pre-trained model; the fine-tuning is achieved by freezing the general feature layer of the pre-trained model and adjusting the classification layer; the classification labels include A, B, C, and D, where: A is bilateral patency, B is unilateral patency, C is bilateral partial patency, and D is bilateral obstruction; The conditional guidance generation module includes an occlusion experiment unit and a healthy image block generation unit, where: The occlusion experiment unit randomly generates Gaussian noise blocks from the HSG images input by the image classification module, and simulates image damage or occlusion by blocking different parts of the image block by block; The healthy image block generation unit generates healthy image blocks similar to class A HSG images through an adversarial network guided by the classification label, replaces the Gaussian noise blocks, and generates new sample images; The anomaly detection module determines the anomaly region by calculating the difference score of each region by subtracting the prediction scores of the original image and the new sample image; The heatmap display module marks the anomaly region according to the difference score, and superimposes the heatmap on the original image to display the distribution and severity of the anomaly region.

[0019] As an embodiment of the present invention, as Figure 2 shown, the image processing stage includes image cropping, normalization, denoising, image enhancement, normalization, and image tiling. Among them, the original HSG image is cropped to retain the key parts of the fallopian tubes and related anatomical structures, reduce background noise, and ensure that the image focuses on the analysis target area; normalization is performed on the cropped image to make the fallopian tube area clearer; denoising is to remove random noise and speckle and enhance the contour and details of the fallopian tubes; the image enhancement technology highlights the image details and makes the abnormal features easier to be recognized; normalization adjusts the pixel values of the image to the range of [0, 1] to reduce the influence of illumination and scale changes; finally, through image tiling, the image is divided into multiple small blocks to provide a more detailed analysis basis for subsequent classification and generation steps.

[0020] As an embodiment of the present invention, as Figure 3 shown, the classification neural network model adopts the ResNet50 network structure, and through multiple layers of convolution and skip connections, effectively captures the multi-level features of HSG images, enabling the classifier to accurately classify complex images and obtain classification results.

[0021] As an embodiment of the present invention, as Figure 4 shown, the image classification module includes a first model and a second model, wherein, The first model is trained using a large medical image dataset; The second model is obtained by training and fine-tuning a pre-trained model using HSG data; The second model is used to predict the category to which the HSG image belongs and the probability of the prediction. The classification result is not only used to judge the patency status of the fallopian tubes, but also serves as the input of the conditional guidance generation module to guide the generation of image patches conforming to the characteristics of a specific category, so as to replace the noise patches and generate new samples.

[0022] As an embodiment of the present invention, as Figure 5 shown, the method for generating healthy image samples of an image based on a generative adversarial network (GAN) includes two parts, wherein, The generator (G) takes a random noise vector and a category label as inputs and generates an image patch similar to a specific category (such as healthy class A).

[0023] The discriminator (D) extracts the features of the input image through convolutional layers, combines the category label, and calculates the authenticity score of the input image patch. It judges whether these image patches are consistent with the real data, and gradually improves its generation quality through continuous adjustment, making the output image patches closer to the real samples.

[0024] As an embodiment of the present invention, as Figure 6 shown, an automated medical image classification and anomaly detection method based on deep learning includes the following steps: S1: Train a convolutional neural network model using a large amount of medical image data to obtain a first model.

[0025] S2: The image preprocessing module performs preprocessing operations such as standardization, cropping, denoising, normalization, and image block division on the collected HSG images to ensure that the image quality meets the analysis requirements and provides a good basis for subsequent classification and generation.

[0026] S3: Further train and fine-tune the first model with the preprocessed images in S2 to obtain a second model.

[0027] S4: Generate new samples for the HSG images, including two parts: occlusion experiment and conditional guidance generation, specifically including the following steps: S41: Randomly generate Gaussian noise blocks in the image and replace different parts of the image block by block to simulate the situation of image damage or occlusion; S42: Use classification labels to guide the conditional generative adversarial network (CGAN) to generate healthy image patches similar to the normal bilateral patency (A) samples for replacing the noise patches in the occlusion experiment, thereby generating new samples.

[0028] S5: For anomaly detection, classify the new samples generated in S42 to obtain prediction scores, and perform subtraction calculation on the prediction scores of the generated new samples and the original images to quantify the difference scores for each region. S6: Generate a heatmap matrix based on the difference scores to mark the abnormal regions in the image. The redder the color on the heatmap, the greater the impact on the classification result, indicating that this region is more likely to be an abnormal region.

[0029] As an embodiment of the present invention, as Figure 7 shown, the method of difference calculation determines the abnormal regions by comparing the classification results of the generated new samples and the original samples. Quantify the difference between the generated samples and the original samples, that is, perform subtraction calculation on the prediction scores of the generated new sample X1 and the original HSG image X to obtain the difference scores. Through this comparison, it is possible to identify which regions have a large difference from the images of the normal class, thereby indicating potential abnormal regions.

[0030] As an embodiment of the present invention, as Figure 8 shown, the heatmap matrix shows the difference scores of each region in the image. Through the combination of color and value, the larger the value, the greater the difference between the features of this region and the healthy samples. At the same time, the color gradually changes from blue (low difference) to red (high difference). The redder the color of the region, the greater the impact on the classification result, that is, the higher the difference score of this region, indicating that there may be an abnormality.

[0031] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings, and thus do not limit the scope of rights of the embodiments of the present application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall fall within the scope of rights of the embodiments of the present application.

Claims

1. An automated medical image classification and anomaly detection system based on deep learning, characterized by: The system includes the following modules: The image processing module is responsible for preprocessing the collected HSG images; The image classification module includes a first model and a second model, which are used to identify the category of the HSG image and output the HSG image with a classification label; wherein: The first model was obtained by training a convolutional neural network model on a large medical image dataset; The second model is a pre-trained model obtained by training the first model with the pre-processed HSG image, and then obtained by fine-tuning the pre-trained model; the fine-tuning is achieved by freezing the general feature layer of the pre-trained model and adjusting the classification layer; the classification labels include A, B, C and D, where: A means bilateral unobstructed, B means unilateral unobstructed, C means bilateral unobstructed, and D means bilateral blocked; The conditional guided generation module includes an occlusion experiment unit and a healthy image block generation unit, wherein: The occlusion experiment unit randomly generates Gaussian noise blocks in the HSG image with classification labels input by the image classification module, and simulates image damage or occlusion by occluding different parts of the image block by block; Generate a healthy image block unit, generate a healthy image block similar to the HSG image of class A by using the classification label to guide the generated adversarial network, replace the Gaussian noise block, and generate a new sample image; The anomaly detection module determines the abnormal area by subtracting the prediction scores of the original image and the new sample image and quantifying the difference score of each area; The heat map display module marks the abnormal areas according to the difference scores and overlays the heat map on the original image to show the distribution and severity of the abnormal areas.

2. The deep learning-based automated medical image classification and anomaly detection system according to claim 1, characterized in that: The preprocessing includes cropping, standardization, denoising, image enhancement, normalization and image segmentation, wherein: The cropping is performed on the original HSG image to retain the key parts of the fallopian tube and related anatomical structures, reduce background noise, and ensure that the image is focused on the target area for analysis; The normalization is performed on the cropped HSG images to enhance the clarity of the fallopian tube region; The denoising is to remove random noise and speckle to enhance the contour and details of the fallopian tube; The image enhancement is to highlight the details of HSG images to identify abnormal features; The normalization is to adjust the pixel value of the HSG image to the range of [0, 1] to reduce the influence of illumination and scale changes; The image segmentation is to divide the image into several areas.

3. The deep learning-based automated medical image classification and anomaly detection system according to claim 2, characterized in that: The standardization is to unify the image resolution, brightness, contrast and size.

4. The deep learning-based automated medical image classification and anomaly detection system according to claim 2, characterized in that: The denoising method is any one of Gaussian filtering, adaptive filtering, median filtering, bilateral filtering and wavelet denoising.

5. The deep learning-based automated medical image classification and anomaly detection system according to claim 2, characterized in that: The image enhancement method is any one of histogram equalization, adaptive histogram equalization, Laplace enhancement, and sharpening filter enhancement.

6. The deep learning-based automated medical image classification and anomaly detection system according to claim 1, characterized in that: The convolutional neural network model is any one of ResNet50, Vision Transforme, EfficientNet, VGG neural network models and their derivative models.

7. A method for automated medical image classification and anomaly detection based on deep learning, characterized in that: The following steps are involved: S1. Use the image processing module to perform cropping, standardization, denoising, normalization and image segmentation operations on the collected HSG images to ensure that the image quality meets the analysis requirements; S2, inputting the HSG image preprocessed in step S1 into the image classification module, classifying it through the second model, and outputting the HSG image with classification labels, wherein the classification labels include A, B, C and D, wherein: A means bilateral unobstructed, B means unilateral unobstructed, C means bilateral unobstructed and D means bilateral blocked; S3, inputting the HSG image with classification labels output by the image classification module into the condition-guided generation module, and sequentially generating a new sample image through the occlusion experiment unit and the healthy image block generation unit; S4, inputting the HSG image preprocessed by step S1 and the new sample image generated by step S3 into the anomaly detection module, performing anomaly detection on the new sample image and the HSG image respectively through the anomaly detection module, obtaining the prediction scores of the new sample image and the HSG image, and performing subtraction calculation on the prediction score of the new sample image and the prediction score of the HSG image to quantify the difference score of each area; S5. Input the difference scores obtained in step S4 into the heat map display module to generate a heat map matrix, mark the abnormal areas in the image, and overlay the heat map on the original image to intuitively display the distribution and severity of the abnormal areas.

8. The method of automatic medical image classification and anomaly detection based on deep learning according to claim 7, characterized in that: The process of generating the new sample image comprises the following steps: S31, the HSG image with classification labels is subjected to an occlusion experiment unit, a Gaussian noise block is randomly generated in the HSG image, and all regions in the input HSG image with classification labels are replaced block by block to simulate different image damage or occlusion conditions; S32. The HSG image with classification label generates a healthy image block unit, guides the generative adversarial network with the classification label, generates a healthy image block similar to the sample with classification label A, replaces the Gaussian noise block in the occlusion experimental unit, and generates a new sample image.

Images