An automated medical image classification and anomaly detection system based on deep learning

Abstract

The present invention belongs to the technical field of medical image analysis and artificial intelligence, and relates to an automated medical image classification and anomaly detection system based on deep learning. An image preprocessing module is used to preprocess the collected HSG images; the preprocessed HSG images are input into an image classification module to output HSG images with classification labels; the HSG images with classification labels are input into a condition-guided generation module to generate new sample images; the preprocessed HSG images and the new sample images are input into an anomaly detection module for anomaly detection, and a heat map display module is used to intuitively display the distribution and severity of the abnormal areas; the present invention provides an automated, efficient and accurate image anomaly detection and localization method, which can effectively reduce the false detection and missed detection rates, improve the efficiency and accuracy of clinical diagnosis, and provide a more reliable auxiliary diagnosis tool for doctors.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image analysis and artificial intelligence, and relates to an automated medical image classification and anomaly detection system based on deep learning. Background Art

[0002] The fast-paced life and work pressure in modern society have made female fertility problems receive increasing attention. Among them, tubal factors are one of the important causes of female infertility. Hysterosalpingography (HSG), as the main examination method for evaluating tubal patency and structural abnormalities, is widely used in clinical practice. However, traditional HSG image analysis mainly relies on doctors' experience and visual judgment, which is not only time-consuming but also easily affected by subjective factors, resulting in poor diagnostic accuracy and consistency. With the increasing demand for fertility and the improvement of the requirements for precision medicine, it is obviously difficult to meet clinical needs solely by manual analysis. Especially when facing complex conditions and large amounts of data, the limitations of traditional analysis methods are gradually emerging.

[0003] At present, some automated methods based on image processing and analysis have been introduced into the medical field. However, in the application of HSG images, the existing technologies still fail to achieve ideal results in dealing with the fine structures of the fallopian tubes and identifying abnormalities such as blockage or partial blockage. Most of these methods cannot accurately capture the individual differences and lesion characteristics of the fallopian tubes, resulting in misdiagnosis or missed diagnosis from time to time. In addition, with the aggravation of the aging population in China, the demand for the detection of tubal abnormalities in elderly women has increased, further highlighting the deficiencies of the existing technologies in efficient and accurate analysis. Therefore, there is an urgent need to develop a more intelligent HSG image analysis technology to improve the efficiency and accuracy of diagnosis and provide more precise medical services for patients.

[0004] Automated medical image classification and anomaly detection algorithms based on deep learning, as an emerging image analysis method, show great potential in image recognition and anomaly detection. Through deep learning technology, precise analysis of complex structures in HSG images can be achieved, and abnormalities such as tubal patency, stenosis, and obstruction can be automatically identified. By using deep learning models such as convolutional neural networks, this technology can extract features, classify, and detect anomalies in images, effectively reducing the deviation of human judgment and realizing automated and efficient processing of HSG images. However, most of the existing algorithms are based on fixed models and parameters and do not fully consider the differences among individual patients, resulting in unsatisfactory detection effects in some complex or special cases.

[0005] To achieve individualized and precise HSG image analysis, the following core issues must be addressed: Firstly, the extraction and classification of individualized features. The algorithm needs to be able to flexibly adjust model parameters to adapt to the specific anatomical structures and lesion characteristics of the fallopian tubes of different patients. Common solutions include using pre-trained deep learning models for feature extraction and optimizing the classification model through transfer learning and fine-tuning methods to enable accurate identification of various situations such as bilateral patency, unilateral patency, bilateral partial obstruction, and bilateral obstruction. Secondly, the problem of precise localization of abnormal regions. The abnormal features in HSG images are usually very subtle. How to identify the abnormal regions during the image analysis process. We propose a brand-new method. By using Gaussian filtering occlusion and a conditional generative adversarial network (CGAN) to generate new samples of relatively healthy image patches, calculate the differences by comparing the original image and the predicted scores after replacing with healthy image patches, and generate a heatmap matrix to mark the abnormal regions. In addition, the generated heatmap needs to have good interpretability, be able to intuitively display the degree of influence and location of the abnormal regions, and help doctors make diagnoses and decisions quickly. Therefore, it is necessary to further optimize the difference analysis algorithm and the heatmap generation method to enhance the accuracy of the results and the interpretability of clinical applications. These improvements will help more accurately locate the abnormal regions in the fallopian tubes and significantly improve the effectiveness and reliability of diagnosis. Summary of the Invention

[0006] In view of the problems that the current HSG image analysis technology relies on manual judgment during the diagnosis process, is easily affected by subjective factors, and is difficult to accurately identify subtle abnormalities, the present invention proposes an automated medical image classification and anomaly detection system and method based on deep learning. The system includes an image processing module, an image classification module, a conditional guidance generation module, an anomaly detection module, and a heatmap display module, forming an overall logic of "acquisition - preprocessing - classification and guidance generation - difference analysis - localization". Through the automated processing and individualized analysis of HSG images, the system can real-time identify and locate the abnormal regions of the fallopian tubes, overcoming the limitations of existing image analysis methods. The present invention not only improves the accuracy and efficiency of diagnosis, but also reduces the dependence on doctors' experience, provides an individualized and automated image analysis solution, and provides strong support for clinical diagnosis and treatment decisions. The core innovation of the present invention lies in using a deep learning model fine-tuned with individualized data to real-time analyze the patency and lesion characteristics of the fallopian tubes and accurately locate the abnormal regions, significantly enhancing the reliability and effectiveness of hysterosalpingography diagnosis.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] An automated medical image classification and anomaly detection system based on deep learning, comprising:

[0009] The image processing module is responsible for preprocessing the collected HSG images;

[0010] The image classification module includes a first model and a second model, which are used to identify the category of the HSG image and output the HSG image with classification labels; where:

[0011] The first model is obtained by training a convolutional neural network model with a large medical image dataset;

[0012] The second model is obtained by training the first model with the preprocessed HSG images to obtain a pre-trained model, and then fine-tuning the pre-trained model; the fine-tuning is achieved by freezing the general feature layer of the pre-trained model and adjusting the classification layer; the classification labels include A, B, C, and D, where: A is bilateral patency, B is unilateral patency, C is bilateral patency with obstruction, and D is bilateral obstruction;

[0013] The conditional guidance generation module includes an occlusion experiment unit and a healthy image block generation unit, where:

[0014] The occlusion experiment unit randomly generates Gaussian noise blocks from the HSG images input by the image classification module, and simulates the image damage or occlusion situation by occluding different parts of the image block by block;

[0015] The healthy image block generation unit generates healthy image blocks similar to the HSG images of class A through an adversarial network guided by the classification label, replaces the Gaussian noise blocks, and generates new sample images;

[0016] The anomaly detection module determines the anomaly region by subtracting the prediction scores of the HSG images preprocessed by the image processing module and the new sample images, and quantifying the difference scores of each region;

[0017] The heat map display module marks the anomaly region according to the difference scores, and superimposes the heat map on the original image to display the distribution and severity of the anomaly region.

[0018] As a preferred solution of the present invention, the preprocessing includes cropping, standardization, denoising, image enhancement, normalization, and image chunking, where:

[0019] The cropping is performed on the collected HSG images to retain the key parts of the fallopian tubes and related anatomical structures, reduce background noise, and ensure that the images focus on the analysis target area;

[0020] The standardization is performed on the cropped HSG images to enhance the clarity of the fallopian tube region;

[0021] The denoising is to remove the random noise and speckles of the standardized HSG images to enhance the contour and details of the fallopian tubes;

[0022] The image enhancement is to highlight the details of the denoised HSG image to identify abnormal features;

[0023] The normalization is to adjust the pixel values of the HSG image after image enhancement to the range of [0, 1] to reduce the influence of illumination and scale changes;

[0024] The image block division is to divide the normalized HSG image into several regions.

[0025] As a preferred solution of the present invention, the standardization is to unify the image resolution, brightness, contrast and size.

[0026] As a preferred solution of the present invention, the denoising method is any one of Gaussian filtering, adaptive filtering, median filtering, bilateral filtering and wavelet denoising.

[0027] As a preferred solution of the present invention, the image enhancement method is any one of histogram equalization, adaptive histogram equalization, Laplacian enhancement, and sharpening filter enhancement.

[0028] As a preferred solution of the present invention, the convolutional neural network model is any one of ResNet50, Vision Transformer, EfficientNet, VGG neural network model and its derivative models.

[0029] An automated medical image classification and anomaly detection method based on deep learning, comprising the following steps:

[0030] S1. Use an image processing module to preprocess the collected HSG image. The preprocessing is completed by sequentially performing cropping, standardization, denoising, normalization and image block division operations to ensure that the image quality meets the analysis requirements;

[0031] S2. Input the HSG image preprocessed in step S1 into an image classification module, classify it through a second model, and output an HSG image with classification labels. The classification labels include A, B, C, and D, where: A is bilateral patency, B is unilateral patency, C is bilateral partial patency, and D is bilateral obstruction;

[0032] S3. Input the HSG image with classification labels output by the image classification module into a conditional guidance generation module, and generate new sample images through an occlusion experiment unit and a healthy image block generation unit in sequence; where:

[0033] The occlusion experiment unit randomly generates Gaussian noise blocks in the HSG image with classification labels input by the image classification module, and simulates image damage or occlusion situations by occluding different parts of the image block by block;

[0034] The healthy image block generation unit generates healthy image blocks similar to class A HSG images by using a classification label-guided generative adversarial network, replaces the Gaussian noise blocks, and generates a new sample image;

[0035] S4. Input the HSG image preprocessed in step S1 and the new sample image generated in step S3 into the anomaly detection module. The anomaly detection module performs anomaly detection on the new sample image and the HSG image respectively to obtain the prediction scores of the new sample image and the HSG image, and performs subtraction calculation on the prediction score of the new sample image and the prediction score of the HSG image to quantify the difference scores of each region;

[0036] S5. Input the difference scores obtained in step S4 into the heatmap display module to generate a heatmap matrix, mark the abnormal regions in the image, and overlay the heatmap on the original image to visually display the distribution and severity of the abnormal regions.

[0037] As a preferred solution of the present invention, the generation process of the new sample image includes the following steps:

[0038] S31. The HSG image with classification labels passes through the occlusion experiment unit, Gaussian noise blocks are randomly generated in the HSG image, and all regions in the HSG image with classification labels are replaced block by block to simulate different image damage or occlusion situations;

[0039] S32. The HSG image with classification labels passes through the healthy image block generation unit, uses a classification label-guided generative adversarial network to generate healthy image blocks similar to the samples of classification label A, replaces the noise blocks in the occlusion experiment unit, and generates a new sample image.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows: An innovative deep learning-based automated medical image classification and anomaly detection system and method are provided. This method realizes accurate judgment of fallopian tube patency and abnormalities through automated image processing, classification, and anomaly detection. It can analyze HSG images in real time, accurately judge the patency of the fallopian tubes, and locate abnormal regions. By generating healthy image blocks guided by classification and using difference calculation to effectively identify and locate subtle abnormal regions in the image, it solves the problem that traditional manual analysis is easily affected by subjective factors and difficult to accurately identify. Through heatmap visualization, it provides efficient auxiliary diagnosis suggestions for clinicians. This method is widely applicable to the detection needs of fallopian tube abnormalities and is expected to bring breakthrough progress to the field of HSG image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is the schematic diagram of the deep learning-based automated medical image classification and anomaly detection system;

[0042] Figure 2 It is the workflow diagram of the HSG image data processing module;

[0043] Figure 3 It is the network structure diagram of the four-classification of HSG images;

[0044] Figure 4 It is the workflow diagram of the image classification module;

[0045] Figure 5 It is the workflow diagram of generating healthy image samples based on the Generative Adversarial Network (GAN);

[0046] Figure 6 It is the workflow diagram of the automated medical image classification and anomaly detection method based on deep learning;

[0047] Figure 7 It is the workflow diagram of difference calculation;

[0048] Figure 8 It is a sample diagram of the heat matrix. Detailed implementation manners

[0049] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0050] As an embodiment of the present invention, as Figure 1 shown, an automated medical image classification and anomaly detection system based on deep learning includes: an image processing module responsible for preprocessing the collected HSG images;

[0051] The image classification module includes a first model and a second model, which are used to identify the category of the HSG image and output the HSG image with a classification label; where:

[0052] The first model is obtained by training a convolutional neural network model with a large medical image dataset;

[0053] The second model is obtained by training the first model with the preprocessed HSG images to obtain a pre-trained model, and then fine-tuning the pre-trained model; the fine-tuning is achieved by freezing the general feature layer of the pre-trained model and adjusting the classification layer; the classification labels include A, B, C, and D, where: A is bilateral patency, B is unilateral patency, C is bilateral patency with obstruction, and D is bilateral obstruction;

[0054] The conditional guidance generation module includes an occlusion experiment unit and a healthy image block generation unit, where:

[0055] The occlusion experiment unit randomly generates Gaussian noise blocks from the HSG images input by the image classification module, and simulates the image damage or occlusion situation by occluding different parts of the image block by block;

[0056] A healthy image block generation unit generates healthy image blocks similar to Class A HSG images by using a generative adversarial network guided by classification labels, replaces the Gaussian noise blocks, and generates new sample images;

[0057] An anomaly detection module determines anomaly regions by calculating the difference scores of each region through subtracting the prediction scores of the preprocessed HSG images and the new sample images by the image processing module;

[0058] A heatmap display module marks the anomaly regions according to the difference scores, overlays the heatmap on the original image, and displays the distribution and severity of the anomaly regions.

[0059] As an embodiment of the present invention, as Figure 2 shown, the image processing stage includes image cropping, normalization, denoising, image enhancement, normalization, and image tiling. Among them, the collected HSG images are cropped to retain the key parts of the fallopian tubes and related anatomical structures, reduce background noise, and ensure that the images focus on the analysis target region; normalization is to process the cropped images to make the fallopian tube region clearer; denoising is to remove random noise and speckles and enhance the contours and details of the fallopian tubes; image enhancement techniques highlight the image details to make the abnormal features easier to identify; normalization adjusts the pixel values of the images to the range of [0, 1] to reduce the influence of illumination and scale changes; finally, through image tiling, the image is divided into multiple small blocks to provide a more detailed analysis basis for subsequent classification and generation steps.

[0060] As an embodiment of the present invention, as Figure 3 shown, the classification neural network model adopts the ResNet50 network structure, and through multi-layer convolution and skip connections, effectively captures the multi-level features of the HSG images, enabling the classifier to accurately classify complex images and obtain classification results.

[0061] As an embodiment of the present invention, as Figure 4 shown, the image classification module includes a first model and a second model, where

[0062] The first model is trained using a large medical image dataset;

[0063] The second model is obtained by training and fine-tuning a pre-trained model using HSG data;

[0064] The second model is used to predict the category and prediction probability of the HSG image. The classification result is not only used to judge the patency state of the fallopian tubes, but also serves as the input of the conditional generation module to guide the generation of image blocks conforming to the specific category features, thereby replacing the noise blocks and generating new samples.

[0065] As an embodiment of the present invention, as Figure 5 shown, the health image sample generation based on the generative adversarial network (GAN) includes two parts, where

[0066] The generator (G) takes a random noise vector and a class label as inputs and generates image patches similar to a specific class (such as healthy class A).

[0067] The discriminator (D) extracts the features of the input image through convolutional layers, combines the class label, and calculates the authenticity score of the input image patch. It determines whether these image patches are consistent with the real data and gradually improves its generation quality through continuous adjustment, making the output image patches closer to the real samples.

[0068] As an embodiment of the present invention, as Figure 6 shown, an automated medical image classification and anomaly detection method based on deep learning includes the following steps:

[0069] S1: Train a convolutional neural network model using a large amount of medical image data to obtain a first model.

[0070] S2: The image preprocessing module performs preprocessing operations such as standardization, cropping, denoising, normalization, and image chunking on the collected HSG images to ensure that the image quality meets the analysis requirements and provides a good foundation for subsequent classification and generation.

[0071] S3: Further train and fine-tune the first model with the preprocessed images in S2 to obtain a second model.

[0072] S4: Generate new samples for the HSG images, including two parts: occlusion experiment and conditional guided generation, specifically including the following steps:

[0073] S41: Randomly generate Gaussian noise blocks in the image and replace different parts of the image block by block to simulate image damage or occlusion.

[0074] S42: Use the classification label-guided generative adversarial network (CGAN) to generate healthy image patches similar to normal bilateral patency (A) samples to replace the noise blocks in the occlusion experiment, thereby generating new samples.

[0075] S5: Use the new samples generated in S42 for anomaly detection classification to obtain prediction scores, and perform subtraction calculation on the prediction scores of the generated new samples and the original images to quantify the difference scores of each region.

[0076] S6: Generate a heatmap matrix based on the difference scores and mark the abnormal regions in the image. The redder the color on the heatmap, the greater the impact on the classification result, indicating that this region is more likely to be an abnormal region.

[0077] As an embodiment of the present invention, as Figure 7 shown, the method of difference calculation determines the abnormal region by comparing the classification results of the generated new samples and the original samples. Quantify the difference between the generated samples and the original samples, that is, subtract the predicted score of the generated new sample X1 from the predicted score of the original HSG image X to obtain the difference score. Through this comparison, it is possible to identify which regions have a large difference from the images of the normal class, thus indicating potential abnormal regions.

[0078] As an embodiment of the present invention, as Figure 8 shown, the heatmap matrix shows the difference scores of each region in the image. Through the combination of color and value, the larger the value, the greater the difference between the characteristics of this region and the healthy samples. At the same time, the color gradually changes from blue (low difference) to red (high difference). The redder the color of the region, the greater the impact on the classification result, that is, the higher the difference score of this region, indicating that there may be an abnormality.

[0079] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. However, this does not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall fall within the scope of the rights of the embodiments of the present application.

Claims

1. An automated medical image classification and anomaly detection system based on deep learning, characterized in that: The system includes the following modules: An image processing module, which is responsible for preprocessing the collected HSG images; An image classification module, including a first model and a second model, which is used to identify the category of the HSG image and output the HSG image with classification labels; where: The first model is obtained by training a convolutional neural network model with a large medical image dataset; The second model is a pre-trained model obtained by training the first model with the preprocessed HSG images, and then obtained by fine-tuning the pre-trained model; the fine-tuning is achieved by freezing the general feature layer of the pre-trained model and adjusting the classification layer; the classification labels include A, B, C, and D, where: A is bilateral patency, B is unilateral patency, C is bilateral patency with obstruction, and D is bilateral obstruction; A conditional guidance generation module, including an occlusion experiment unit and a healthy image block generation unit, where: The occlusion experiment unit randomly generates Gaussian noise blocks in the HSG image with classification labels input by the image classification module, and simulates image damage or occlusion by occluding different parts of the image block by block; The healthy image block generation unit generates healthy image blocks similar to the HSG images of class A through an adversarial network guided by the classification label, replaces the Gaussian noise blocks, and generates new sample images; An anomaly detection module determines the anomaly region by calculating the difference score of each region by subtracting the prediction scores of the HSG images preprocessed by the image processing module and the new sample images; A heatmap display module marks the anomaly region according to the difference score, and superimposes the heatmap on the original image to display the distribution and severity of the anomaly region.

2. The automated medical image classification and anomaly detection system based on deep learning according to claim 1, characterized in that: The preprocessing includes cropping, standardization, denoising, image enhancement, normalization, and image tiling, where: The cropping is performed on the collected HSG images to retain the key parts of the fallopian tubes and related anatomical structures, reduce background noise, and ensure that the image focuses on the analysis target region; The standardization is performed on the cropped HSG images to enhance the clarity of the fallopian tube region; The denoising is to remove the random noise and speckles of the standardized HSG images to enhance the contour and details of the fallopian tubes; The image enhancement is to highlight the details of the denoised HSG images to identify abnormal features; The normalization is to adjust the pixel values of the image-enhanced HSG images to the range of [0, 1] to reduce the influence of illumination and scale changes; The image tiling is to divide the normalized HSG images into several regions.

3. An automated medical image classification and anomaly detection system based on deep learning according to claim 2, characterized in that: The standardization is to unify the image resolution, brightness, contrast, and size.

4. An automated medical image classification and anomaly detection system based on deep learning according to claim 2, characterized in that: The method of the denoising is any one of Gaussian filtering, adaptive filtering, median filtering, bilateral filtering, and wavelet denoising.

5. An automated medical image classification and anomaly detection system based on deep learning according to claim 2, characterized in that: The method of the image enhancement is any one of histogram equalization, adaptive histogram equalization, Laplacian enhancement, and sharpening filter enhancement.

6. An automated medical image classification and anomaly detection system based on deep learning according to claim 1, characterized in that: The convolutional neural network model is any one of ResNet50, Vision Transforme, EfficientNet, VGG neural network model, and its derivative models.

7. An automated medical image classification and anomaly detection method based on deep learning, characterized in that: It includes the following steps: S1. Use an image processing module to preprocess the collected HSG images. The preprocessing is completed by sequentially performing cropping, normalization, denoising, normalization, and image chunking operations to ensure that the image quality meets the analysis requirements; S2. Input the HSG images preprocessed in step S1 into the image classification module, classify them through a second model, and output HSG images with classification labels. The classification labels include A, B, C, and D, where: A is bilateral patency, B is unilateral patency, C is bilateral partial patency, and D is bilateral obstruction; S3. Input the HSG images with classification labels output by the image classification module into the conditional guidance generation module, and generate new sample images through an occlusion experiment unit and a healthy image chunk generation unit in sequence; where: Occlusion experiment unit: In the HSG images with classification labels input by the image classification module, randomly generate Gaussian noise chunks, and simulate image damage or occlusion situations by occluding different parts of the image block by block; Healthy image chunk generation unit: Generate healthy image chunks similar to class A HSG images by using an adversarial network guided by classification labels, replace the Gaussian noise chunks, and generate new sample images; S4. Input the HSG images preprocessed in step S1 and the new sample images generated in step S3 into the anomaly detection module. The anomaly detection module performs anomaly detection on the new sample images and HSG images respectively, obtains the prediction scores of the new sample images and HSG images, and performs subtraction calculation on the prediction scores of the new sample images and HSG images to quantify the difference scores of each region; S5. Input the difference scores obtained in step S4 into the heat map display module to generate a heat map matrix, mark the abnormal regions in the image, and overlay the heat map on the original image to intuitively display the distribution and severity of the abnormal regions.

8. An automated medical image classification and anomaly detection method based on deep learning according to claim 7, characterized in that: The generation process of the new sample images includes the following steps: S31. The HSG images with classification labels pass through the occlusion experiment unit, randomly generate Gaussian noise chunks in the HSG images, and replace all regions in the input HSG images with classification labels block by block to simulate different image damage or occlusion situations; S32. The HSG images with classification labels pass through the healthy image chunk generation unit, use the classification labels to guide the generation of an adversarial network, generate healthy image chunks similar to the samples of classification label A, replace the Gaussian noise chunks in the occlusion experiment unit, and generate new sample images.

Images