Fetal craniocerebral detection method combining deep learning and region-of-interest pooling

Abstract

The invention discloses a deep learning and region-of-interest pooling combined fetal craniocerebral detection method, and relates to the technical field of intelligent medical assistance, and the method comprises the steps: collecting historical fetal image data of different pregnancy periods, fetal positions and imaging conditions, and carrying out the preprocessing; according to the method, a fetus craniocerebral detection model is constructed, a multi-task learning method is introduced, detection and segmentation tasks are carried out at the same time, a self-attention mechanism is used for capturing long-distance dependence, and a model architecture comprises a shared feature extraction network, an ROI Pooling layer, a detection branch and a segmentation branch; pre-training a fetus brain detection model by using the pre-processed fetus image data; and based on a pre-trained fetal craniocerebral detection model and the target fetal image data, predicting to obtain a fetal craniocerebral detection result. According to the method, the efficiency and accuracy of fetal brain detection are improved, the adaptivity of the model under different fetal positions, pregnancy periods and imaging conditions is enhanced, and more reliable and flexible technical support is provided for clinical diagnosis.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical assistance technology, and more specifically, to a fetal brain detection method that combines deep learning with region-of-interest pooling. Background Technology

[0002] Medical imaging technology is a crucial tool in modern medical diagnosis, widely used for early disease detection, treatment efficacy evaluation, and postoperative monitoring. In recent years, advancements in technologies such as magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound have significantly improved the resolution and quality of medical images. These technologies provide physicians with clear images of internal structures, aiding in accurate diagnoses. However, despite the continuous development of imaging equipment, the processing and analysis of medical image data still face many challenges. Traditional medical image analysis primarily relies on manual interpretation and rule-based image processing methods. While manual interpretation offers high accuracy, it is inefficient and susceptible to subjective influences when dealing with large volumes of image data. Particularly in fetal brain examinations, physicians need to analyze images frame-by-frame to determine normal brain development and identify potential abnormalities. This method is not only time-consuming but also demands extremely high levels of expertise from physicians. Rule-based image processing methods typically employ a series of predefined algorithms, such as edge detection, region segmentation, and morphological manipulation. These methods perform well when processing simple and high-contrast images, but when faced with complex medical images, especially those with noise, artifacts, or uneven contrast, these traditional algorithms often fall short.

[0003] Fetal cranial examination is a crucial part of routine prenatal checkups, aiming to ensure normal fetal brain development and detect potential abnormalities promptly. Normal brain development is vital for the fetal nervous system and overall health, while abnormal brain structures may indicate serious neurological disorders such as hydrocephalus or cerebral palsy. If these abnormalities are not identified and addressed in a timely manner, they can have long-term negative impacts on fetal growth and development. However, fetal cranial examination faces several challenges. First, the frequent postures and movements of the fetus in the uterus lead to unstable image quality, potentially resulting in blurriness or noise interference. This makes accurate identification and localization of brain regions in images extremely difficult. Second, existing image processing techniques often fail to provide sufficient accuracy and robustness when processing these complex images, especially in segmentation and localization tasks, where traditional algorithms often perform poorly.

[0004] Current fetal brain detection technologies rely on manual interpretation and rule-based algorithms. These methods have the following main drawbacks: Traditional manual analysis methods are inefficient and time-consuming when processing large amounts of image data. Doctors need to examine images frame by frame, perform manual annotation and diagnosis, which is not only labor-intensive but also prone to errors. Rule-based image processing methods often exhibit poor performance when faced with image noise, blur, or uneven contrast, resulting in insufficient accuracy in the segmentation and localization of brain regions, which may lead to missed or false detections. Traditional algorithms usually rely on preset rules and thresholds, lacking adaptive capabilities and unable to effectively handle images with different fetal positions, gestational ages, and imaging conditions, which limits their application in complex and variable clinical environments. Summary of the Invention

[0005] The present invention provides a fetal brain detection method that combines deep learning with region-of-interest pooling, which can alleviate the above-mentioned problems.

[0006] To alleviate the above problems, the technical solution adopted by the present invention is as follows:

[0007] This invention provides a method for fetal brain detection combining deep learning and region-of-interest pooling, comprising the following steps:

[0008] S1. Collect historical fetal image data under different gestational ages, fetal positions, and imaging conditions, and perform preprocessing;

[0009] S2. Construct a fetal brain detection model, which introduces a multi-task learning method to perform detection and segmentation tasks simultaneously. It uses a self-attention mechanism to capture long-distance dependencies. The architecture includes a shared feature extraction network, a ROIPooling layer, a detection branch, and a segmentation branch.

[0010] S3. Pre-train the fetal brain detection model using the pre-processed fetal image data;

[0011] S4. Fetal brain detection results are obtained based on the pre-trained fetal brain detection model and target fetal image data prediction.

[0012] This invention significantly reduces the workload of doctors analyzing fetal brain images and improves the efficiency of processing large amounts of image data through automated deep learning algorithms. Utilizing the self-learning capability of the fetal brain detection model reduces errors caused by human factors and improves diagnostic accuracy. The fetal brain detection model, through training, can adapt to images with different fetal positions, gestational ages, and imaging conditions, enhancing the algorithm's generalization ability. Compared to traditional rule-based methods, the fetal brain detection model can adaptively adjust parameters without manually setting thresholds, improving the algorithm's flexibility and accuracy. Through multi-task learning, the model can simultaneously perform detection and segmentation tasks, improving its efficiency and accuracy. Using a self-attention mechanism, the model can capture long-distance dependencies in images, facilitating the analysis of complex images. Through region-of-interest pooling layers, the model can focus more on key regions in the image, further improving detection accuracy. This invention has broad clinical application potential, providing doctors with more accurate and reliable diagnostic information, thereby assisting in the early detection and treatment of fetal brain diseases. Pre-training with a large amount of historical data allows the model to learn richer features, improving the reliability of detection results.

[0013] Specifically, preprocessing methods for fetal image data include annotation, data augmentation, and normalization. Annotation refers to labeling the regions of interest in the fetal brain. Data augmentation includes geometric transformation, color transformation, noise addition, image flipping, and cutout. Normalization refers to normalizing the image pixel values ​​to the range of [0,1] or [-1,1].

[0014] This approach ensures accurate identification of the fetal brain region through precise annotation. Data augmentation techniques improve the model's robustness to image variations. By employing geometric and color transformations, noise addition, image flipping, and cutout, various real-world scenarios are simulated. Normalization ensures the consistency and stability of the model's input. These comprehensive measures significantly enhance the generalization ability and accuracy of the fetal brain detection model, laying a solid foundation for subsequent deep learning model training.

[0015] Specifically, the shared feature extraction network is used to extract image features from preprocessed fetal image data using ResNet, and a self-attention mechanism is added to enhance the feature representation to obtain a feature map.

[0016] The shared feature extraction network combines ResNet and self-attention mechanism to effectively extract and enhance fetal image features. The generated feature map has deeper semantic information, which not only improves the model's ability to analyze complex images, but also significantly enhances the accuracy and robustness of detection.

[0017] Specifically, the ROIPooling layer is used to combine the feature map output by the shared feature extraction network with the labeled fetal brain region of interest, and perform pooling operation to obtain the feature map after ROI pooling.

[0018] The ROIPooling layer technology achieves effective pooling of feature maps by accurately locating the fetal brain region, ensuring the retention of key information and the removal of irrelevant information. This process significantly improves the model's recognition accuracy of brain structures, while optimizing computational efficiency, providing refined and representative input for subsequent detection and segmentation tasks.

[0019] Specifically, the detection branch is used to classify and locate the feature map after ROI pooling using convolutional and fully connected layers.

[0020] The detection branch, through a deep learning architecture of convolutional and fully connected layers, achieves accurate classification and localization prediction of ROI pooled feature maps, improving the model's recognition rate of cranial abnormalities and ensuring the accuracy and reliability of detection results.

[0021] Specifically, the segmentation branch is used to perform pixel-level segmentation of the feature map after ROI pooling using the U-Net structure, and outputs a mask of the brain.

[0022] The segmentation branch uses a U-Net structure for pixel-level accurate segmentation, effectively outputting a mask of the fetal brain region, enhancing the model's ability to identify brain boundaries, improving segmentation accuracy, and providing more accurate anatomical information for clinical diagnosis.

[0023] Preferably, for the loss function used to train the fetal brain detection model, cross-entropy loss is selected for classification, smooth L1 loss for localization, and Dice loss for segmentation.

[0024] The loss function design is tailored to different task characteristics. Cross-entropy loss is used to optimize classification accuracy, smoothed L1 loss is used to improve the accuracy of localization prediction, and Dice loss is used to enhance the effect of pixel-level segmentation. This combination strategy of multiple loss functions not only ensures the balanced optimization of the model in classification, localization and segmentation tasks, but also significantly improves the overall detection performance, enabling the model to more accurately identify and distinguish fetal brain structures, and providing a more accurate analytical tool for clinical diagnosis.

[0025] Preferably, after using the pre-trained fetal brain detection model to predict the target fetal image data and obtain the prediction results, the prediction results need to be post-processed using the non-maximum suppression method to remove duplicate detection boxes.

[0026] Post-processing the prediction results using nonmaximum suppression effectively removes overlapping detection boxes, improving the clarity and accuracy of the detection results, optimizing the model output, avoiding redundant information, and ensuring the uniqueness and representativeness of the detection boxes. This enhances the practicality of the model in real-world applications and the accuracy of clinical diagnosis. The post-processing steps also improve the model's adaptability to complex scenarios, providing doctors with a more concise and accurate diagnostic basis.

[0027] Preferably, during the application of the pre-trained fetal brain detection model, it is periodically retrained or fine-tuned using the latest fetal image data to optimize model performance. This is achieved by using incremental learning to retain the knowledge of the old model and incorporate new data.

[0028] By employing incremental learning through regular retraining or fine-tuning, the fetal brain imaging model continuously adapts to new clinical data and changes. This method preserves the model's existing knowledge while effectively incorporating new image data, improving the model's adaptability and generalization ability to new situations. The retraining process not only optimizes model performance but also enhances its stability and reliability under different gestational stages, fetal positions, and imaging conditions. Furthermore, through incremental learning, the model can quickly absorb new knowledge, reducing the time and resources required for training from scratch and improving the efficiency of model updates. This provides clinicians with a continuously evolving and optimized diagnostic tool.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] This invention combines deep learning technology with multi-task learning, introduces a self-attention mechanism to effectively capture long-distance dependencies in images, enhances the model's ability to recognize complex image features, and utilizes ROIPooling technology to focus on key regions, significantly improving detection accuracy. The overall solution not only improves the efficiency and accuracy of fetal brain detection, but also enhances the model's adaptability under different fetal positions, gestational periods, and imaging conditions, reduces human error, and provides more reliable and flexible technical support for clinical diagnosis.

[0031] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, embodiments of the present invention are described in detail below. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0033] This invention provides a method for fetal brain detection combining deep learning and region-of-interest pooling, comprising the following steps:

[0034] S1. Collect historical fetal image data under different gestational ages, fetal positions, and imaging conditions, and perform preprocessing.

[0035] In this embodiment, we can collaborate with multiple hospitals to obtain fetal images at different gestational ages (e.g., 12 weeks, 20 weeks, 32 weeks), different fetal positions (e.g., cephalic presentation, breech presentation, transverse presentation), and different imaging conditions (e.g., 2D ultrasound, 3D ultrasound, MRI). The collected image data is stored in a secure database to ensure data privacy and security.

[0036] The preprocessing of historical fetal image data includes annotation, data augmentation, and normalization.

[0037] The professional medical image annotation tools used can be Labelbox or VGG Image Annotator (VIA) to annotate the fetal brain region in each image, draw the bounding box or outline of the ROI (Region of Interest), and review the annotation after completion to ensure the accuracy and consistency of the annotation.

[0038] Data augmentation can generate more samples, improving the robustness and generalization ability of the model. Data augmentation methods include geometric transformation, color transformation, noise addition, image flipping, and cutout.

[0039] Geometric transformations include rotation, translation, scaling, and shearing. Rotation randomly rotates an image by a certain angle (e.g., ±10 degrees); translation randomly shifts the image's position in the horizontal or vertical direction; scaling randomly scales an image to generate images of different sizes; and shearing randomly crops an image, causing distortion.

[0040] Color transformation includes adjusting the brightness, contrast, and saturation of an image. Brightness adjustment randomly increases or decreases the image's brightness; contrast adjustment randomly increases or decreases the image's contrast; and saturation adjustment randomly increases or decreases the image's saturation.

[0041] Noise addition involves adding Gaussian noise, salt-and-pepper noise, etc., to an image to simulate noise conditions in real-world imaging. Image flipping involves horizontally and vertically flipping the image to generate images in different orientations. Cutout randomly occludes parts of an image to increase data diversity.

[0042] Normalization includes pixel value normalization, mean normalization, and standard deviation normalization.

[0043] Pixel value normalization normalizes image pixel values ​​to the range of [0,1] or [-1,1], facilitating the training of deep learning models. For example, if the pixel value range of the input image is [0,255], then the formula can be used. Normalize the pixel values ​​to the range [0,1].

[0044] Normalizing an image by mean and standard deviation gives the image data a zero mean and a unit standard deviation. For example, for each image X, the formula can be used... Normalization is performed, where μ and σ are the mean and standard deviation of the training set image data, respectively.

[0045] S2. Construct a fetal brain detection model, which introduces a multi-task learning method to perform detection and segmentation tasks simultaneously. A self-attention mechanism is used to capture long-distance dependencies. The architecture includes a shared feature extraction network, a ROIPooling layer, a detection branch, and a segmentation branch.

[0046] A shared feature extraction network is used to extract image features from preprocessed fetal image data using ResNet, and a self-attention mechanism is added to enhance the feature representation, resulting in a feature map F. The residual connections of ResNet can alleviate the gradient vanishing problem, and the self-attention mechanism improves the model's ability to capture global information.

[0047] The ROIPooling layer combines the feature map F output by the shared feature extraction network with the labeled regions of interest (ROIs) of the fetal brain, performing a pooling operation to obtain the ROI-pooled feature map P. ij ROIPooling transforms ROI features of different sizes into a fixed size, facilitating subsequent processing. The specific formula is as follows:

[0048]

[0049] Among them, P ij This is the output after pooling, R ij It is the ij-th ROI pooling region, F xy These are the pixel values ​​of the input feature map.

[0050] The detection branch is used to process the feature map P after pooling the ROI using convolutional and fully connected layers. ij Classification and localization predictions are performed to obtain detection results including individual characteristics and location. Classification predicts the presence or absence of the fetal brain, and localization predicts the specific location of the brain. The specific formula is as follows:

[0051] Class(P ij =Softmax(W) c ·P ij +b c )

[0052] Among them, Class(P) ij () is the classification result, indicating whether the fetal brain is present. c This is the weight matrix for the classification task, bc It is the bias vector.

[0053] Box(P ij ) = W b ·P ij +b b

[0054] Among them, Box(P) ij This is the positioning result, indicating the location of the fetal brain (Bounding Box). b This is the weight matrix for the localization task, b b It is the bias vector.

[0055] The split branch is used to pool the feature map P of the ROI using the U-Net structure. ij Pixel-level segmentation is performed, outputting a mask of the brain. The segmentation branch provides fine-grained pixel-level information, improving overall detection performance; the specific formula is as follows:

[0056] S out =σ(W s *P ij +b s )

[0057] Among them, S out It is the predicted segmentation mask, representing the probability that each pixel belongs to the brain, σ is the activation function (such as Sigmoid), and W s It is the convolution kernel, b s It is a bias.

[0058] S3. Use the preprocessed fetal image data to pre-train the fetal brain detection model.

[0059] In this embodiment, the preprocessed fetal image data is divided into a training set, a validation set, and a test set. Data segmentation can effectively evaluate the performance of the model and prevent overfitting.

[0060] For the loss functions used in training the fetal brain detection model, cross-entropy loss was chosen for classification, smooth L1 loss for localization, and Dice loss for segmentation. Using multiple loss functions comprehensively optimizes model performance.

[0061] The cross-entropy loss function is:

[0062]

[0063] Among them, L CE This is the cross-entropy loss for a classification task, where N is the number of samples. It's a real label. It is a predicted probability.

[0064] The smoothed L1 loss function is:

[0065]

[0066] Among them, L SmoothL1 It is a smoothed L1 loss. These are the actual bounding box coordinates. These are the predicted BoundingBox coordinates.

[0067] The Dice loss function is:

[0068]

[0069] Among them, L Dice It is Dice loss, S out YYY is the predicted segmentation mask, and YYY is the actual segmentation mask.

[0070] During training, the Adam optimizer is selected to adjust model parameters, which can efficiently handle large-scale data training. The validation set is input into the optimized model to evaluate model performance, and the model effect can be improved by adjusting hyperparameters and network structure.

[0071] In the test set, we used 10 labeled fetal brain images to evaluate the trained fetal brain detection model. The model made predictions on the test images and compared them with the real labeled images, as shown in Table 1.

[0072] Table 1

[0073]

[0074] Evaluation metrics include precision, recall, F1 score, and average IOU. Based on the data in the table above, calculate the model's evaluation metrics:

[0075] 1. Precision rate refers to the proportion of samples predicted as positive that are actually positive. The calculation process is as follows:

[0076]

[0077] 2. Recall rate refers to the proportion of samples that are actually positive but are correctly predicted as positive. The calculation process is as follows:

[0078]

[0079] 3. The F1 score is the harmonic mean of precision and recall, and its calculation process is as follows:

[0080]

[0081] 4. IOU is the ratio of the intersection to the union of the predicted bounding box and the ground truth bounding box. The average IOU is 0.76.

[0082] The results show that the model achieved an average precision of 90% on the test set, indicating that 90% of the predicted brain regions were correct; the average recall was 90%, indicating that 90% of the real brain regions were correctly detected by the model. The average F1 score was 90%, which is the harmonic mean of precision and recall, indicating that the model performed well in balancing precision and recall. Furthermore, the average IOU was 76%, indicating a high degree of overlap between the predicted bounding boxes and the ground truth bounding boxes.

[0083] In summary, the performance of the trained fetal brain detection model on the test set demonstrates its high accuracy and robustness in fetal brain detection tasks, and it can be effectively applied to practical medical image analysis.

[0084] The trained fetal brain imaging model is deployed to the medical imaging system, specifically as follows:

[0085] Deployment environment selection: If the medical institution has ample computing resources and strict requirements for data privacy, the model can be deployed on a local server; or computing resources from cloud service providers (such as AWS, Google Cloud, Azure) can be used to dynamically adjust computing power according to needs.

[0086] Deployment methods: The model is encapsulated as an API service, and the medical imaging system interacts with the model via HTTP requests. Common technologies include Flask, FastAPI (for Python), or Java Spring Boot (for Java). Alternatively, the model can be embedded into the local system of the medical imaging device for offline processing.

[0087] Save the trained model in a deployment-appropriate format (such as ONNX, TensorFlow SavedModel, or PyTorchTorchScript). Ensure all dependencies and environment configurations are met in the deployment environment. For example, use Docker containers to package the application and its dependencies.

[0088] The system input interface supports accepting various medical image formats (such as DICOM, JPEG, and PNG), providing an image upload interface or an interface for integration with a Picture Archiving and Communication System (PACS). The system output interface displays the detection results (such as brain location and segmentation masks) in a visual manner. Output results may include classification labels, bounding boxes, segmentation masks, etc., and support storage of results in the medical record system. Ensuring data transmission security and privacy protection utilizes encrypted transmission protocols (such as HTTPS). Data de-identification is implemented to avoid leaking patient privacy.

[0089] S4. Fetal brain detection results are obtained based on the pre-trained fetal brain detection model and target fetal image data prediction.

[0090] In this embodiment, a pre-trained fetal brain detection model is used to predict the target fetal image data. After obtaining the prediction results, the non-maximum suppression method is used to post-process the prediction results to remove duplicate detection boxes.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A fetal brain detection method combining deep learning and region of interest pooling, characterized in that: The following steps are involved: S1. Collect historical fetal image data of different gestational periods, fetal positions and imaging conditions, and perform preprocessing; S2. Construct a fetal brain detection model, which introduces a multi-task learning method to perform detection and segmentation tasks simultaneously, using a self-attention mechanism to capture long-range dependencies. The model architecture includes a shared feature extraction network, a ROIPooling layer, a detection branch, and a segmentation branch. S3, using the preprocessed fetal image data to pre-train a fetal cranial brain detection model; S4. Predicting the fetal brain detection result based on the pre-trained fetal brain detection model and the target fetal image data.

2. The fetal brain detection method combining deep learning and region of interest pooling according to claim 1 is characterized in that: The methods for preprocessing fetal image data include labeling, data enhancement and normalization, where labeling refers to labeling the fetal brain region of interest, data enhancement includes geometric transformation, color transformation, noise addition, image flipping and Cutout, and normalization refers to normalizing the image pixel values ​​to the range of [0,1] or [-1,1].

3. The fetal cranial brain detection method combined with deep learning and region of interest pooling according to claim 1 is characterized in that: The shared feature extraction network is used to extract image features from the preprocessed fetal image data using ResNet, and a self-attention mechanism is added to enhance the feature representation to obtain a feature map.

4. The fetal brain detection method combining deep learning and region of interest pooling according to claim 3 is characterized in that: The ROIPooling layer is used to combine the feature map output by the shared feature extraction network with the annotated fetal brain region of interest, perform a pooling operation, and obtain a feature map after ROI pooling.

5. The fetal brain detection method combining deep learning and region of interest pooling according to claim 4 is characterized in that: The detection branch is used to classify and locate the feature map after ROI pooling using convolutional layers and fully connected layers.

6. The fetal cranial brain detection method combined with deep learning and region of interest pooling according to claim 5 is characterized in that: The segmentation branch is used to perform pixel-level segmentation on the feature map after ROI pooling using the U-Net structure and output the mask of the brain.

7. The fetal brain detection method combining deep learning and region of interest pooling according to claim 1, characterized in that: For the loss function used in training the fetal brain detection model, cross entropy loss is selected for classification, smooth L1 loss is used for positioning, and Dice loss is used for segmentation.

8. The fetal cranial brain detection method combined with deep learning and region of interest pooling according to claim 1, characterized in that: After using the pre-trained fetal brain detection model to predict the target fetal image data and obtain the prediction results, it is necessary to use the non-maximum suppression method to post-process the prediction results to remove duplicate detection frames.

9. The fetal cranial brain detection method combined with deep learning and region of interest pooling according to claim 1, characterized in that: During the application of the pre-trained fetal cranial brain detection model, the latest fetal image data is used to regularly retrain or fine-tune it to optimize the model performance, which uses an incremental learning method to retain the knowledge of the old model and incorporate new data.

10. The fetal brain detection method combining deep learning and region of interest pooling according to claim 1, characterized in that: The method is applied to medical imaging systems involving fetal brain detection.

Images