Methods for screening fetal chromosomal abnormalities based on nuchal translucency thickness measurements

Abstract

This invention discloses a method for screening fetal chromosomal abnormalities based on the measurement of nuchal translucency thickness in the field of image processing technology. The method comprises the following steps: Step 1: Reading fetal scan data and performing noise reduction preprocessing to obtain video data to be processed; Step 2: Processing the video data to be processed according to preset rules and then performing image processing to obtain first preprocessed data; Step 3: Importing the first preprocessed data into the NT_SEG segmentation model to obtain second preprocessed data; Step 4: Performing contour segmentation on the second preprocessed data using the Snake algorithm, using PCA to obtain its minor axis as the nuchal translucency thickness, and obtaining the minimum bounding rectangle of the NT elliptical region to obtain the NT thickness measurement value; Step 5: Comparing the NT thickness measurement value with the actual standard value to determine whether the fetal chromosome is abnormal. This invention utilizes image processing technology and a deep learning model to achieve automatic identification and measurement of the nuchal translucency region of the fetus, enabling doctors to make a preliminary judgment on chromosomal abnormalities in the fetus.

Description

Technical Field

[0001] This invention relates to the fields of image classification, image segmentation, and traditional digital image processing, specifically a method for screening fetal chromosomal abnormalities based on measurements of nuchal translucency thickness. Background Technology

[0002] Nuchal translucency (NT) detection and thickness measurement are milestones in predicting fetal and chromosomal abnormalities in ultrasound images. Nuchal translucency refers to fluid accumulation at the base of the fetal neck and is closely associated with chromosomal abnormalities leading to early pregnancy cardiac arrest. In hospitals, sonographers typically assess nuchal translucency thickness based on the midsagittal plane, a crucial step in prenatal screening.

[0003] Pathological deposits of cervical fluid (edema and cysts), primarily observed in the mid and late stages of pregnancy, are associated with several chromosomal abnormalities. Analyzing cervical fluid in the first trimester to determine the likelihood of chromosomal differences at each stage is a routine procedure performed by technicians, which is very time-consuming and requires skilled personnel. Additionally, nuchal translucency (NT) is observable and can be determined via ultrasound imaging between 11 and 14 weeks of gestation. An enlarged NT is associated with numerous fetal chromosomal and non-chromosomal differences. Increasing evidence suggests that excessive NT thickness in the first trimester can be influenced by various fetal structural differences, genetic disorders, cardiac problems, and adverse perinatal conditions such as miscarriage and intrauterine death. A reference value for nuchal translucency in relation to crown-rump length should be considered in the first trimester to determine an acceptable cutoff threshold for assessing defects associated with increased NT thickness. Therefore, chromosomal disorders can be associated with increased nuchal translucency, and a range of pathologies resulting from chromosomal disorders are associated with Down syndrome, Edwards syndrome, Pato syndrome, Turner syndrome, and other cardiac defects; obtaining NT thickness using traditional methods is a very time-consuming procedure and heavily relies on the operator's expertise. Summary of the Invention

[0004] The purpose of this invention is to provide a method for screening fetal chromosomal abnormalities based on nuchal translucency thickness measurements, in order to solve the problem mentioned in the background art that obtaining NT thickness using traditional methods is a very time-consuming task and heavily dependent on the operator's professional skills.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for screening fetal chromosomal abnormalities based on nuchal translucency thickness measurements, specifically comprising the following steps:

[0006] Step 1: Read the fetal scan video data, and perform noise reduction preprocessing on the video data to obtain the video data to be processed;

[0007] Step 2: The video data to be processed is filtered according to preset rules and then processed by image processing to obtain the first preprocessed data;

[0008] Step 3: Import the first preprocessed data into the NT_SEG segmentation model for processing to obtain the second preprocessed data;

[0009] Step 4: The second preprocessed data is segmented using the Snake algorithm to calculate and generate the target contour and perform post-processing to obtain the segmented contour. The segmented contour is then displayed in the original image. PCA is used to obtain its minor axis as the thickness of NT, and the minimum bounding rectangle of the NT elliptical region is obtained. The width of the rectangle is calculated, and the measured value of the NT thickness is obtained.

[0010] Step 5: Compare the measured NT thickness with the actual standard value to determine if the fetal chromosomes are abnormal.

[0011] Preferably, in step 1, after performing noise reduction preprocessing on the video data, the video data to be processed is obtained by using a video filtering network to filter the video quality of the video data and removing video frames with quality estimates below a threshold, thus obtaining the video data to be processed.

[0012] Preferably, the video filtering network is a ResNet-based regression network, with an input of size 3×512×512 and an output of image quality estimate.

[0013] Preferably, in step 2, the video data to be processed is processed and filtered according to preset rules, and after image processing, the first preprocessed data is obtained. Specifically, after the video data to be processed is filtered according to preset rules to obtain the best midsagittal plane, it is preprocessed and then dynamically cropped using ROI regions. The NT annotation coordinate information is calculated, the minimum bounding box of the annotation region is extracted, the annotation region is expanded, the boundary is expanded according to 15% of the width and height of the bounding box, and then the original image is cropped according to the same size to obtain the original image and the corresponding mask image used for segmentation as the first preprocessed data.

[0014] Specifically, the image preprocessing involves converting the original RGB color image into a grayscale image, using Wiener filtering for image denoising, and then processing the denoised image using a limited contrast histogram.

[0015] Preferably, the preset rule is to use YOLOv5 to detect the video data to be processed, calculate the weighted probability value of the video data to be processed, and select the video data with a probability value greater than a preset probability threshold as the best midsagittal plane.

[0016] The preset probability threshold is 80%.

[0017] Preferably, the NT_SEG segmentation model includes an encoding branch and a decoding branch;

[0018] The encoding branch includes a spatial encoding branch and a semantic encoding branch. The spatial encoding branch introduces a GAM module. The semantic encoding branch uses the Mobilenetv2 model as the backbone model for downsampling, and performs feature downsampling by 2, 4, 8, 16, and 32 times respectively. The decoding branch introduces a GAM module in its upsampling path.

[0019] Preferably, in step 4, obtaining the minimum bounding rectangle of the NT elliptical region specifically involves using OpenCV to obtain the minimum bounding rectangle of the NT elliptical region.

[0020] Compared with existing technologies, the beneficial effects of this invention are as follows: Quality assessment of ultrasound video improves the quality of data input to the model and reduces the interference of invalid frames on model performance; ROI region identification and cropping maximizes the learning of the target region and reduces interference from a large amount of background information; NT_SEG nuchal translucency segmentation enables preliminary segmentation of the NT region. To further optimize the segmentation effect, this method combines a boundary-based active contour segmentation method, using the NT_SEG segmentation mask to identify the NT contour. Finally, after processing, an optimized segmentation image is obtained. This allows for the automatic identification and measurement of the fetal nuchal translucency area using image processing technology and a deep learning model. Based on the measurement indicators, doctors can initially assess the risk of chromosomal abnormalities in the fetus, helping to detect fetal chromosomal abnormalities in a timely manner and facilitate timely diagnosis and treatment. Attached Figure Description

[0021] Figure 1 This is a flowchart of the present invention;

[0022] Figure 2 This is a flowchart of the NT_SEG segmentation model of the present invention;

[0023] Figure 3 This is a schematic diagram of the GAM module structure of the present invention;

[0024] Figure 4 This is a schematic diagram of the semantic downsampling module structure of the present invention;

[0025] Figure 5 This is a schematic diagram of the ultrasound image processing process of the present invention;

[0026] Figure 6 This is a schematic diagram of the video filtering network of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Please see Figure 1 :

[0029] Step 1: Obtain high-quality fetal ultrasound scan video data; design a video filtering network to filter the video quality of fetal ultrasound scan video data, and remove video frames with quality estimates below the threshold, thereby obtaining high-quality scan videos and improving the reliability of the data source.

[0030] Among them, such as Figure 6 As shown, the video filtering network is a ResNet-based regression network. The input of the video filtering network is 3×512×512, and the output of the video filtering network is the image quality estimate.

[0031] (1) The training data consists of ultrasound images and image quality ground truth values ​​annotated by doctors. The image quality evaluation is a continuous value between 0 and 1. Low quality images are usually caused by the probe acquiring too fast, black screen due to poor contact between the probe and the body, ghosting caused by obstruction, and meaningless frames that do not contain any organs. The data is divided into training set and test set in a 4:1 ratio.

[0032] (2) The model was built using the PyTorch deep learning framework and trained using an NVIDIA GTX2080Ti. The minimum batch size was set to 32. The Adam optimizer with an initial learning rate of 0.0001 was used. The learning rate was reduced by 0.1 times every 60 iterations of the entire training set, for a total of 100 training iterations. The training updated the model parameters by minimizing the mean squared error (MSE) loss. Data augmentation such as random rotation, brightness, contrast, horizontal flipping, image compression, and blurring were used to accelerate model training and improve generalization.

[0033] (3) During inference, the video frame with a height and width of 244 is output as the image quality estimate.

[0034] (4) For the image quality estimate output in (3), when the image quality estimate is less than the threshold, the current frame is judged as a low-quality video frame, and the current frame is removed from the original video to obtain a high-quality scan video.

[0035] Please see Figure 1Step 2: Use YOLOv5 to detect the video processed in Step 1 (YOLOv5 is used for the recognition of midsagittal plane (MSP) anatomical structures), calculate the target weighted probability value, and when the detection probability is greater than the probability threshold (the probability threshold is set to 80%), the best MSP (midsagittal plane) is detected; by using YOLOv5 to detect the midsagittal plane in the video data, the efficiency of the subsequent NT segmentation measurement task is ensured.

[0036] Please see Figure 1 Step 3: Preprocess the midsagittal plane obtained in Step 2 by converting the original RGB color image into a grayscale image, using Wiener filtering for image denoising, and then using a limited contrast histogram to enhance the edge information of the image and increase the contrast of anatomical structures in the ultrasound image; the input image resolution is uniformly set to 512x512 to reduce the impact of excessively large image resolution on the task.

[0037] Please see Figure 1 and Figure 5 Step 4: Dynamic cropping of ROI regions (to improve segmentation efficiency since the NT region occupies a small proportion of the entire image), calculate the NT annotation coordinate information, extract the minimum bounding box of the annotation region, expand the annotation region, expand the boundary by 15% of the width and height of the bounding box, and then crop the original image according to the same size to obtain the original image and the corresponding mask image used for segmentation.

[0038] Please see Figure 1 and Figure 2 Step 5: Design the NT_SEG segmentation model, which consists of an encoding branch and a decoding branch. The encoding branch is further divided into a spatial encoding branch and a semantic encoding branch. The spatial encoding branch utilizes a larger-scale feature map to encode richer spatial information, while the semantic encoding branch acquires a larger receptive field of image features and learns high-level semantic information. In addition, a GAM module is introduced into the spatial encoding information and the upsampling path to re-optimize and combine the model features. The original image and mask image after cropping and adjustment in Step 4 are input into the model for training to obtain the optimal segmentation model weights. The model is then used to predict the test set to obtain the predicted mask image.

[0039] Among them, such as Figure 2 and Figure 4As shown, step 51: The NT_SEG segmentation model, inspired by the U-shaped encoder-decoder framework, adopts a U-shaped encoding and decoding form. That is, in the encoding path, this model uses a dual-branch structure: the left branch is the spatial encoding branch, and the right branch is the semantic encoding branch. Simultaneously, a GAM module is introduced into the spatial encoding branch to optimize the spatial distribution of low-level features, suppress interference from background features, and enhance the representability of low-dimensional features. In the semantic downsampling module of the semantic encoding branch, the MobileNetv2 model is used as the backbone model for downsampling, performing feature downsampling at factors of 2, 4, 8, 16, and 32 respectively to learn higher-dimensional semantic information. In downsampling, a global information extraction layer is designed to better capture high-level semantic information. Global average pooling and residual connections are used to learn global contextual feature information. In addition, in the model upsampling path, the traditional form of connecting corresponding features at the encoder and decoder ends is not adopted because this simple feature fusion method will not efficiently recover the feature loss during downsampling. Instead, it will greatly increase the model's computational parameters and waste resources. The fusion of feature information from the two branches at the encoder end can completely make up for the feature loss during downsampling. The GAM module in the upsampling process can suppress the amplification interference of image noise, making the model's image scale recovery more accurate.

[0040] Step 52: First, divide the preprocessed image and the corresponding mask image into training and validation sets (ratio of 8:2, the processed data are the original image and mask image after cropping and adjustment in step 4). Input the images and masks in the training set into the NT_SEG segmentation model for training, and automatically adjust the input scale to 512x512.

[0041] Step 53: The input image passes through both the spatial encoding branch and the semantic encoding branch in the encoding path. In the spatial encoding branch, the model goes through three sets of convolutional structures with batch normalization and ReLU activation function and a stride of 2. At the same time, it goes through the GAM module to obtain three sets of spatial features at different scales: 32x256x256, 64x128x128, and 128x64x64. Then it goes through a convolutional structure with batch normalization and ReLU activation function and a stride of 1 to obtain the spatial encoded output feature of 128x64x64.

[0042] Step 54: The input image processed in step 52 is subjected to five sets of semantic downsampling to obtain feature maps of the following scales: 32x256x256, 64x128x128, 128x64x64, 256x32x32, and 512x16x16. After passing through the global information extraction layer, the final semantic feature encoding path output of 512x16x16 is obtained.

[0043] Step 55: Perform a one-step downsampling on the feature output of Step 53 to obtain the downsampled feature output of the spatial coding branch: 256x32x32. Perform a one-step upsampling on the feature output of Step 54 to obtain the upsampled output of the semantic coding branch: 256x32x32.

[0044] Step 56: Multiply the final output of the spatial coding branch generated in Step 53 with the upsampled output of the semantic coding branch element-wise to obtain the features. Figure 1 The scale is 256x32x32; the final output of the semantic encoding branch generated in step 54 is multiplied element-wise with the downsampled output of the spatial encoding branch to obtain the features. Figure 2 Its scale is 256x32x32; finally, the features are... Figure 1 With features Figure 2 The summation yields the final output feature of the encoder, with a scale of 256x32x32.

[0045] Step 57: For the feature output at the encoding end, perform a one-step convolution, and then input it into the decoding path for four upsampling steps for decoding. At the decoding end, each time the feature is input into the upsampling structure, it must first go through the GAM module to adjust the feature. After four upsampling steps, feature maps of 128x64x64, 64x128x128, 32x256x256, and 3x513x512 are obtained. Finally, after one-step convolution and softmax activation function, the final segmentation result is obtained.

[0046] Step 58: Build the model using the PyTorch deep learning framework and train it on a GTX 2080Ti. Set the minimum batch size to 4. Use the Adam optimizer with an initial learning rate of 0.00001. Reduce the learning rate by a factor of 0.1 every 60 iterations of the entire training set, for a total of 100 training iterations. Design a loss function, introducing the Focal loss function as the objective function. During training, it reduces the loss contribution of simple multi-sample training, making the model focus more on learning targets that are difficult to train and have few samples. The calculation formula is: FL(p t )=-α t (1-p t ) γ log(p t ), where α t To balance the focus loss, p represents the ratio of positive to negative samples. t This is the predicted label, and γ is the modulation factor. When γ is 0, the focus loss function is equivalent to the classification cross-entropy. As γ increases, the modulation factor (1-p)... t The effect of α also increases, and experiments have shown that the effect is best when α is 0.25 and γ is 2.

[0047] Step 59: Save the model trained in Step 58 and verify the model's segmentation performance.

[0048] It should be noted that, as Figure 3 As shown, the workflow of the GAM module is as follows:

[0049] Step S1: Perform adaptive average pooling on the input feature map. Assume the input feature map is F. input ∈R H ×W×C Feature F is obtained through adaptive average pooling operation. aav ∈R 1×1×C Adaptive average pooling does not change the number of channels in the feature map, but compresses the feature map to a minimum, which not only reduces the amount of computation, but also helps the model learn the salient information in the feature map.

[0050] Step S2: Perform a convolution operation on the feature map generated in step S1, setting the kernel size to 1, further compressing the number of feature channels with a compression ratio of 8, and finally activating it using the ReLU function to obtain the feature map F. conv_1 ∈R 1×1×C / 8 .

[0051] Step S3: Perform convolution channel restoration on the feature map generated in step S2, set the convolution kernel size to 1, and finally apply Sigmoid activation to obtain feature map F. conv_2 ∈R 1×1×C .

[0052] Step S4: Multiply the input feature map from step S1 with the output feature map from step S3 to obtain the channel attention feature output F. c_att ∈R H×W×C .

[0053] Step S5: Apply max pooling to the feature map generated in step S4 along the channel dimension to enhance semantic information, resulting in feature map F. c_mp ∈R 1×H×W After completing the spatial feature extraction and mapping, the feature is activated by the Sigmoid activation function. The output feature is then multiplied with the feature map generated in step S4 to obtain the final output feature F. space ∈R H×W×C This refers to spatial attention characteristics.

[0054] Please see Figure 1 and Figure 5Step 6: Obtain the target contour region based on the predicted mask, perform contour segmentation on the target region using the edge-based active contour segmentation method (i.e., Snake algorithm for contour segmentation), calculate and generate the target contour, and then perform post-processing on the contour to obtain the final segmented contour, and display it in the original image.

[0055] Please see Figure 1 and Figure 5 Step 7: The segmented NT region in Step 6 is a connected component. The region is approximated as an ellipse. PCA is used to obtain its minor axis as the thickness of the NT. OpenCV is used to obtain the minimum bounding rectangle of the NT elliptical region, and then the width of the rectangle is calculated, which is approximated as the thickness of the NT.

[0056] Please see Figure 1 Step 8: Compare the NT thickness value in Step 7 with the actual standard value. If the thickness difference exceeds the normal range, it indicates that there is a risk of abnormal chromosomal development in the fetus, and further diagnosis and treatment are required. If the measured value is within the specified range, no treatment is required.

[0057] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for screening fetal chromosomal abnormalities based on nuchal translucency thickness measurements, characterized in that: Specifically, the following steps are included: Step 1: Read the fetal scan video data, and perform noise reduction preprocessing on the video data to obtain the video data to be processed; Step 2: The video data to be processed is filtered according to preset rules and then processed by image processing to obtain the first preprocessed data; Step 3: Import the first preprocessed data into the NT_SEG segmentation model for processing to obtain the second preprocessed data; Step 4: The second preprocessed data is segmented using the Snake algorithm to calculate and generate the target contour and perform post-processing to obtain the segmented contour. The segmented contour is then displayed in the original image to obtain the segmented NT region. The NT region is approximated as an NT ellipse. PCA is used to obtain its minor axis as the thickness of the NT, and the minimum bounding rectangle of the NT ellipse region is obtained. The width of the rectangle is calculated to obtain the measured value of the NT thickness. Step 5: Compare the measured NT thickness with the actual standard value to determine if the fetal chromosomes are abnormal; In step 2, the video data to be processed is filtered and processed by preset rules, and after image processing, the first preprocessed data is obtained. Specifically, after the video data to be processed is filtered by preset rules to obtain the optimal midsagittal plane, the optimal midsagittal plane is preprocessed, and the ROI region is dynamically cropped. The NT annotation coordinate information is calculated, the minimum bounding box of the annotation region is extracted, the annotation region is expanded, the boundary is expanded according to 15% of the width and height of the bounding box, and then the original image is cropped according to the same size to obtain the original image and the corresponding mask image used for segmentation as the first preprocessed data. Specifically, the image preprocessing of the optimal midsagittal plane involves converting the original RGB color image into a grayscale image, using Wiener filtering for image denoising, and then processing the denoised image using a limited contrast histogram. The preset rule specifically involves using YOLOv5 to detect the video data to be processed, calculating the weighted probability value of the video data to be processed, and selecting video data with a probability value greater than a preset probability threshold as the optimal midsagittal plane. The preset probability threshold is 80%. The NT_SEG segmentation model includes an encoding branch and a decoding branch; The encoding branch includes a spatial encoding branch and a semantic encoding branch. The spatial encoding branch introduces a GAM module. The semantic encoding branch uses the Mobilenetv2 model as the backbone model for downsampling, and performs feature downsampling by 2, 4, 8, 16, and 32 times respectively. The decoding branch introduces a GAM module in its upsampling path. The first preprocessed data is imported into the NT_SEG segmentation model for processing to obtain the second preprocessed data. Specifically, the original image and the corresponding mask image used for segmentation are input into the NT_SEG segmentation model for training to obtain the optimal segmentation model weights. The test set is then used to make predictions, and the predicted mask image is obtained as the second preprocessed data.

2. The method for screening fetal chromosomal abnormalities based on nuchal translucency thickness measurement according to claim 1, characterized in that: In step 1, after performing noise reduction preprocessing on the video data, the video data to be processed is obtained by using a video filtering network to filter the video quality of the video data and removing video frames with quality estimates below a threshold, thus obtaining the video data to be processed.

3. The method for screening fetal chromosomal abnormalities based on nuchal translucency thickness measurement according to claim 2, characterized in that: The video filtering network is a ResNet-based regression network. The input of the video filtering network is 3×512×512, and the output of the video filtering network is an image quality estimate.

4. The method for screening fetal chromosomal abnormalities based on nuchal translucency thickness measurement according to claim 1, characterized in that: In step 4, obtaining the minimum bounding rectangle of the NT elliptical region specifically involves using OpenCV to obtain the minimum bounding rectangle of the NT elliptical region.

Images