Ultrasound imaging method of diaphragm

Abstract

A diaphragmatic ultrasound radiomics method, after achieving automatic segmentation of the diaphragm region, constructs a U-Net segmentation model with an encoder-decoder structure offline. This model is trained on a pre-processed dataset, and the trained U-Net segmentation model is used to test bimodal videos to obtain predicted segmentation results. These predicted segmentation results are then post-processed. Keyframes are selected from the input video by combining the post-processed predicted segmentation results with the video's dynamic information. High-throughput radiomics quantitative parameters are extracted from the predicted segmentation results corresponding to the keyframes and filtered. These filtered radiomics parameters, combined with other variable information, are used to train a support vector machine (SVM)-based classification model, which is then tested online. This invention utilizes the dynamic information of video and the radiomics features of bimodal images, combining these features with other variable information, resulting in greater convenience and efficiency.

Description

Technical Field

[0001] This invention relates to a technique in the field of image processing, specifically a diaphragmatic ultrasound imaging method. Background Technology

[0002] Bedside ultrasound of the diaphragm is widely used due to its advantages of being non-invasive, radiation-free, easy to operate, and providing indicators related to diaphragmatic function. Shear wave elastography (SWE) is an emerging ultrasound imaging technique that provides biomechanical information related to tissue structure. Its key feature is quantifying the velocity of shear waves and converting it into modulus values. Related studies have shown that SWE technology can reflect diaphragmatic function by quantifying the modulus values ​​of diaphragmatic tissue. Clinically, doctors usually need to manually delineate the diaphragm region from SWE images to obtain modulus values. This process is not only inconvenient but also easily influenced by subjectivity, and it also overlooks important imaging features outside the selected area. Summary of the Invention

[0003] This invention addresses the shortcomings of existing medical radiomics analysis techniques, such as failing to incorporate dynamic information from video, inability to integrate multimodal information, lack of post-processing or optimization of predicted segmentation results, insufficient feature extraction from each image leading to many features being overlooked, and reliance solely on quality assessment for image selection without considering the inherent characteristics of the images, resulting in incomplete and inefficient selection methods. This invention proposes a diaphragmatic ultrasound radiomics method. After automatically segmenting the diaphragm region, the predicted segmentation results are post-processed and mapped to another modality. Keyframes are then selected by combining the predicted segmentation results with the dynamic information of the video. High-throughput radiomics quantitative parameters are automatically extracted from the predicted segmentation results of these keyframes and then filtered. The filtered radiomics quantitative parameters, along with other variable information, are used for classification. This invention utilizes the dynamic information of video, leverages the radiomics features of bimodal images, and combines these features with other variable information, resulting in greater convenience and efficiency.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a diaphragmatic ultrasound radiomics method. It involves constructing a U-Net segmentation model with an encoder-decoder structure offline, training it on a preprocessed dataset, testing the trained U-Net segmentation model on bimodal videos to obtain predicted segmentation results, post-processing the predicted segmentation results, and then selecting keyframes from the input video by combining the post-processed predicted segmentation results with the dynamic information of the input video. High-throughput radiomics quantitative parameters are extracted from the predicted segmentation results corresponding to the keyframes and filtered. The filtered radiomics parameters, combined with other variable information, are used to train a support vector machine (SVM)-based classification model. A grid search method is used to tune the parameters to find the optimal classification model, and the trained classification model is then tested online.

[0006] The U-Net segmentation model includes: a 5-layer encoder for extracting feature maps, a 4-layer decoder for restoring the size of the feature maps, 4 skip connection layers connecting the encoder and decoder, and a 1×1 convolutional layer placed after the 4th decoder layer.

[0007] The U-Net segmentation model described above performs frame extraction on the video data input during training, but does not perform frame extraction on the video data input during online testing. The model can learn a lot of information from a small amount of data, making it efficient and with good generalization.

[0008] The preprocessing refers to: automatically detecting the boundaries of B-type images and SWE images using edge detection and Hough transform, cropping them using boundary coordinate points, and then filtering the images using the spontaneous breathing test (SBT) index.

[0009] The classification model incorporates a grid search method during parameter tuning. Specifically, within a given parameter range, the parameters are iteratively adjusted at a specified step size, and a classifier is trained based on these parameters. Then, the set of parameters that enables the classifier to achieve the highest accuracy is found from all parameter combinations to construct the optimal classification model.

[0010] This invention relates to a diaphragmatic ultrasound radiomics system for implementing the above-mentioned method, comprising: a data preprocessing unit, a segmentation model training and validation unit, a predicted segmentation result post-processing and keyframe selection unit, a feature extraction and filtering unit, and a classification model cross-validation unit. Specifically: the data preprocessing unit acquires B-mode-SWE bimodal diaphragmatic ultrasound videos and performs video frame extraction, image cropping, image conversion, image annotation, and data partitioning to obtain a dataset; the segmentation model training and validation unit constructs a segmentation network for segmentation testing and trains the segmentation network using a training set; the predicted segmentation result post-processing and keyframe selection unit optimizes the predicted segmentation results output by the segmentation network and selects keyframes from the input video based on the optimized predicted segmentation results and dynamic information from the input video; the feature extraction and filtering unit extracts high-throughput radiomics quantitative parameters from the predicted segmentation results corresponding to the keyframes and filters out radiomics parameters that are helpful for classification; the classification model cross-validation unit trains the classification network based on radiomics parameters and other variable information and uses it for online testing.

[0011] Technical effect

[0012] This invention segments the data in one modality and then maps the predicted segmentation results to another modality. It post-processes the predicted segmentation results using tissue characteristics and morphological processing methods. Keyframes are selected by combining the optimized predicted segmentation results with the video's dynamic information. Classification is performed using image omics parameters and other variables. Compared to existing technologies, this invention employs frame extraction during segmentation network training, efficiently learning a large amount of information from a small amount of data; the predicted segmentation results in one modality can be mapped to another; the keyframe selection method, combined with the video's dynamic information, is highly efficient; and the classification model exhibits stable performance. Attached Figure Description

[0013] Figure 1 This is a flowchart of the present invention;

[0014] Figure 2 This is a schematic diagram illustrating the data preprocessing and image segmentation network training process.

[0015] In the figure: a is a schematic diagram of data preprocessing principle; b is a schematic diagram of U-Net segmentation model;

[0016] Figure 3 This is a schematic diagram of the network segmentation testing process;

[0017] Figure 4 Schematic diagram of diaphragmatic ultrasound radiomics method;

[0018] Figure 5 This is an example diagram illustrating the predicted segmentation results;

[0019] In the figure: a shows the gold standard predicted segmentation result and the predicted segmentation result for the Type B image in Example 1; b shows the gold standard predicted segmentation result and the predicted segmentation result for the paired SWE image in Example 1; c shows the gold standard predicted segmentation result and the predicted segmentation result for the Type B image in Example 2; d shows the gold standard predicted segmentation result and the predicted segmentation result for the paired SWE image in Example 2. Detailed Implementation

[0020] like Figure 1 As shown, this embodiment relates to a diaphragmatic ultrasound imaging method, including:

[0021] Step 1: Obtain the dataset: Obtain B-mode-SWE bimodal diaphragmatic ultrasound video and other variable information.

[0022] Other variables include: age, gender, physiological parameters, and ventilation time.

[0023] Step 2: Data preprocessing to save annotation costs, remove irrelevant regions from video frames, and convert images to the required format, specifically as follows: Figure 2 As shown, it includes:

[0024] Step 2.1, Data partitioning: Divide the data into a training set, a validation set, and a test set.

[0025] Step 2.2, Video Frame Extraction: Since the frame rate of the B-mode SWE bimodal diaphragmatic ultrasound video is 7 frames per second, and the similarity between adjacent frames is high, frame extraction is performed on the videos used for segmentation training and validation sets, i.e., one frame is taken every 3 frames. The SBT index of the frame is then checked to see if it meets the condition, i.e., whether the number of stars in the upper right corner is greater than 3. If it is greater than 3, the frame is considered to be of acceptable quality, and the image is saved for later use.

[0026] Step 2.3, Image Cropping: Edge detection and Hough transform are used to detect the boundary lines and intersection points of two rectangular target regions in the image. The intersection points are the vertices of the rectangular regions. Using the obtained vertices, a paired B-type image and a SWE image with a grayscale background are cropped. The B-type image is subtracted from the SWE image with the grayscale background to obtain the SWE image.

[0027] Step 2.4, Image Conversion: The color bar in the upper left corner of the image shows the mapping relationship between the colors in the SWE image and the actual elastic modulus values. By finding the optimal corresponding position of the color of each pixel in the SWE image on the color bar, the SWE image can be converted into a softness map.

[0028] Step 2.5, Image Annotation: Use the Labelme annotation tool to outline the diaphragm boundary in the B-type image and generate a binary label image.

[0029] Step 3, Training and Testing of the Segmentation Network: Constructing a network as follows Figure 2 The model shown in b, used for automatic segmentation of the diaphragm region in type B images, was trained and tested to obtain predicted segmentation results, specifically including:

[0030] Step 3.1: Construct a U-Net segmentation model with an encoder-decoder structure, specifically including: a 5-layer encoder for extracting feature maps, a 4-layer decoder for restoring the size of the feature maps, and a 1×1 convolutional layer placed after the decoder. There are skip connections between the encoder and decoder layers to fuse the corresponding feature maps. The 1×1 convolutional layer determines whether a pixel is background or foreground based on the output of the decoder and generates a predicted segmentation result.

[0031] In the five-layer encoder for extracting feature maps, the convolutional layers in the shallow encoder have a smaller receptive field and focus on extracting the texture features of the image; the convolutional layers in the deep encoder have a larger receptive field and focus on extracting the semantic information of the image. Each encoder layer consists of two 3×3 convolutional layers with rectified linear units (ReLU) and a 2×2 max pooling layer.

[0032] The feature map output by the coding layer is cropped at the center to meet the size requirements for feature map fusion.

[0033] In the decoder, each layer consists of one upsampling layer and two 3×3 convolutional layers with ReLU.

[0034] Step 3.2: Training the segmentation model: The images in the training and validation sets need to be standardized in size before being input into the network for training. To avoid overfitting during training, the validation set is used to validate the model after each training round, and the model that performs best on the validation set is saved as the final model.

[0035] Step 3.3: Test the segmentation network: After framing all B-type-SWE bimodal diaphragmatic ultrasound videos into ordered image sequences and performing image cropping and image transformation, input the B-type image sequences one by one into the trained segmentation model to obtain the predicted segmentation results. Post-process the predicted segmentation results. Since the paired B-type images and SWE images have positional consistency, the optimized predicted segmentation results can be mapped to the SWE modality.

[0036] The post-processing refers to: using the positional characteristics of the diaphragm in the B-type image and morphological processing methods to optimize the predicted segmentation results obtained from the test and reduce the erroneous parts in the predicted segmentation results. Specifically, the dilation and erosion method is first used to fill the small holes in the segmentation results and remove the small foreground regions, and then the foreground connected region closest to the bottom of the image is retained to obtain the optimized segmentation results.

[0037] Step 4, as follows Figure 3 As shown, key frames of the video are selected: the diaphragm thickness change curve is calculated based on the predicted segmentation results obtained in step 3, and the paired B-type image and SWE image and their softness map corresponding to the highest and lowest values ​​of diaphragm thickness in each video are saved as key frames.

[0038] By setting a dual threshold, the region in the softness map whose diaphragm modulus value lies between the two thresholds is retained according to the 1% quantile, and regions with poor imaging quality are removed, so as to more accurately calculate the statistical characteristic parameters of the elastic modulus value and reduce errors.

[0039] Step 5: Extract and filter the radiomics quantitative parameters of the keyframes obtained in Step 4, specifically including:

[0040] Step 5.1 Extraction of radiomics quantitative parameters: First, using the softness map obtained in Step 4 after removing inaccurate regions, calculate the maximum, minimum, mean, and standard deviation of the elastic modulus values ​​of the remaining regions; then, extract high-throughput radiomics quantitative parameters using the original B-type image and its predicted segmentation results and the softness image paired with the B-type image.

[0041] Step 5.2 Screening of radiomics quantitative parameters: First, examine the radiomics quantitative parameters obtained in Step 5.1. If the parameters conform to a normal distribution, perform a T-test; otherwise, perform a Mann-Whitney U test to preliminarily screen for radiomics quantitative parameters with statistical differences. Then, perform LASSO regression on the preliminarily screened radiomics quantitative parameters and retain only those whose regression coefficients meet the threshold conditions to achieve parameter compression, i.e., feature dimensionality reduction.

[0042] Step 6: Construct and train the classification model: Construct a classifier based on support vector machine (SVM) as the classification model. Input the compressed radiomics quantitative parameters and other variable information obtained in step 5 into the classification model for training. Use five-fold cross-validation and grid search to tune the parameters to find the optimal model. Then, use the trained classification model for online phase testing.

[0043] Through specific practical experiments, the segmentation model was trained and tested. Specifically, there were 179 cases in the control group (466 videos in total) and 35 cases in the mechanical ventilation group (115 videos in total). Among them, 68 cases (54 cases in the control group and 14 cases in the mechanical ventilation group, a total of 234 videos) were used for training the segmentation model, and 8 cases (6 cases in the control group and 2 cases in the mechanical ventilation group, a total of 18 videos) were used for validating the segmentation model.

[0044] Images from both the training and validation sets were uniformly scaled to 256×256 before being input into the network for training. Data augmentation of the training set was performed using random horizontal flipping. During training, the batch size was set to 8, the number of epochs to 50, the optimizer was stochastic gradient descent, and the initial learning rate was set to 0.01, decreasing to 0.1 times the previous epoch's learning rate at epochs 15, 25, and 35, respectively. To avoid overfitting during training, the validation set was used to validate the model after each training epoch, and the model performing best on the validation set was saved as the final model. The optimal segmentation model achieved a Dice coefficient of 95.42% on the training set and 87.89% on the validation set. The segmentation performance of the optimal segmentation model on the validation set is shown below. Figure 5 As shown, the red line represents the label, and the yellow line represents the predicted segmentation result obtained by the segmentation network.

[0045] After feature extraction, a total of 4524 radiomics quantitative parameters were extracted from the two sets of paired B-type images, their soft maps, and predicted segmentation results for each video. In the automatic classification task between the control group and the mechanical ventilation group, 2596 parameters with statistical differences were identified. After LASSO regression, 11 parameters with significant contributions remained. In the automatic classification task of weaning results in the mechanical ventilation group, 803 parameters with statistical differences were identified. After LASSO regression, 24 parameters with significant contributions remained.

[0046] In this embodiment, the optimal model achieved the following automatic classification tasks for the control group and the mechanically ventilated group: area under the curve (AUC), accuracy, sensitivity, and specificity were 84.01%, 81.76%, 85.41%, and 66.96%, respectively. In the automatic classification task for the weaning results of the mechanically ventilated group, the optimal model achieved the following AUC, accuracy, sensitivity, and specificity: 94.37%, 89.57%, 80.36%, and 98.31%, respectively.

[0047] Compared with existing technologies, this method employs frame extraction during the training of the segmentation network, utilizing a small amount of data for learning, thus reducing the amount of annotation and training time; the predicted segmentation result in one modality can be mapped to another modality, eliminating the need for separate training and testing on each modality, making it more efficient; the post-processing of the predicted segmentation result utilizes the characteristics of the tissue in the image and incorporates morphological processing methods, making it more scientific; the method of selecting keyframes incorporates the dynamic information of the video, resulting in higher information utilization; and the classification model exhibits stable performance.

[0048] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A diaphragmatic ultrasound radiomics method, characterized in that, By constructing a U-Net segmentation model with an encoder-decoder structure offline and training it on a preprocessed dataset, the trained U-Net segmentation model is used to test B-mode SWE bimodal videos to obtain predicted segmentation results. The predicted segmentation results are then post-processed. Keyframes are selected from the input video by combining the post-processed predicted segmentation results with the dynamic information of the input video. High-throughput image omics quantitative parameters are extracted from the predicted segmentation results corresponding to the keyframes and filtered. The filtered image omics parameters are combined with other variable information to train a support vector machine (SVM)-based classification model. The trained classification model is then used for online testing. The extraction and filtering include: Step 5.1 Extraction of radiomics quantitative parameters: First, using the soft map after removing inaccurate regions, calculate the maximum, minimum, mean, and standard deviation of the elastic modulus values ​​of the remaining regions; then, extract high-throughput radiomics quantitative parameters using the original B-type image and its predicted segmentation results, as well as the soft map paired with the B-type image. Step 5.2 Screening of radiomics quantitative parameters: First, examine the radiomics quantitative parameters obtained in Step 5.

1. If the parameters conform to a normal distribution, perform a T-test; otherwise, perform a Mann-Whitney U test to preliminarily screen for radiomics quantitative parameters with statistical differences. Then, perform LASSO regression on the preliminarily screened radiomics quantitative parameters and retain only those whose regression coefficients meet the threshold conditions to achieve parameter compression, i.e., feature dimensionality reduction.

2. The diaphragmatic ultrasound imaging method according to claim 1, characterized in that, The U-Net segmentation model includes: a 5-layer encoder for extracting feature maps, a 4-layer decoder for restoring the size of the feature maps, 4 skip connection layers connecting the encoder and decoder, and a 1×1 convolutional layer placed after the 4th decoder layer; The U-Net segmentation model described above performs frame extraction on the video data input during training, but does not perform frame extraction on the video data input during online testing.

3. The diaphragmatic ultrasound imaging method according to claim 1, characterized in that, The preprocessing refers to: automatically detecting the boundaries of B-type images and SWE images using edge detection and Hough transform, cropping them using boundary coordinate points, and then filtering the images using the spontaneous breathing test (SBT) index.

4. The diaphragmatic ultrasound imaging method according to claim 1, characterized in that, The post-processing refers to: using the positional characteristics of the diaphragm in the B-type image and morphological processing methods to optimize the predicted segmentation results obtained from the test and reduce the erroneous parts in the predicted segmentation results. Specifically, the dilation and erosion method is first used to fill the small holes in the segmentation results and remove the small foreground regions, and then the foreground connected region closest to the bottom of the image is retained to obtain the optimized segmentation results.

5. The diaphragmatic ultrasound imaging method according to claim 1, characterized in that, The classification model incorporates a grid search method during parameter tuning. Specifically, within a given parameter range, the parameters are adjusted sequentially at specified step sizes. The adjusted parameters are then used to train the classifier. From all parameter combinations, the set of parameters that enables the classifier to achieve the highest accuracy is found to construct the optimal classification model.

6. A diaphragmatic ultrasound radiomics system for implementing any one of the diaphragmatic ultrasound radiomics methods according to claims 1-5, characterized in that, include: The system comprises a data preprocessing unit, a segmentation model training and validation unit, a predicted segmentation result post-processing and keyframe selection unit, a feature extraction and filtering unit, and a classification model cross-validation unit. Specifically: the data preprocessing unit acquires B-mode-SWE bimodal diaphragmatic ultrasound videos, performs video frame extraction, image cropping, image transformation, image annotation, and data partitioning to obtain a dataset; the segmentation model training and validation unit constructs a segmentation network for segmentation testing and trains the network using a training set; the predicted segmentation result post-processing and keyframe selection unit optimizes the predicted segmentation results output by the segmentation network and selects keyframes from the input video based on the optimized predicted segmentation results and dynamic information from the input video; the feature extraction and filtering unit extracts high-throughput radiomics quantitative parameters from the predicted segmentation results corresponding to the keyframes and filters out radiomics parameters that are helpful for classification; and the classification model cross-validation unit trains the classification network based on radiomics quantitative parameters and other variable information and uses it for online testing.

Images