Ultrasound contrast classification method with time-intensity curve feature machine learning

Abstract

The application discloses a kind of time-intensity curve feature machine learning's ultrasonic contrast classification method.The method is based on machine learning analysis focus area, non-focus area, focus area / non-focus area and non-focus area-focus area time-intensity curve feature combination based on deep learning analysis video feature, by multi-factor logistic regression combination model is constructed to carry out ultrasonic contrast classification.Make video data and cut its region of interest to obtain brightness value and draw time-intensity curve extraction area under curve, peak time, peak intensity, contrast agent average transit time, rising slope, falling slope, average brightness value, average brightness standard deviation feature is respectively input deep learning model and machine learning model, by multi-factor logistic regression combination model is constructed.Based on classification index comprehensive evaluation the classification performance of the method to ultrasonic contrast result.

Description

Technical Field

[0001] This invention pertains to video processing technology in the field of computer vision, and specifically relates to a method for processing medical ultrasound contrast imaging videos. More specifically, it is a method for classifying ultrasound contrast imaging by using machine learning to construct a combined model of ultrasound contrast imaging videos and time-intensity curve features obtained therefrom. Background Technology

[0002] Contrast-Enhanced Ultrasound (CEUS) is an advanced medical imaging technique that provides information on tissue blood flow and perfusion within a lesion area. The Time Intensity Time (TIC) curve represents the change in intensity of the ultrasound contrast agent over time within the region of interest, enabling quantitative analysis of lesions and objectively reflecting the characteristic parameters of the region of interest.

[0003] Traditional methods may be limited to the analysis of specific parameters, making it difficult to achieve a comprehensive evaluation of ultrasound contrast imaging results, especially at the level of multi-parameter integrated analysis.

[0004] By analyzing the TICs (Transmission Intensities) of lesion areas, non-lesion areas, lesion / non-lesion areas, and non-lesion-lesion areas, and combining this with OpenCV library for video processing, key features were extracted. The area under the curve reflects the overall contrast agent distribution; the time to peak indicates the lesion's response speed to the contrast agent; the peak intensity reflects the contrast agent concentration in the lesion area; the average transit time provides the average time for the contrast agent to pass through the lesion area; the rising slope provides the rate at which the intensity of the ultrasound contrast agent increases over time within the lesion; the falling slope indicates the rate at which the intensity of the ultrasound contrast agent decreases over time within the lesion; and the average brightness value and average brightness standard deviation reflect the overall condition of the lesion area. The comprehensive application of these features can fully evaluate the classification performance of ultrasound contrast imaging.

[0005] These features are used as a time-intensity curve sample dataset and fed into a machine learning model. The original ultrasound contrast images are fed into a deep learning model, and the model is trained using k-fold cross-validation to obtain a pre-trained model. A multivariate logistic regression model is used to construct a combined model of the machine learning model and the deep learning model. The test set is evaluated on the combined model, and the classification performance of the ultrasound contrast results based on the time-intensity curve feature combined model is evaluated using metrics such as accuracy, recall, precision, F1 score, and area under the receiver operating system (AOS). Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and to propose an ultrasound contrast imaging classification method based on time-intensity curve feature machine learning, which can be used for the classification and evaluation of lesions in ultrasound contrast imaging.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A time-intensity curve feature-based ultrasound contrast classification method, characterized by the following steps:

[0009] (1) Obtain M cases of ultrasound contrast imaging video V i Manually select the V i The rectangular region of interest (ROI) is defined, and a j×j rectangular region is cropped from the center of the ROI as the lesion area C. i Video data, cropping a j×j rectangular region at each of the four corners of the ROI as the non-lesion area P i The video data is 3≤j≤20, where i represents the i-th example.

[0010] (2) Use the OpenCV library to process the lesion area C. i P in non-lesion areas i For each frame of the video, obtain its luminance value matrix [B]. ci B p1i B p2i B p3i B p4i The final average brightness value of each frame in the lesion area is B. ci The average brightness value per frame in the non-lesion area is B. pi , Lesion area / non-lesion area C i / P i Average brightness value per frame B cpi B cpi =B ci ÷B pi Non-lesion area - lesion area P i -C i Average brightness value per frame B pci B pci =B pi -B ci The average grayscale value is used as a representative of brightness.

[0011] (3) Based on the lesion area C i Non-lesion area P i Lesion area / non-lesion area C i / P i Non-lesion area - Lesion area P i -C i Brightness value B i Plot the corresponding time-intensity curves (TICs), which are curves showing the change in intensity of the ultrasound contrast agent over time within the region of interest.o Using a filter, the curve is smoothed to obtain the filtered TIC. f .

[0012] (4) For the TIC f Feature extraction is performed, and the area under the curve (Area) is obtained through numerical integration. i The time to peak (TTP) is obtained by detecting the occurrence time of the peak value. i The peak intensity PI is obtained by extracting the highest gray value on the curve. i The mean transit time (MTT) of the contrast agent was obtained by measuring the time from the start of enhancement to the reduction of enhancement intensity to half of its peak value. i By selecting the starting and ending points of the rising and falling phases, the rising slope and falling slope AS are obtained by comparing the peak intensity difference between the two points with the time difference. i and DA i The average brightness value (MB) is obtained by averaging the brightness values ​​of each frame. i The mean luminance standard deviation (MBSD) is obtained by taking the average standard deviation of the luminance values ​​in each frame. i .

[0013] (5) The ultrasound contrast images of the M cases are transmitted through the TIC f Obtain the lesion area C i Non-lesion area P i Lesion area / non-lesion area C i / P i Non-lesion area - lesion area P i -C i The feature values ​​are used as the TIC sample dataset.

[0014] (6) Divide the M ultrasound contrast video sample datasets into training set and test set in a ratio of 8:2.

[0015] (7) Transfer the training set to TIC f The extracted features and the original M ultrasound contrast videos were trained using k-fold cross-validation on a machine learning network model and a deep learning convolutional neural network model, respectively (3 ≤ k ≤ 10). The experiment was conducted for n rounds, and the average of the results from the n rounds was selected as the pre-trained model. The classification result probability P was... TIC and P Video The value is between 0 and 1.

[0016] (8) Using a multi-factor logistic regression model, the classification probabilities of the machine learning network model and the classification probabilities of the deep learning convolutional neural network are combined to form a feature matrix. Each row represents a sample, and each column corresponds to the output of a model. A combined model is established to obtain the trained model. The output probability value of the combined model is P. Combined P Combined =σ(w1·P TIC +w2·P Video +b), where σ is the logistic sigmoid function, w1 and w2 are the weight parameters of the model, and b is the bias term.

[0017] The ultrasound contrast imaging classification method based on time-intensity curve feature machine learning proposed in this invention has the following beneficial effects:

[0018] 1. This invention proposes a method for ultrasound contrast imaging classification that utilizes machine learning combined with key features of time-intensity curves, including area under the curve, time to peak, peak intensity, average passage time of contrast agent, rising slope, falling slope, average brightness value, and average brightness standard deviation. This method innovatively mines the features of time-intensity curves of lesion areas, non-lesion areas, lesion / non-lesion areas, and non-lesion area-lesion area, enabling accurate ultrasound contrast imaging classification results.

[0019] 2. This invention proposes a combined model that uses machine learning to process key features of time-intensity curves and deep learning to process features of ultrasound contrast imaging videos to classify ultrasound contrast imaging. Attached Figure Description

[0020] Figure 1 : Flowchart of the present invention;

[0021] Figure 2 : Obtain ultrasound contrast imaging process images of lesion area and non-lesion area. Point C is the cropped lesion area, and P1, P2, P3, and P4 are the cropped non-lesion areas;

[0022] Figure 3 Time-intensity curves before and after filtering: the dashed line is the time-intensity curve before filtering, and the solid line is the time-intensity curve after filtering.

[0023] Figure 4 : Time-intensity curve characteristic value diagram, A is the area under the curve, T0 is the time to peak, T1 is the average transit time of the contrast agent, Imax is the peak intensity, L1 is the rising slope line, L2 is the falling slope line. Detailed Implementation

[0024] The ultrasound contrast imaging classification method based on time-intensity curve feature machine learning of the present invention is a combined model obtained by inputting the features of the lesion area, non-lesion area, lesion / non-lesion area and non-lesion area-lesion area obtained from the time-intensity curve of the region of interest in the ultrasound contrast imaging video into the machine learning classification network model and the original ultrasound contrast imaging video into the deep learning classification network model, and finally obtaining the classification of ultrasound contrast imaging. Figure 1 The flowchart of the method of the present invention mainly includes the following steps:

[0025] (1) Obtain M cases of ultrasound contrast imaging video V i Manually select the V i The rectangular region of interest (ROI) is defined, and a j×j rectangular region is cropped from the center of the ROI as the lesion area C. i Video data, cropping a j×j rectangular region at each of the four corners of the ROI as the non-lesion area P i Video data, 3≤j≤20, where i represents the i-th case. In this embodiment, M=259, j=5. The process of acquiring contrast-enhanced ultrasound images of the lesion area and non-lesion area is as follows: Figure 2 As shown.

[0026] (2) Use the OpenCV library to process the lesion area C. i P in non-lesion areas i For each frame of the video, obtain its luminance value matrix [B]. ci B p1i B p2i B p3i B p4i The final average brightness value of each frame in the lesion area is B. ci The average brightness value per frame in the non-lesion area is B. pi , Lesion area / non-lesion area C i / P i Average brightness value per frame B cpi B cpi =B ci ÷B pi Non-lesion area - lesion area P i -C i Average brightness value per frame B pci B pci =B pi -B ci The average grayscale value is used as a representative of brightness.

[0027] (3) Based on the lesion area C i Non-lesion area P i Lesion area / non-lesion area C i / P i Non-lesion area - Lesion area Pi -C i Brightness value B i Plot the corresponding time-intensity curves (TICs), which are curves showing the change in intensity of the ultrasound contrast agent over time within the region of interest. o Using a filter, the curve is smoothed to obtain the filtered TIC. f The process of plotting the corresponding time-intensity curve is as follows: Figure 3 As shown. In this embodiment, the smoothing filtering algorithm used is the Savitzky-Golay filtering algorithm (Schafer R W. What is a Savitzky-Golay filter? [J]. IEEE Signal Processing Magazine, 2011, 28(4): 111-117), which is an algorithm that performs n-order polynomial fitting on data points within a certain length window to obtain the fitted result. In this embodiment, the filter window width is 199, and the polynomial order of the fitted samples is set to 2. The time-intensity curves before and after filtering optimization are shown in the figure. Figure 3 As shown.

[0028] (4) For the TIC f Feature extraction is performed, and the area under the curve (Area) is obtained through numerical integration. i The time to peak (TTP) is obtained by detecting the occurrence time of the peak value. i The peak intensity PI is obtained by extracting the highest gray value on the curve. i The mean transit time (MTT) of the contrast agent was obtained by measuring the time from the start of enhancement to the reduction of enhancement intensity to half of its peak value. i By selecting the starting and ending points of the rising and falling phases, the rising slope and falling slope AS are obtained by comparing the peak intensity difference between the two points with the time difference. i and DA i The average brightness value (MB) is obtained by averaging the brightness values ​​of each frame. i The mean luminance standard deviation (MBSD) is obtained by taking the average standard deviation of the luminance values ​​in each frame. i The process of obtaining the characteristic values ​​of the time-intensity curve is as follows: Figure 4 As shown.

[0029] (5) The ultrasound contrast images of the M cases are transmitted through the TIC f Obtain the lesion area C i Non-lesion area P i Lesion area / non-lesion area C i / P i Non-lesion area - lesion area P i -Ci The feature values ​​are used as the sample dataset. In this embodiment, M = 259.

[0030] (6) Divide the M ultrasound contrast video sample datasets into training set and test set in a ratio of 8:2.

[0031] (7) Transfer the training set to TIC f The extracted features and the original M ultrasound contrast videos were trained using k-fold cross-validation on a machine learning network model and a deep learning convolutional neural network model, respectively (3 ≤ k ≤ 10). The experiment was conducted for n rounds, and the average of the results from the n rounds was selected as the pre-trained model. The classification result probability P was... TIC and P Video The value is between 0 and 1. In this example, the machine learning models we use are: Support Vector Machine (Hearst MA, Dumais ST, Osuna E, et al. Support vector machines[J].IEEE Intelligent Systems and Their Applications,1998,13(4):18-28), K-Nearest Neighbor (Peterson L EK-nearest neighbor[J].Scholarpedia,2009,4(2):1883), Random Forest (Breiman L. Random forests[J].Machinelearning,2001,45:5-32.), Decision Tree (Quinlan LR, Rivest R L. Inferring decision trees using the minimum description length principle[J].Information and Computation,1987,80(3):227-248). There are 32 TIC features in total. A total of 50 rounds are performed. The best machine learning model result is selected, and the average value of its cross-validation results is taken as P. TICThe deep learning model used was an R(2+1)D convolutional neural network (Tran D, Wang H, Torresan L, et al. A closer look at spatiotemporal convolutions for action recognition. [C] / / Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, 6450-6459.), with an input format of (BatchSize, Channels, Frames, Width, Height), specifically (8, 3, 64, 112, 112). Adaptive moment estimation was used, which could adaptively adjust the learning rate during the learning process. Cross-entropy loss function was used for 50 rounds of cross-validation training, and the average value of the cross-validation results was taken as P. Video .

[0032] (8) Using a multi-factor logistic regression model, the classification probabilities of the machine learning network model and the classification probabilities of the deep learning convolutional neural network are combined to form a feature matrix. Each row represents a sample, and each column corresponds to the output of a model. A combined model is established to obtain the trained model. The output probability value of the combined model is P. Combined P Combined =σ(w1·P TIC +w2·P Video +b), where σ is the logistic sigmoid function, w1 and w2 are the model's weight parameters, and b is the bias term. In this embodiment, the test set is tested on the combined model, and the experimental results are evaluated for classification performance using accuracy, recall, precision, F1 score, and area under the receiver curve. Accuracy represents the overall accuracy of prediction, recall measures the model's ability to capture positive samples, precision represents the model's accuracy in predicting positives, the F1 score combines recall and precision, and the area under the receiver curve provides a measure of overall classification performance.

Claims

1. A time-intensity curve feature-based ultrasound contrast classification method, characterized in that, Includes the following steps: Step 1: Obtain ultrasound contrast imaging videos of M cases. i Manually select the V i The rectangular region of interest (ROI) is defined, and a j×j rectangular region is cropped from the center of the ROI as the lesion area C. i Video data, cropping a j×j rectangular region at each of the four corners of the ROI as the non-lesion area P i Video data, 3≤j≤20, where i represents the i-th example; Step 2: Use the OpenCV library to process the lesion area C. i P in non-lesion areas i For each frame of the video, obtain its luminance value matrix [B]. ci B p1i B p2i B p3i B p4i The final average brightness value of each frame in the lesion area was B. ci The average brightness value per frame in the non-lesion area is B. pi , Lesion area / non-lesion area C i / P i Average brightness value per frame B cpi B cpi =B ci ÷B pi Non-lesion area - lesion area P i -C i Average brightness value per frame B pci B pci =B pi -B ci The average grayscale value is used as a representative of brightness. Step 3, based on the lesion area C i Non-lesion area P i Lesion area / non-lesion area C i / P i Non-lesion area - Lesion area P i -C i Brightness value B i Plot the corresponding time-intensity curves (TICs), which are curves showing the change in intensity of the ultrasound contrast agent over time within the region of interest. o Using a filter, the curve is smoothed to obtain the filtered TIC. f ; Step 4, for the TIC f Feature extraction is performed, and the area under the curve (Area) is obtained through numerical integration. i The time to peak (TTP) is obtained by detecting the occurrence time of the peak value. i The peak intensity PI is obtained by extracting the highest gray value on the curve. i The mean transit time (MTT) of the contrast agent was obtained by measuring the time from the start of enhancement to the reduction of enhancement intensity to half of its peak value. i By selecting the starting and ending points of the rising and falling phases, the rising slope and falling slope AS are obtained by comparing the peak intensity difference between the two points with the time difference. i and DA i The average brightness value (MB) is obtained by averaging the brightness values ​​of each frame. i The mean luminance standard deviation (MBSD) is obtained by taking the average standard deviation of the luminance values ​​in each frame of the image. i ; Step 5: Transmit the M ultrasound contrast images through the TIC. f Obtain the lesion area C i Non-lesion area P i Lesion area / non-lesion area C i / P i Non-lesion area - lesion area P i -C i The feature values ​​are used as the TIC sample dataset; Step 6: Divide the M ultrasound contrast video sample datasets into training set and test set in a ratio of 8:2; Step 7: Transfer the training set via TIC f The extracted features and the original M ultrasound contrast videos were trained using k-fold cross-validation on a machine learning network model and a deep learning convolutional neural network model, respectively (3 ≤ k ≤ 10). The experiment was conducted for n rounds, and the average of the results from the n rounds was selected as the pre-trained model. The classification result probability P was... TIC and P Video The value is between 0 and 1; Step 8: Using a multi-factor logistic regression model, combine the classification probabilities of the machine learning network model and the deep learning convolutional neural network to form a feature matrix. Each row represents a sample, and each column corresponds to the output of a model. Establish a combined model to obtain the trained model. The output probability value of the combined model is P. Combined P Combined =σ(w1·P TIC +w2·P Video +b), where σ is the logistic sigmoid function, w1 and w2 are the weight parameters of the model, and b is the bias term.

Images