Enhanced ultrasonic target segmentation method based on multi-view learning

Through the deep segmentation method of multi-view learning, the U-shaped network and evidence theory are used to fusion of multi-view prediction results, which solves the problem of failing to fully utilize the multi-view features in the dynamic process in the prior art, and significantly improves the enhanced ultrasound target segmentation Trustworthiness.

CN119067987APending Publication Date: 2024-12-03NANJING UNIV OF AERONAUTICS & ASTRONAUTICS +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411019556.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-07-26
Filing Date
2024-07-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing enhanced ultrasound data segmentation methods fail to fully utilize the multi-view features in dynamic processes, and traditional methods are difficult to explain the entire segmentation process, resulting in insufficient credibility of the segmentation results.

Method used

Using a deep segmentation method based on multi-view learning, a multi-view ultrasound image is obtained for preprocessing, and a deep segmentation model for multi-view learning is constructed. U-shaped network and evidence theory are used to fuse multi-view prediction results, and the model is trained and tested to improve segmentation The credibility of the results.

Benefits of technology

Make full use of multi-view features in ultrasound data and complete segmentation with multiple view information, which significantly improves the credibility of segmentation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067987A_ABST
    Figure CN119067987A_ABST
Patent Text Reader

Abstract

The invention discloses an enhanced ultrasonic target segmentation method based on multi-view learning, and the method comprises the steps: obtaining a multi-view ultrasonic image, and carrying out the preprocessing of the multi-view ultrasonic image, and obtaining a multi-view sample data set; constructing a deep segmentation model of multi-view learning: for each view, constructing a single-view segmentation model based on a U-shaped network to obtain a prediction result of the single view; fusing prediction results of multiple views based on an evidence theory; a trained multi-view learning depth segmentation model is obtained; and segmenting the ultrasonic image to be segmented through the multi-view learning depth segmentation model. Multi-view features in the ultrasonic data are utilized, segmentation is completed in combination with information of multiple views, and the credibility of a segmentation result is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to bioinformatics, and in particular to an enhanced ultrasound target segmentation method based on multi-view learning. Background Art

[0002] Ultrasound contrast imaging is based on conventional ultrasound examination. By injecting contrast agent intravenously, the blood flow distribution and perfusion of the lesion can be clearly displayed through blood circulation. At the same time, ultrasound contrast imaging can display in real time and dynamically, which is more conducive to differential diagnosis of lesions. The combination of ultrasound contrast imaging and grayscale ultrasound can improve the detection rate of lesions and distinguish between benign and malignant lesions.

[0003] At present, most of the segmentation methods for enhanced ultrasound data are based on key frames, ignoring the fact that ultrasound enhancement is a dynamic process and failing to fully utilize the multi-view features of ultrasound enhanced data. In addition, traditional segmentation methods cannot well explain the entire process. Summary of the invention

[0004] Purpose of the invention: In view of the above shortcomings, the present invention provides an enhanced ultrasonic target segmentation method based on multi-view learning to improve segmentation credibility.

[0005] Technical solution: To solve the above problems, the present invention adopts an enhanced ultrasonic target segmentation method based on multi-view learning, comprising the following steps:

[0006] (1) Acquire multi-view ultrasound images and perform preprocessing to obtain a multi-view sample data set;

[0007] (2) Construct a deep segmentation model for multi-view learning, including:

[0008] (2.1) For each view, a single view segmentation model is constructed based on the U-shaped network to obtain the prediction result of the single view;

[0009] (2.2) Fusion of prediction results from multiple views based on evidence theory;

[0010] (3) Train and test the multi-view learning deep segmentation model through the sample data set to obtain a trained multi-view learning deep segmentation model;

[0011] (4) The ultrasound image to be segmented is segmented through a deep segmentation model based on multi-view learning.

[0012] Furthermore, the multi-view ultrasound image includes a grayscale ultrasound image and an enhanced ultrasound image.

[0013] Furthermore, the pre-processing step includes:

[0014] (1.1) Extract key frame images from ultrasound images and extract images of different perfusion stages according to perfusion intensity;

[0015] (1.2) Correct the retained key frame images to eliminate lesion displacement caused by organ movement or probe jitter;

[0016] (1.3) Multi-view data pair construction: the key frame images of different views of the same lesion section are matched;

[0017] (1.4) Image normalization: Normalize the pixel intensity of the key frame image and unify the image size.

[0018] Furthermore, the prediction results of the fusion of multiple views based on the evidence theory in step (2.2) specifically include:

[0019] (2.2.1) Construct an evidence module and use the evidence network as an encoder to obtain evidence of prediction results of different views;

[0020] (2.2.2) Using the DS evidence theory fusion rule to perform multi-view fusion, the joint quality fusion process of the views is recorded as:

[0021]

[0022] in, represents the joint quality after multi-view fusion, represents the joint quality of the v-th view, represents the belief quality that the pixel at row i and column j in view v is predicted to be of category k, represents the uncertainty of the prediction result of the pixel in the i-th row and j-th column in the v-th view, and K represents the total number of prediction categories; b k,i,j represents the belief quality of the pixel in row i and column j predicted as category k after view fusion, u i,j Represents the uncertainty of the pixel prediction result in the i-th row and j-th column after view fusion;

[0023] (2.2.3) Obtain evidence after multi-view fusion:

[0024]

[0025] (2.2.4) Based on the fused evidence, the prediction result of multi-view fusion is obtained:

[0026] R pixel(i,j) =max{e 1,i,j ,…,e K,i,j}

[0027] R={R pixel(i,j)}

[0028] Among them, R pixel(i,j) represents the predicted category of the pixel pixel(i,j) in the i-th row and j-th column, and R represents the prediction result of multi-view fusion.

[0029] Furthermore, the joint quality after the multi-view fusion satisfies:

[0030]

[0031]

[0032] Where C represents the measure of the conflict between the grayscale ultrasound image and the enhanced ultrasound image, represents the belief quality that the pixel in the i-th row and j-th column in the grayscale ultrasound image is predicted to be of category s, represents the belief quality that the pixel in the i-th row and j-th column of the enhanced ultrasound image is predicted to be of category t, represents the uncertainty of the pixel prediction result of the i-th row and j-th column in the grayscale ultrasound image, Represents the uncertainty of the prediction result of the pixel in the i-th row and j-th column in the grayscale ultrasound image.

[0033] Furthermore, the calculation formulas for belief quality and uncertainty in step (2.2.2) are:

[0034]

[0035] in, represents the evidence that the pixel in row i and column j in view v is predicted to be of category k, represents the Dirichlet distribution intensity of the pixel at the i-th row and j-th column in the v-th view.

[0036] Furthermore, the Dirichlet distribution intensity calculation formula for each view is:

[0037]

[0038] in, represents the distribution parameter of the Dirichlet distribution of the pixel in the i-th row and j-th column in the v-th view; Indicates the priority weight of the pixel in the i-th row and j-th column in the v-th view predicted as the k-th category, satisfying When there is no classification priority,

[0039] Furthermore, the loss function of the deep segmentation model in step (3) is Dice loss:

[0040]

[0041] Among them, R represents the segmentation prediction map, and G represents the actual segmentation map.

[0042] The present invention also adopts a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0043] The present invention also adopts a computer-readable storage medium on which a computer program is stored, and the computer program implements the steps of the above method when executed by a processor.

[0044] Beneficial effects: Compared with the prior art, the significant advantage of the present invention is that it fully utilizes the multi-view features in ultrasound data, combines the information of multiple views to complete segmentation, and improves the credibility of the segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the process of the segmentation method of the present invention.

[0046] Figure 2 Schematic diagram of the processing process of single-view ultrasound images in the present invention.

[0047] Figure 3 Schematic diagram of the multi-view fusion process in the present invention. DETAILED DESCRIPTION

[0048] like Figure 1 As shown, an enhanced ultrasonic target segmentation method based on multi-view learning in this embodiment includes the following steps:

[0049] Step 1: Acquire multi-view ultrasound images and perform preprocessing to obtain a multi-view sample data set.

[0050] Step 1.1: The acquired ultrasound image data is saved in the format of jpg\bmp\tif, etc.;

[0051] Step 1.2: Extract image key frames, extract images of different perfusion stages according to the perfusion intensity, including peak period images, peak and subsidence stage images when the perfusion intensity is half of the peak intensity;

[0052] Step 1.3: Motion compensation: Correct the retained key frame images to eliminate the lesion displacement caused by organ movement or probe jitter;

[0053] Step 1.4: Multi-view data pair construction, taking two views as an example, including grayscale ultrasound images corresponding to the same lesion section (view Figure 1 ) and enhanced ultrasound images ( Figure 2 );

[0054] Step 1.5: Image normalization: normalize the pixel intensity of the keyframe image and unify the image size of each view by scaling.

[0055] Step 2: Build a deep segmentation model for multi-view learning, including:

[0056] Step 2.1 For each view, a single view segmentation model is constructed based on the U-type network to obtain the prediction result of the single view.

[0057] like Figure 2 As shown in the figure, a single view segmentation model is constructed based on the U-shaped network, and the uncertainty of the prediction result is estimated. For a sample X in the dataset, it contains V views, that is, X = {X 1 ,…,X V}, using the U-shaped network as the encoder f(·) to capture information from different views and obtain vector representations h of different views v =f(X v ).

[0058] Step 2.2: If Figure 3 As shown, the prediction results of multi-view fusion based on evidence theory.

[0059] Step 2.2.1: Construct the evidence module. It is required to use an activation function with non-negative output and use the evidence network as the encoder g(·) to obtain evidence E under different views. v =g(h v ),in Indicates the evidence that the pixel in row i and column j is predicted to be the kth category;

[0060] Step 2.2.2: Calculate the distribution parameter α of the Dirichlet distribution under each view of the sample:

[0061]

[0062] in, Indicates the priority weight of the pixel in the i-th row and j-th column in the v-th view predicted as the k-th category, satisfying When there is no classification priority, The above formula is simplified to:

[0063]

[0064] Step 2.2.3: Calculate the distribution strength S of the Dirichlet distribution under each view of the sample:

[0065]

[0066] in, represents the Dirichlet distribution intensity of the pixel at the i-th row and j-th column in the v-th view.

[0067] Step 2.2.4: Calculate the joint quality of each view of the sample The joint quality consists of the belief quality b and the uncertainty u:

[0068]

[0069] in, represents the belief quality that the pixel at row i and column j in view v is predicted to be of category k, represents the uncertainty of the prediction result of the pixel in the i-th row and j-th column in the v-th view, represents the joint quality of the vth view and satisfies

[0070] Step 2.2.5: Use Dempster's (DS evidence theory) fusion rule for multi-view fusion: (taking two views as an example), given two views, The view fusion process is recorded as:

[0071]

[0072] in The belief quality and uncertainty satisfy:

[0073]

[0074]

[0075] in, The conflict between the two views is measured. Similarly, the fusion of the views Also satisfied Expanding to more views, the fusion process is

[0076] Step 2.2.6: Obtain evidence after multi-view fusion:

[0077]

[0078] Step 2.2.7: Calculate the segmentation prediction map of the fused multi-view, and find the subscript with the maximum belief quality at each pixel on the fused evidence:

[0079] R pixel(i,j) =max{e 1,i,j ,…,e K,i,j}

[0080] Among them, R pixel(i,j)represents the predicted category of the pixel (i, j) at the i-th row and j-th column, R = {R pixel(i,j)} represents the prediction map of multi-view fusion segmentation.

[0081] Step 3: Train and test the deep segmentation model for multi-view learning to obtain a trained deep segmentation model for multi-view learning.

[0082] Step 3.1: Import multi-view data. For a sample X in the dataset, it contains V views, that is, X = {X 1 ,…,X V}, in this embodiment, two views are used as columns: grayscale ultrasound image or enhanced ultrasound image;

[0083] Step 3.2: Divide the data set into a training set and a test set according to a given ratio, with 20% of the training set data reserved as a validation set;

[0084] Step 3.3: Set the parameters of the deep segmentation model for multi-view learning, use the training set data as input, use Dice as the loss function, train the deep segmentation model, and use the validation set to determine the best model in the training phase; the loss function is:

[0085]

[0086] Among them, R represents the segmentation prediction map, and G represents the actual segmentation map.

[0087] Step 3.4: Input the test set into the multi-view learning deep segmentation model for training to obtain a trained multi-view learning deep segmentation model;

[0088] Step 3.5: Make an uncertainty distribution map or a certainty distribution map. To save the uncertainty distribution map, you need to convert {u i,j}Save it in jpg\bmp\tif and other formats. Save it as a deterministic distribution map and change {1-u i,j}Save in jpg\bmp\tif and other formats.

[0089] Step 4: Segment the ultrasound image to be segmented through the deep segmentation model learned by multi-view.

[0090] Step 5: Post-processing of the segmentation results of the region of interest, including:

[0091] Step 5.1: Select a region of interest (ROI) in the segmentation prediction map based on the doctor's instructions or the ultrasound imaging report content;

[0092] Step 5.2: Perform noise processing on the segmentation results within the ROI, and use Gaussian filtering to weight the neighborhood around the pixel to smooth the image and reduce noise;

[0093] Step 5.3: Use the conditional random field combined with the original image color and the segmentation map predicted by the depth segmentation model to model the segmentation results of the ROI part, use the energy function to represent the quality of the segmentation effect, and obtain better segmentation results by minimizing the energy function;

[0094] Step 5.4: Visualize the prediction results. Create a color label map based on the segmentation label value, map different labels to different colors, and overlay the predicted segmentation map with the original image for visualization.

Claims

1. An enhanced ultrasonic target segmentation method based on multi-view learning, characterized in that: The following steps are involved: (1) Acquire multi-view ultrasound images and perform preprocessing to obtain a multi-view sample data set; (2) Construct a deep segmentation model for multi-view learning, including: (2.1) For each view, a single view segmentation model is constructed based on the U-shaped network to obtain the prediction result of the single view; (2.2) Fusion of prediction results from multiple views based on evidence theory; (3) Train and test the multi-view learning deep segmentation model through the sample data set to obtain a trained multi-view learning deep segmentation model; (4) The ultrasound image to be segmented is segmented through a deep segmentation model based on multi-view learning.

2. The enhanced ultrasonic target segmentation method according to claim 1, characterized in that: The multi-view ultrasound image includes a grayscale ultrasound image and an enhanced ultrasound image.

3. The enhanced ultrasonic target segmentation method according to claim 2, characterized in that: The pre-processing step comprises: (1.1) Extract key frame images from ultrasound images and extract images of different perfusion stages according to perfusion intensity; (1.2) Correct the retained key frame images to eliminate lesion displacement caused by organ movement or probe jitter; (1.3) Multi-view data pair construction: the key frame images of different views of the same lesion section are matched; (1.4) Image normalization: Normalize the pixel intensity of the key frame image and unify the image size.

4. The enhanced ultrasonic target segmentation method according to claim 2, characterized in that: The prediction results of the fusion of multiple views based on the evidence theory in step (2.2) specifically include: (2.2.1) Construct an evidence module and use the evidence network as an encoder to obtain evidence of prediction results of different views; (2.2.2) Using the DS evidence theory fusion rule to perform multi-view fusion, the joint quality fusion process of the views is recorded as: in, represents the joint quality after multi-view fusion, represents the joint quality of the v-th view, represents the belief quality that the pixel at row i and column j in view v is predicted to be of category k, represents the uncertainty of the prediction result of the pixel in the i-th row and j-th column in the v-th view, and K represents the total number of prediction categories; b k,i,j represents the belief quality of the pixel in row i and column j predicted as category k after view fusion, u i,j represents the uncertainty of the pixel prediction result of the i-th row and j-th column after view fusion, and V represents the total number of view types; (2.2.3) Obtain evidence after multi-view fusion: (2.2.4) Based on the fused evidence, the prediction result of multi-view fusion is obtained: R pixel(i,j) =max{and 1,i,j ,…,and K,i,j } R={R pixel(i,j) } Among them, R pixel(i,j) represents the predicted category of the pixel pixel(i,j) in the i-th row and j-th column, and R represents the prediction result of multi-view fusion.

5. The enhanced ultrasonic target segmentation method according to claim 4, characterized in that: The joint quality after the multi-view fusion satisfies: Where C represents the measure of the conflict between the grayscale ultrasound image and the enhanced ultrasound image, represents the belief quality that the pixel in the i-th row and j-th column in the grayscale ultrasound image is predicted to be of category s, represents the belief quality that the pixel in the i-th row and j-th column of the enhanced ultrasound image is predicted to be of category t, represents the uncertainty of the pixel prediction result of the i-th row and j-th column in the grayscale ultrasound image, Represents the uncertainty of the prediction result of the pixel in the i-th row and j-th column in the grayscale ultrasound image.

6. The enhanced ultrasonic target segmentation method according to claim 4, characterized in that: The calculation formulas for belief quality and uncertainty in step (2.2.2) are: in, represents the evidence that the pixel in row i and column j in view v is predicted to be of category k, represents the Dirichlet distribution intensity of the pixel at the i-th row and j-th column in the v-th view.

7. The enhanced ultrasonic target segmentation method according to claim 6, characterized in that: The Dirichlet distribution intensity calculation formula for each view is: in, represents the distribution parameter of the Dirichlet distribution of the pixel in the i-th row and j-th column in the v-th view; Indicates the priority weight of the pixel in the i-th row and j-th column in the v-th view predicted as the k-th category, satisfying When there is no classification priority, 8. The enhanced ultrasonic target segmentation method according to claim 7, characterized in that: The loss function of the deep segmentation model in step (3) is Dice loss: Among them, R represents the segmentation prediction map, and G represents the actual segmentation map.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Multi-source uncertainty-driven medical image classification method

    CN117994564A