A Coronary Artery Intima Boundary Segmentation Method Based on the Consistency of Adjacent Frames

Abstract

The present invention discloses a method for segmenting the coronary artery intima boundary based on adjacent frame consistency, comprising: acquiring an OCT image to be measured and inputting it into a deep learning network; extracting image features based on a multi-scale dense connection dilated convolution module and establishing a correspondence between the optical coherence tomography image and the coronary artery intima; calculating the consistency of adjacent frames in the optical coherence tomography image based on an adjacent frame information supplement module and obtaining an adjacent frame supplement relationship; and outputting a boundary result of the intima boundary segmentation according to the correspondence between the optical coherence tomography image and the coronary artery intima and the adjacent frame supplement relationship. By using the present invention, the coronary artery intima boundary can be segmented quickly and accurately. As a method for segmenting the coronary artery intima boundary based on adjacent frame consistency, the present invention can be widely applied to the field of medical image processing.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a method for segmenting the coronary artery intima boundary based on adjacent frame consistency. Background Art

[0002] The recognition of the intima of the coronary artery is the first step in clinically quantifying coronary artery cross-sectional data. The quantified vascular indexes are important reference indexes for guiding clinical diagnosis, and are the first step in identifying vulnerable plaques, evaluating the effect of interventional treatment, and the intima coverage after drug-eluting stent implantation. Therefore, accurately identifying the intima of the coronary artery is very important for the diagnosis of patients with coronary artery diseases. Clinically, the optical coherence tomography (OCT) technology can clearly show the cross-section and internal structure of the coronary artery, so OCT images are widely used clinically to detect the coronary artery intima. However, it is very time-consuming and inefficient for doctors to manually segment the intima and adventitia of the coronary artery in OCT images. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a method for segmenting the coronary artery intima boundary based on adjacent frame consistency, which can quickly and accurately segment the coronary artery intima boundary.

[0004] The first technical solution adopted by the present invention is: a method for segmenting the coronary artery intima boundary based on adjacent frame consistency, including the following steps:

[0005] Obtain the OCT image to be measured and input it into the deep learning network;

[0006] The deep learning network includes a multi-scale densely connected dilated convolution module and an adjacent frame information supplement module;

[0007] Based on the multi-scale densely connected dilated convolution module, extract image features and establish the corresponding relationship between the optical coherence tomography image and the coronary artery intima;

[0008] Based on the adjacent frame information supplement module, calculate the consistency of adjacent frames in the optical coherence tomography image and obtain the adjacent frame supplement relationship;

[0009] According to the corresponding relationship between the optical coherence tomography image and the coronary artery intima and the adjacent frame supplement relationship, output the intima boundary segmentation boundary result.

[0010] Further, the step of calculating the consistency of adjacent frames in the optical coherence tomography image and obtaining the adjacent frame supplement relationship based on the adjacent frame information supplement module specifically includes:

[0011] Calculate the cosine similarity vector diagram between adjacent frame features based on the adjacent frame information supplement module;

[0012] Judge the consistency of adjacent frames in the corresponding optical coherence tomography image according to the cosine similarity vector diagram between adjacent frame features;

[0013] Obtain the adjacent frame supplement relationship.

[0014] Furthermore, the formula of the adjacent frame information supplement module is expressed as follows:

[0015] c a,b (i,j;f a ,f b ) = max σ(f a (i,j),f b (i,j))

[0016] In the above formula, σ represents the cosine similarity vector diagram between two adjacent frame features, and c a,b (i,j;f a ,f b ) represents the maximum consistency of the pixel points f a (i,j) and f b (i,j).

[0017] Furthermore, before the step of obtaining the OCT image to be measured and inputting it into the deep learning network, it further includes:

[0018] Construct a training data set and train the deep learning network based on the training data set to obtain a trained deep learning network.

[0019] Furthermore, the step of constructing a training data set and training the deep learning network based on the training data set to obtain a trained deep learning network specifically includes:

[0020] Collect OCT images with different health conditions and the corresponding coronary artery intima segmentation results as the training data set;

[0021] Based on the training data set, use OCT images with different health conditions as the input and the corresponding coronary artery intima segmentation results as the output to train the deep learning network;

[0022] Adjust the network parameters of the deep learning network until the error rate reaches the preset range to obtain a trained deep learning network.

[0023] Furthermore, the network parameters of the deep learning network include the number of convolutional layers, the number of dilated convolutional layers, the number of BN layers, the number of RELU layers, the number of pooling layers, the number of upsampling layers, the number of output layers, the initial weight value, and the bias value.

[0024] The beneficial effects of the method of the present invention are as follows: By utilizing the self-learning ability of the deep learning network, the present invention establishes the correspondence between OCT images and the intimal segmentation results of coronary arteries, and determines the intimal segmentation results of the current coronary artery corresponding to the current image features, improving the efficiency of the intimal segmentation process of coronary arteries, making the segmentation results more accurate, realizing the supplementation of adjacent frame features, enriching the features of the current frame and having strong scalability. Description of the Drawings

[0025] Figure 1 is a method for segmenting the intimal boundary of coronary arteries based on adjacent frame consistency according to the present invention;

[0026] Figure 2 is a schematic structural diagram of the deep learning network in a specific embodiment of the present invention;

[0027] Figure 3 is a schematic structural diagram of the dilated convolution module with multi-scale dense connections in a specific embodiment of the present invention;

[0028] Figure 4 is a schematic structural diagram of the adjacent frame information supplementation module in a specific embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of the segmentation result in a specific embodiment of the present invention. Detailed Embodiments

[0030] The following further elaborates on the present invention in detail with reference to the drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of explanation and do not impose any limitation on the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0031] As Figure 1 shown, the present invention provides a method for segmenting the intimal boundary of coronary arteries based on adjacent frame consistency, and the method includes the following steps:

[0032] S0. Construct a training data set and train a deep learning network based on the training data set to obtain a trained deep learning network.

[0033] S0.1. Collect OCT images with different health conditions and the corresponding intimal segmentation results of coronary arteries as the training data set;

[0034] S0.2. Based on the training data set, use OCT images with different health conditions as the input and the corresponding intimal segmentation results of coronary arteries as the output to train the deep learning network;

[0035] S0.3. Adjust the network parameters of the deep learning network until the error rate reaches the preset range to obtain a trained deep learning network.

[0036] The network parameters of the deep learning network include the number of convolution layers, the number of dilated convolution layers, the number of BN layers, the number of RELU layers, the number of pooling layers, the number of upsampling layers, the number of output layers, the initial weights and the bias values.

[0037] Specifically, the OCT image sequences and coronary intima and adventitia segmentation results of several volunteers were selected as sample data, and the neural network was learned and trained. By adjusting the network structure and the weights between network nodes, the neural network was made to fit the relationship between the OCT images and the coronary intima segmentation results. Finally, the neural network was able to accurately fit the correspondence between the OCT image sequences and coronary intima segmentation results of different patients.

[0038] S1, obtain the OCT image to be tested and input it into the deep learning network;

[0039] S2, the deep learning network includes a multi-scale densely linked hole convolution module and an adjacent frame information supplement module;

[0040] S3, a multi-scale densely linked dilated convolution module to extract image features and establish the correspondence between optical coherence tomography images and the coronary artery intima;

[0041] Specifically, the features include high-level abstract coronary intima features and high-level abstract coronary intima features.

[0042] The S110 multi-scale densely linked dilated convolution module is used to establish the correspondence between OCT images and the coronary artery endothelium. The subnetwork structure is referenced Figure 3 As shown. 1 ×n 1 (n 2 )@n 3 Indicates that the step length is n 2 , the convolution kernel is n 3 n 1 ×n 1 Convolution operation; Dn 1 ×n 1 (n 2 rn 4 )@n 3 Indicates that the step length is n 2 , the convolution kernel is n 3 , the void factor is n 4 n 1 ×n 1 The dilated convolution operation

[0043] Among them, n 1 , n 2 , n 3 , and n 4Indicates the number at the corresponding position in the figure.

[0044] S4. Calculate the consistency of adjacent frames in the optical coherence tomography image based on the adjacent frame information supplement module, and obtain the adjacent frame supplement relationship;

[0045] S4.1. Calculate the cosine similarity vector map between adjacent frame features based on the adjacent frame information supplement module;

[0046] S4.2. Judge the consistency of adjacent frames in the corresponding optical coherence tomography image according to the cosine similarity vector map between adjacent frame features;

[0047] S4.3. Obtain the adjacent frame supplement relationship.

[0048] Specifically, the self-network structure diagram of the adjacent frame information supplement module refers to Figure 4 , and the formula is expressed as follows

[0049] c a,b (i,j;f a ,f b ) = max σ(f a (i,j),f b (i,j))

[0050] In the above formula, σ represents the cosine similarity vector map between two adjacent frame features, and c a,b (i,j;f a ,f b ) represents the maximum consistency of the pixel points f a (i,j) and f b (i,j).

[0051] c * a,b (i,j;f a ,f b ,y a ,y b ) = max σ(f a (i,j),f b (i,j))

[0052] c * a,b (i,j;f a ,f b ) represents the maximum consistency of adjacent pixel points supervised by the segmentation results y a , y b .

[0053] S5. Output the inner membrane boundary segmentation boundary result according to the corresponding relationship between the optical coherence tomography image and the coronary artery intima and the adjacent frame supplement relationship.

[0054] To quantify the perceptual consistency of the two-frame segmentation decision, calculate c a,b (i,j; f a , f b ) and c * a,b (i,j; f a , f b ) ratio, and fuse it onto adjacent frames.

[0055]

[0056] ρ(·) represents the similarity between each frame and its adjacent frame, H is the height of the image frame, and W represents the width of the image frame. represents the mean of the consistency of different frames.

[0057] A method for segmenting the coronary artery intima boundary based on adjacent frame consistency, comprising:

[0058] A training module for constructing a training data set and training a deep learning network based on the training data set to obtain a trained deep learning network;

[0059] An acquisition module for acquiring an OCT image to be measured and inputting it into the deep learning network, where the deep learning network includes a multi-scale densely connected dilated convolution module and an adjacent frame information supplement module;

[0060] The multi-scale densely connected dilated convolution module is used to extract image features and establish the correspondence between the optical coherence tomography image and the coronary artery intima;

[0061] The adjacent frame information supplement module is used to calculate the consistency of adjacent frames in the optical coherence tomography image and obtain the adjacent frame supplement relationship;

[0062] An output module for outputting the intima boundary segmentation boundary result according to the correspondence between the optical coherence tomography image and the coronary artery intima and the adjacent frame supplement relationship.

[0063] A device for segmenting the coronary artery intima boundary based on adjacent frame consistency:

[0064] At least one processor;

[0065] At least one memory for storing at least one program;

[0066] When the at least one program is executed by the at least one processor, the at least one processor implements the method for segmenting the coronary artery intima boundary based on adjacent frame consistency as described above.

[0067] The content in the above method embodiments is applicable to the present device embodiments. The functions specifically implemented in the present device embodiments are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0068] A storage medium storing instructions executable by a processor, characterized in that: the instructions executable by the processor are used to implement the above-mentioned method for coronary artery intima boundary segmentation based on adjacent frame consistency when executed by the processor.

[0069] The content in the above method embodiments is applicable to the present storage medium embodiments. The functions specifically implemented in the present storage medium embodiments are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0070] The above is a specific description of the preferred embodiments of the present invention. However, the present invention is not limited to the described embodiments. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for segmenting the coronary artery intima boundary based on adjacent frame consistency, characterized in that, it includes the following steps: Obtain the OCT image to be measured and input it into the deep learning network; The deep learning network includes a dilated convolution module with multi-scale dense connections and an adjacent frame information supplement module; Based on the dilated convolution module with multi-scale dense connections, extract image features and establish the correspondence between the optical coherence tomography image and the coronary artery intima; Based on the adjacent frame information supplement module, calculate the consistency of adjacent frames in the optical coherence tomography image and obtain the adjacent frame supplement relationship; According to the correspondence between the optical coherence tomography image and the coronary artery intima and the adjacent frame supplement relationship, output the boundary segmentation result of the intima boundary; The step of calculating the consistency of adjacent frames in the optical coherence tomography image and obtaining the adjacent frame supplement relationship based on the adjacent frame information supplement module specifically includes: Based on the adjacent frame information supplement module, calculate the cosine similarity vector map between adjacent frame features; Judge the consistency of adjacent frames in the corresponding optical coherence tomography image according to the cosine similarity vector map between adjacent frame features; Obtain the adjacent frame supplement relationship; The formula of the adjacent frame information supplement module is expressed as follows: c a,b (i,j; f a , f b ) = max σ(f a (i,j), f b (i,j)) In the above formula, σ represents the cosine similarity vector diagram between the features of two adjacent frames, and c a,b (i,j; f a , f b ) represents the maximum consistency of the pixel points f a (i,j) and f b (i,j); ρ(·) represents the similarity between each frame and its adjacent frames, H is the height of the image frame, and W represents the width of the image frame. represents the mean value of the consistency of different frames.

2. The method for segmenting the coronary artery intima boundary based on adjacent frame consistency according to claim 1, characterized in that, before the step of obtaining the OCT image to be measured and inputting it into the deep learning network, it further includes: Construct a training data set and train the deep learning network based on the training data set to obtain a trained deep learning network.

3. The method for segmenting the coronary artery intima boundary based on adjacent frame consistency according to claim 2, characterized in that, the step of constructing a training data set and training the deep learning network based on the training data set to obtain a trained deep learning network specifically includes: Collect OCT images with different health conditions and the corresponding coronary artery intima segmentation results as the training data set; Based on the training data set, use OCT images with different health conditions as input and the corresponding coronary artery intima segmentation results as output to train the deep learning network; Adjust the network parameters of the deep learning network until the error rate reaches the preset range to obtain a trained deep learning network.

4. The method for segmenting the coronary artery intima boundary based on adjacent frame consistency according to claim 3, characterized in that, the network parameters of the deep learning network include the number of convolutional layers, the number of dilated convolutional layers, the number of BN layers, the number of RELU layers, the number of pooling layers, the number of upsampling layers, the number of output layers, the initial weight and the bias value.

Images