An automatic evaluation system for the coverage rate of neointima of stents based on the OCT platform

Abstract

This application is applicable to the field of medical image processing technology, and provides an automatic evaluation system for the coverage rate of neointima of a stent based on an OCT platform. The system includes: an acquisition module, a processing module, and an evaluation module; the acquisition module is used to acquire an OCT image of a target lumen, and an embedded stent is arranged in the target lumen; the processing module is used to input the OCT image into a preset image segmentation model for processing, and output a mask image corresponding to the OCT image, and the mask image is used to identify the lumen and the embedded stent; the evaluation module is used to calculate the coverage rate of the neointima between the lumen and the embedded stent according to the mask image. Directly using the preset image segmentation model to identify the embedded stent in the OCT image can effectively avoid the problem that the judgment result is inaccurate due to using the recognition results of the stent and the lumen in the OCT image to judge whether the stent is an embedded stent, speeds up the detection speed of the coverage degree of the embedded stent, and improves the accuracy of the judgment result.

Description

Technical Field

[0001] This application belongs to the technical field of medical image processing, and particularly relates to an automatic evaluation system for the neointimal coverage rate of stents based on an OCT platform. Background Art

[0002] Generally, stents in blood vessels include two types: embedded stents and non-embedded stents. In the prior art, first, it is necessary to identify the stents and lumens in the collected Optical Coherence Tomography (OCT) images. Based on the identified stents, the distance between the stents and the lumens is measured. By comparing the distance between the stents and the lumens with a preset distance threshold, it is determined whether the identified stents are embedded stents, and then the healing condition of the embedded stents is determined according to the judgment result, so as to formulate reasonable and effective medical orders.

[0003] However, in the above method, if there are errors in the recognition results, the measured distance between the stents and the lumens will be inaccurate, affecting the judgment results of the embedded stents, reducing the recognition accuracy, and further affecting the doctor's judgment on the healing condition of the embedded stents. Summary of the Invention

[0004] This application provides an automatic evaluation system for the neointimal coverage rate of stents based on an OCT platform, which can obtain the neointimal coverage rate between the lumen and the embedded stent by analyzing the collected OCT images.

[0005] In a first aspect, this application provides an automatic evaluation system for the neointimal coverage rate of stents based on an OCT platform. The system includes: an acquisition module, a processing module, and an evaluation module. The acquisition module is used to acquire OCT images of a target lumen, and an embedded stent is arranged in the target lumen; the processing module is used to input the OCT images into a preset image segmentation model for processing and output a mask image corresponding to the OCT images, and the mask image is used to identify the lumen and the embedded stent; the evaluation module is used to evaluate the neointimal coverage rate between the lumen and the embedded stent according to the mask image.

[0006] In a possible implementation of the first aspect, the preset segmentation model includes a first convolutional layer, a first downsampling module, a second downsampling module, a first upsampling module, a second upsampling module, a third upsampling module, and a first activation layer. The first convolutional layer, the first downsampling module, the second downsampling module, and the first upsampling module are connected in sequence. The feature map output by the first upsampling module and the feature map output by the first downsampling module are input to the second upsampling module after being processed by a first splicing operation and a second activation function. The feature map output by the first downsampling module is input to the third upsampling module for processing, and the obtained feature map, the feature map output by the first convolutional layer, and the feature map processed by the second upsampling module are output to obtain a mask image after being processed by a second splicing operation and a first activation function.

[0007] In a possible implementation of the first aspect, the first downsampling module and the second downsampling module each include a downsampling layer and a second convolutional layer, and the first upsampling module, the second upsampling module, and the third upsampling module each include an upsampling layer and a third convolutional layer. The first convolutional layer, the second convolutional layer, and the third convolutional layer all use the same mode of convolution.

[0008] In a possible implementation of the first aspect, the training method of the image segmentation model includes: building a network model, where the network model includes an initial image segmentation model and a SE-ResNet model;

[0009] According to a preset loss function and a training set, the network model is trained to obtain a trained image segmentation model by training the initial image segmentation model. The SE-ResNet model is used to extract the predicted edge mask of the OCT image samples in the training set.

[0010] In a possible implementation of the first aspect, the training set includes OCT image samples, mask image samples corresponding to each OCT image sample, and edge mask samples corresponding to each OCT image sample. The loss function is used to constrain the error between the predicted image corresponding to the OCT image sample and the mask image sample, and the error between the predicted edge mask corresponding to the OCT image sample and the edge mask sample. The predicted image is the image obtained by processing the OCT image sample by the initial image segmentation model, and the predicted edge mask is the image obtained by processing the OCT image sample by the SE-ResNet model.

[0011] In a possible implementation of the first aspect, the above-mentioned SE-ResNet model includes a fourth convolutional layer, a compression layer, and an excitation layer connected in sequence.

[0012] The output of the fourth convolutional layer and the output of the excitation layer are subjected to a weighted operation, and the output after the weighted operation and the input of the fourth convolutional layer are output as an edge after an addition operation.

[0013] In a possible implementation of the first aspect, the scaling factor in the above-mentioned excitation layer is 16.

[0014] In a second aspect, the present application provides a method for detecting the neointimal coverage rate, which is applied to the automatic evaluation system for the neointimal coverage rate of a stent based on an OCT platform described in the first aspect or any optional implementation of the first aspect. The method includes:

[0015] Obtain an OCT image of a target lumen, where an embedded stent is provided in the target lumen;

[0016] Input the OCT image into a preset image segmentation model for processing, and output a mask image corresponding to the OCT image. The mask image is used to identify the lumen and the embedded stent;

[0017] Evaluate the neointimal coverage rate between the lumen and the embedded stent according to the mask image.

[0018] In a possible implementation of the second aspect, the preset segmentation model includes a first convolutional layer, a first downsampling module, a second downsampling module, a first upsampling module, a second upsampling module, a third upsampling module, and a first activation layer. The first convolutional layer, the first downsampling module, the second downsampling module, and the first upsampling module are connected in sequence. The feature map output by the first upsampling module and the feature map output by the first downsampling module are input to the second upsampling module after being processed by a first splicing operation and a second activation function; the feature map obtained by processing the feature map output by the first downsampling module by the third upsampling module, the feature map output by the first convolutional layer, and the feature map processed by the second upsampling module are output to obtain a mask image after being processed by a second splicing operation and a first activation function.

[0019] In a possible implementation of the second aspect, the first downsampling module and the second downsampling module each include a downsampling layer and a second convolutional layer, and the first upsampling module, the second upsampling module, and the third upsampling module each include an upsampling layer and a third convolutional layer; the first convolutional layer, the second convolutional layer, and the third convolutional layer all use the same mode of convolution.

[0020] In a possible implementation of the second aspect, the training method of the image segmentation model includes: building a network model, where the network model includes an initial image segmentation model and a SE-ResNet model;

[0021] Train the network model according to a preset loss function and a training set, and train the initial image segmentation model to obtain a trained image segmentation model. The SE-ResNet model is used to extract the predicted edge mask of the OCT image samples in the training set.

[0022] In a possible implementation of the second aspect, the training set includes OCT image samples, mask image samples corresponding to each OCT image sample, and edge mask samples corresponding to each OCT image sample; the loss function is used to constrain the error between the predicted image corresponding to the OCT image sample and the mask image sample, and the error between the predicted edge mask corresponding to the OCT image sample and the edge mask sample, where the predicted image is the image obtained by processing the OCT image sample by the initial image segmentation model, and the predicted edge mask is the image obtained by processing the OCT image sample by the SE-ResNet model.

[0023] In a possible implementation of the second aspect, the above-mentioned SE-ResNet model includes a fourth convolutional layer, a squeezing layer, and an excitation layer connected in sequence.

[0024] The output of the fourth convolutional layer and the output of the excitation layer are subjected to a weighted operation, and the output after the weighted operation and the input of the fourth convolutional layer are added to output the edge.

[0025] In a possible implementation of the second aspect, the scaling coefficient in the above-mentioned excitation layer is 16.

[0026] In a third aspect, the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the method as described in the second aspect or any optional implementation of the second aspect when executing the computer program.

[0027] In a fourth aspect, a computer-readable storage medium stores a computer program, and the computer program implements the method as described in the second aspect or any optional implementation of the second aspect when executed by a processor.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product, which causes a terminal device to execute the method as described in the second aspect or any optional implementation of the second aspect when the computer program product runs on the terminal device.

[0029] The beneficial effects of the embodiments of the present application compared with the prior art are:

[0030] Through a stent neointimal coverage automatic evaluation system based on the OCT platform provided by this application, the system directly processes the OCT image of the target lumen by using a preset image segmentation model to obtain a mask image corresponding to the OCT image, and then evaluates the neointimal coverage rate between the lumen and the embedded stent according to the above mask image. Directly using the preset image segmentation model to identify the embedded stent in the OCT image can effectively avoid the problem that the judgment result is inaccurate due to using the recognition results of the stent and the lumen in the OCT image to judge whether the stent is an embedded stent, which affects the doctor's judgment of the healing condition of the embedded stent, speeds up the detection speed of the coverage degree of the embedded stent, and improves the accuracy of the judgment result. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0032] Figure 1 It is a schematic structural diagram of an image segmentation model provided by an embodiment of this application;

[0033] Figure 2 It is a schematic flowchart of a process for generating an image segmentation model provided by an embodiment of this application;

[0034] Figure 3 It is a schematic structural diagram of a SE-ResNet model provided by an embodiment of this application;

[0035] Figure 4 It is a schematic diagram of the results of processing the same image by using different image segmentation models provided by an embodiment of this application;

[0036] Figure 5 It is a schematic diagram of an edge-fitted image provided by an embodiment of this application;

[0037] Figure 6 It is a schematic diagram of a stent neointimal coverage automatic evaluation system based on the OCT platform provided by an embodiment of this application;

[0038] Figure 7 It is a schematic diagram of a terminal device provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] In view of the problem that the current method of determining whether a stent is an embedded stent based on the recognition result of the stent in the OCT image leads to inaccurate judgment results and affects the doctor's judgment of the healing status of the embedded stent, the present application provides an automatic evaluation system for the coverage rate of the neointima of the stent based on the OCT platform. This system directly uses a preset image segmentation model to identify the embedded stent in the OCT image, effectively avoiding the problem that the judgment result is inaccurate due to using the recognition results of the stent and the lumen in the OCT image to determine whether the stent is an embedded stent, which affects the doctor's judgment of the healing status of the embedded stent, accelerating the detection speed of the coverage degree of the embedded stent, and improving the accuracy of the judgment result.

[0040] The technical solution of the present application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0041] As Figure 1 shown is a schematic structural diagram of an image segmentation model provided by an embodiment of the present application. This image segmentation model can be deployed in an optical coherence tomography (OCT) image acquisition device, or can also be deployed in other control devices associated with the OCT image acquisition device. For example, the device for deploying this image segmentation model can be a mobile terminal such as a smart phone, a tablet computer, a camera, etc., or can also be other devices capable of image segmentation such as a desktop computer, a robot, a server, etc.

[0042] Referring to Figure 1 , the image segmentation model provided by the embodiment of the present application is a densely connected U-Net model, and the densely connected U-Net model is used to extract a mask image that integrates different feature information from the OCT image.

[0043] The depth of this densely connected U-Net model is 3 layers, specifically including a first convolutional layer, a first downsampling module, a second downsampling module, a first upsampling module, a second activation function, a second upsampling module, a third upsampling module, and a first activation function. The first convolutional layer, the first downsampling module, the second downsampling module, and the first upsampling module are connected in sequence. The feature map output by the first upsampling module and the feature map output by the first downsampling module are input to the second upsampling module after being processed by the first splicing operation and the second activation function; the feature map output by the first downsampling module is input to the third upsampling module for processing, and the obtained feature map, the feature map output by the first convolutional layer, and the feature map processed by the second upsampling module are output to obtain a first mask image after being processed by the second splicing operation and the first activation function.

[0044] It should be understood that both the first downsampling module and the second downsampling module include a downsampling layer and a second convolutional layer. Among them, the downsampling layer is implemented by a max pooling layer, and the size of the feature image after being processed by the downsampling layer will be reduced by half. Correspondingly, both the first upsampling module, the second upsampling module, and the third upsampling module include an upsampling layer and a third convolutional layer.

[0045] In order to efficiently fuse the feature information between different convolutional layers and make the size of the output feature map consistent with the size of the input feature map. In the embodiments of the present application, the first convolutional layer, the second convolutional layer in the downsampling module, and the third convolutional layer in the upsampling module all use the same mode of convolution.

[0046] It should be noted that the image segmentation model provided by the present application has versatility. It can be applied to the field of medical image segmentation, and can extract organs, tissues, embedded stents, etc. in medical images to complete the segmentation tasks of different organs. It can also be applied to other tasks evaluated by the effect of image segmentation.

[0047] It can be understood that for different image segmentation tasks, the initial image segmentation model can be trained by designing corresponding training sets and loss functions, so as to obtain an image segmentation model applicable to different image segmentation tasks.

[0048] According to actual application requirements, the execution entity for training the image segmentation model and the execution entity for performing the image segmentation task using the image segmentation model can be the same or different.

[0049] Next, taking the segmentation task of lumen and embedded stent from OCT images as an example, the training process and effect of the image segmentation model provided by the present application will be exemplarily described.

[0050] As Figure 2 shown is a schematic flowchart of a process for generating an image segmentation model provided by an embodiment of the present application. Refer to Figure 2 , first, a network model is constructed, and this network model includes an initial image segmentation model (i.e., a densely connected U-Net initial model) and a SE-ResNet model.

[0051] Among them, as Figure 3 shown is a schematic structural diagram of a SE-ResNet model provided by an embodiment of the present application. Refer to Figure 3 , this SE-ResNet model includes a fourth convolutional layer, a compression layer, and an excitation layer. The output of the fourth convolutional layer and the output of the excitation layer are subjected to a weighted operation process. The output after the weighted operation process and the input of the fourth convolutional layer are subjected to an addition operation process and then the edge mask is output.

[0052] It should be understood that the input OCT image is processed by the fourth convolutional layer to obtain a feature map X, where X ∈ R C×W×H , C represents the number of channels of the feature map, W represents the width of the feature map, and H represents the height of the feature map.

[0053] The obtained feature map X is input into the compression layer for processing, and the output is Z. The calculation formula of Z is as follows in formula (1). The compression layer adopts global average pooling operation, that is, global spatial information of the feature map X input into the compression layer is compressed to obtain the output Z.

[0054]

[0055] In the above formula (1), W represents the width of the feature map, H represents the height of the feature map, and (i, j) represents the pixel points in the feature map.

[0056] The Z obtained after compression processing is input into the excitation layer (including two sets of fully connected layers and activation functions set at intervals) for processing to obtain the excitation value S. The calculation formula of S is as follows:

[0057] S = σ(W 2 δ(W 1 Z)) (2)

[0058] In the above formula (2), W 2 δ(W 1 Z) and W 1 Z represent two fully connected layers, σ is the sigmoid function, δ can be the ReLU (Rectified Linear Unit) activation function, and W1 and W2 are the parameter matrices of the two fully connected layers respectively.

[0059] It should be noted that for the lumen and embedded stent segmentation tasks, the scaling parameters of the excitation layer provided in this application are all 16. The scaling parameters of the excitation layer can be determined according to different actual applications, and this application does not limit this.

[0060] The excitation value S is weighted (scale operation) with the feature map obtained after the second convolutional layer processing, and the feature map after the weighted operation is added to the input OCT image to obtain the edge mask X out , where the edge mask X out has the following calculation formula:

[0061] X out = F scale (X, S) (3)

[0062] It should be noted that in the SE-ResNet model, the compression layer compresses the feature maps input to the global average pooling, converts the feature maps of each channel into a real number, and outputs a feature vector with the same number of dimensions as the number of feature channels of the input feature maps. The excitation layer evaluates the weights of the feature maps of each channel to obtain the weight values corresponding to each channel. The weighting operation multiplies the weight values of each channel in the scaled feature maps by the two-dimensional matrices of the corresponding channels in the feature maps obtained after the fourth convolutional layer processing to obtain the feature maps after the weighting operation.

[0063] Next, for the segmentation tasks of the lumen and the embedded stent, the corresponding training set is collected. The training set includes OCT image samples, mask image samples corresponding to the above OCT image samples, and edge mask samples corresponding to the above OCT image samples. Among them, the acquisition methods of OCT image samples include but are not limited to directly using the OCT images in the existing OCT image database and obtaining OCT images from public websites.

[0064] In practical applications, the mask image samples corresponding to the above OCT image samples are manually marked image samples, which are used to compare with the mask images output by the image segmentation model.

[0065] In one possible implementation manner, after the OCT image samples are obtained, the OCT image samples are first subjected to image preprocessing. Exemplarily, the methods of image preprocessing may include grayscale conversion, normalization, histogram equalization, and / or Gamma correction, etc.

[0066] Then, the above network model is iteratively trained according to the OCT image samples in the training set and a preset loss function, and the initial image segmentation model is trained to obtain a trained image segmentation model. The preset loss function is used to constrain the error between the predicted image corresponding to the OCT image sample and the mask image sample, and the error between the predicted edge mask corresponding to the OCT image sample and the edge mask sample, where the predicted image is the image obtained by the initial image segmentation model processing the OCT image sample, and the predicted edge mask is the image obtained by the SE-ResNet model processing the OCT image sample.

[0067] For the segmentation tasks of segmenting the lumen and the embedded stent from the OCT images, the loss function in the embodiments of the present application adopts a combined loss function. The combined loss function includes a region-based Dice loss and an edge-based Boundary loss. The formula of the combined loss function is shown in formula (4):

[0068] L = αL Dice + βL Boundary (4)

[0069] In the above formula (4), L Dice represents the region-based Dice loss, and L Boundary represents the edge-based Boundary loss. α and β represent the balance coefficients, which are used to balance the influence of the region-based loss and the edge-based loss on the output result. The calculation formula of L Dice is shown in the following formula (5), and the calculation formula of L Boundary is shown in the following formula (6).

[0070]

[0071] In the above formula (5), i represents the pixel point in the image, c represents the classification corresponding to the pixel point, g represents whether the classification of the pixel point is correct, represents the probability that the pixel point is classified into a certain category.

[0072]

[0073] In the above formula (6), φG represents the boundary level set, and S θ (ξ) represents the probability output by the SE-ResNet model.

[0074] It should be understood that for the segmentation task of segmenting the lumen and the embedded stent from the OCT image, the image segmentation model trained with the above combined loss function can minimize the image region loss value as much as possible while supplementing the edge information in the image, effectively solving the problem that the segmentation loss value is large during the segmentation of the lumen image, resulting in low image segmentation accuracy.

[0075] In the network model provided by the embodiments of the present application, the dense connection U-Net model can fuse feature information of different depths. Since the Boundary loss in the above combined loss function fully considers the edge loss in the image segmentation process, applying the SE-ResNet model to the image segmentation model can enhance the reuse and expression ability of edge features. Therefore, using the network model composed of the dense connection U-Net model and the SE-ResNet model to segment the obtained OCT image can improve the segmentation accuracy of the model and enhance the recognition degree of the lumen and the embedded stent.

[0076] Based on the segmentation task of segmenting the lumen and the embedded stent from the OCT image, as Figure 4 shown, it is a result diagram of segmenting the same OCT image using the traditional U-Net image segmentation model and the image segmentation model provided by the present application respectively. As Figure 4 (a) in is the OCT image to be segmented, and as Figure 4 (b) in is the result diagram of segmenting the OCT image using the traditional U-Net image segmentation model.Figure 4 In (c), it is the result graph of segmenting the OCT image using the image segmentation model provided by this application. According to Figure 4 it not only verifies the feasibility of the image segmentation model provided by the embodiments of this application, but also from Figure 4 it can be seen that the image segmented by using the image segmentation model provided by the embodiments of this application is more accurate. Compared with the prior art, it can significantly improve the accuracy of image segmentation.

[0077] After segmenting the OCT image according to the image segmentation model provided by the embodiments of this application, a mask image is obtained. The obtained mask image can be applied to the detection task of the neointimal index. Specifically, the neointimal index in the OCT image can be calculated by using the mask image, and then the coverage degree of the inlay stent in the OCT image can be determined according to the neointimal index, so as to complete the detection task.

[0078] It should be understood that the neointimal index represents the ratio of the area enclosed by the edge of the inlay stent in the OCT image to the area enclosed by the lumen edge. The specific calculation formula is shown in the following formula (7):

[0079]

[0080] In the above formula (7), S stent represents the area covered by the inlay stent in the OCT image; S lumen represents the cross-sectional area of the lumen in the OCT image. If the ratio in formula (7) is larger, it means that the degree of the inlay stent close to the blood vessel wall (also called apposition) is higher, and the embedding degree of the inlay stent in the blood vessel is deeper; on the contrary, if the ratio in formula (7) is smaller, it means that the apposition degree of the inlay stent in the blood vessel is lower, and the embedding degree of the inlay stent in the blood vessel is shallower.

[0081] In the actual application process, as Figure 5 shown is a schematic diagram of an edge-fitted image provided by the embodiments of this application. Refer to Figure 5 , for Figure 4 the mask image shown in (c) in it is fitted, and the closed edge of the inlay stent and the closed edge of the lumen are correspondingly formed. As Figure 5 the area enclosed by the dotted line in it is the closed edge of the inlay stent, and the enclosed area is the closed edge of the lumen; then the area S stent is obtained by calculating the area enclosed by the edge of the inlay stent in the mask image, and the area S lumen is obtained by calculating the area enclosed by the lumen edge in the mask image; finally, the neointimal index is calculated according to the above formula (7).

[0082] By measuring the supporting effect of the inlay stent on the inner wall of the blood vessel by calculating the ratio, it avoids the problem of inaccurate calculation results caused by the inconsistent artificial measurement size of the inlay depth of the inlay stent in the blood vessel lumen or over-reliance on a single pixel at the edge, speeds up the detection speed of the degree of intimal coverage of the inlay stent, and improves the accuracy of the calculation results.

[0083] Based on the same inventive concept, as an implementation of the above method, an embodiment of the present application provides an automatic evaluation system for the neointimal coverage rate of an inlay stent based on an OCT platform. Refer to Figure 6 , the automatic evaluation system 600 for the neointimal coverage rate of the inlay stent based on the OCT platform includes:

[0084] An acquisition module 601, configured to acquire an OCT image of a target lumen, and an inlay stent is arranged in the target lumen;

[0085] A processing module 602, configured to input the OCT image into a preset image segmentation model for processing, and output a mask image corresponding to the OCT image, where the mask image is used to identify the lumen and the inlay stent;

[0086] An evaluation module 603, configured to evaluate the neointimal coverage rate between the lumen and the inlay stent according to the mask image.

[0087] Optionally, the preset segmentation model includes a first convolutional layer, a first downsampling module, a second downsampling module, a first upsampling module, a second upsampling module, a third upsampling module, and a first activation layer. The first convolutional layer, the first downsampling module, the second downsampling module, and the first upsampling module are connected in sequence. The feature map output by the first upsampling module and the feature map output by the first downsampling module are input to the second upsampling module after being processed by a first splicing operation and a second activation function; the feature map output by the first downsampling module is input to the third upsampling module for processing, and the feature map obtained, the feature map output by the first convolutional layer, and the feature map processed by the second upsampling module are output to obtain a mask image after being processed by a second splicing operation and a first activation function.

[0088] Optionally, the first downsampling module and the second downsampling module each include a downsampling layer and a second convolutional layer, and the first upsampling module, the second upsampling module, and the third upsampling module each include an upsampling layer and a third convolutional layer; the first convolutional layer, the second convolutional layer, and the third convolutional layer all use the same mode of convolution.

[0089] Optionally, the training method of the image segmentation model includes: building a network model, and the network model includes an initial image segmentation model and a SE-ResNet model;

[0090] The network model is trained according to a preset loss function and a training set, and the initial image segmentation model is trained to obtain a trained image segmentation model. The SE-ResNet model is used to extract the predicted edge mask of the OCT image samples in the training set.

[0091] Optionally, the training set includes OCT image samples, mask image samples corresponding to each OCT image sample, and edge mask samples corresponding to each OCT image sample; the loss function is used to constrain the error between the predicted image corresponding to the OCT image sample and the mask image sample, and the error between the predicted edge mask corresponding to the OCT image sample and the edge mask sample. The predicted image is the image obtained after the initial image segmentation model processes the OCT image sample, and the predicted edge mask is the image obtained after the SE-ResNet model processes the OCT image sample.

[0092] Optionally, the above-mentioned SE-ResNet model includes a fourth convolutional layer, a squeezing layer, and an excitation layer connected in sequence.

[0093] The output of the fourth convolutional layer and the output of the excitation layer are subjected to a weighted operation, and the output after the weighted operation and the input of the fourth convolutional layer are added to output the edge.

[0094] Optionally, the scaling factor in the above-mentioned excitation layer is 16.

[0095] The stent neointimal coverage automatic evaluation system 600 based on the OCT platform provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0096] Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment, which will not be elaborated here.

[0097] Based on the same inventive concept, the embodiment of the present application also provides a terminal device. Figure 7Schematic diagram of the terminal device provided by the embodiment of the present application, as Figure 7 shown, the terminal device provided by this embodiment includes: a memory 701 and a processor 702. The memory 701 is used to store a computer program 703; the processor 702 is used to execute the method described in the above method embodiment when calling the computer program. Alternatively, when the processor 702 executes the computer program 703, it implements the functions of each module / unit in the above device embodiments, for example Figure 6 the functions of the units 601 to 603 shown.

[0098] Exemplarily, the computer program 703 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 701 and executed by the processor 702 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and this instruction segment is used to describe the execution process of the computer program in the terminal device.

[0099] Those skilled in the art can understand that Figure 7 merely examples of the terminal device, and do not constitute a limitation to the terminal device. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, the terminal device 700 may further include input / output devices, network access devices, buses, etc.

[0100] The processor 702 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0101] The memory 701 may be an internal storage unit of the terminal device, such as the hard disk or memory of the terminal device. The memory 701 may also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the terminal device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 701 may also include both the internal storage unit and the external storage device of the terminal device. The memory 701 is used to store the computer program and other programs and data required by the terminal device. The memory 701 may also be used to temporarily store the data that has been output or will be output.

[0102] The terminal device provided in this embodiment may execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0103] This application embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the above method embodiment is implemented.

[0104] This application embodiment also provides a computer program product. When the computer program product runs on a terminal device, the terminal device is enabled to execute the method described in the above method embodiment.

[0105] This application embodiment also provides a chip system, including a processor. The processor is coupled to a memory, and the processor executes the computer program stored in the memory to implement the method described in the above method embodiment. Among them, the chip system may be a single chip or a chip module composed of multiple chips.

[0106] When the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable storage medium can at least include: any entity or device that can carry the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, computer-readable media may not be electrical carrier signals and telecommunication signals.

[0107] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0109] In the embodiments provided in this application, it should be understood that the disclosed device / equipment and method can be implemented in other ways. For example, the device / equipment embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical or other forms.

[0110] It should be understood that, as used in the specification of this application and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their combinations.

[0111] It should also be understood that the term "and / or" as used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0112] As used in the specification of this application and the appended claims, the term "if" may be construed, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrases "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0113] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0114] Reference to "one embodiment" or "some embodiments" or the like described in the specification of this application means that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An automatic evaluation system for the neointimal coverage rate of a stent based on an OCT platform, characterized in that, the system includes: an acquisition module, a processing module and an evaluation module; the acquisition module is used to acquire the OCT image of the target lumen, and an embedded stent is arranged in the target lumen; the processing module is used to input the OCT image into a preset image segmentation model for processing, and output a mask image corresponding to the OCT image, and the mask image is used to identify the lumen and the embedded stent; the evaluation module is used to evaluate the neointimal coverage rate between the lumen and the embedded stent according to the mask image; wherein, the preset image segmentation model includes a first convolutional layer, a first downsampling module, a second downsampling module, a first upsampling module, a second upsampling module, a third upsampling module and a first activation function. The first convolutional layer, the first downsampling module, the second downsampling module and the first upsampling module are connected in sequence. The feature map output by the first upsampling module and the feature map output by the first downsampling module are input into the second upsampling module after being processed by a first splicing operation and a second activation function; the feature map obtained by processing the feature map output by the first downsampling module by the third upsampling module, the feature map output by the first convolutional layer and the feature map processed by the second upsampling module are output to obtain the mask image after being processed by a second splicing operation and the first activation function; the first downsampling module and the second downsampling module each include a downsampling layer and a second convolutional layer, and the first upsampling module, the second upsampling module and the third upsampling module each include an upsampling layer and a third convolutional layer; the first convolutional layer, the second convolutional layer and the third convolutional layer all use the same mode of convolution.

2. The system according to claim 1, characterized in that, the training method of the image segmentation model includes: building a network model, the network model includes an initial image segmentation model and a SE-ResNet model; training the network model according to a preset loss function and a training set, and training the initial image segmentation model to obtain the trained image segmentation model, and the SE-ResNet model is used to extract the predicted edge mask of the OCT image sample in the training set.

3. The system according to claim 2, characterized in that, the training set includes the OCT image sample, the mask image sample corresponding to each OCT image sample and the edge mask sample corresponding to each OCT image sample; the loss function is used to constrain the error between the predicted image corresponding to the OCT image sample and the mask image sample, and the error between the predicted edge mask corresponding to the OCT image sample and the edge mask sample. The predicted image is the image obtained by processing the OCT image sample by the initial image segmentation model, and the predicted edge mask is the image obtained by processing the OCT image sample by the SE-ResNet model.

4. The system according to claim 2, wherein, the SE-ResNet model includes a fourth convolutional layer, a compression layer, and an excitation layer connected in sequence, the output of the fourth convolutional layer and the output of the excitation layer are subjected to a weighted operation, and the output after the weighted operation and the input of the fourth convolutional layer are subjected to an addition operation to output the edge.

5. The system according to claim 4, wherein, the scaling factor in the excitation layer is 16.

6. A method for detecting the neointimal coverage rate, wherein, the method is applied to the automatic evaluation system for the neointimal coverage rate of the stent based on the OCT platform according to any one of claims 1-5, and the method includes: acquiring an OCT image of a target lumen, wherein an embedded stent is provided in the target lumen; inputting the OCT image into a preset image segmentation model for processing, and outputting a mask image corresponding to the OCT image, where the mask image is used to identify the lumen and the embedded stent; evaluating the neointimal coverage rate between the lumen and the embedded stent according to the mask image; wherein, the preset image segmentation model includes a first convolutional layer, a first downsampling module, a second downsampling module, a first upsampling module, a second upsampling module, a third upsampling module, and a first activation function, the first convolutional layer, the first downsampling module, the second downsampling module, and the first upsampling module are connected in sequence, and the feature map output by the first upsampling module and the feature map output by the first downsampling module are input into the second upsampling module after being processed by a first splicing operation and a second activation function; the feature map obtained by processing the feature map output by the first downsampling module by the third upsampling module, the feature map output by the first convolutional layer, and the feature map processed by the second upsampling module are output to obtain the mask image after being processed by a second splicing operation and the first activation function; the first downsampling module and the second downsampling module each include a downsampling layer and a second convolutional layer, and the first upsampling module, the second upsampling module, and the third upsampling module each include an upsampling layer and a third convolutional layer; the first convolutional layer, the second convolutional layer, and the third convolutional layer all use the same mode of convolution.

7. A terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, when the processor executes the computer program, the method according to claim 6 is implemented.

8. A computer-readable storage medium storing a computer program, wherein, when the computer program is executed by a processor, the method according to claim 6 is implemented.

Images