A method and apparatus for calculating the IPA of an intracavity OCT image

Abstract

The present application relates to the technical field of medical devices, and provides a method for calculating the IPA of an intracavitary OCT image, including: obtaining an intracavitary OCT image; determining a calcified plaque region of the intracavitary OCT image; determining an optical attenuation coefficient of the intracavitary OCT image, where the optical attenuation coefficient of the intracavitary OCT image does not include the optical attenuation coefficient of the calcified plaque region; and determining the IPA of the intracavitary OCT image according to the optical attenuation coefficient of the intracavitary OCT image. The above method can improve the accuracy of the IPA.

Description

Technical Field

[0001] The present application relates to the field of medical device technology, and in particular to a method and device for calculating the IPA of an intracavitary OCT image. Background Art

[0002] Intracavitary optical coherence tomography (OCT) has a high resolution and has become a commonly used imaging technology in percutaneous coronary intervention (PCI) surgery. In intracavitary OCT images, different intracavitary tissues have different light attenuation coefficients, so the light attenuation coefficient can be used to distinguish different intracavitary tissues.

[0003] Since TCFA belongs to vulnerable plaque, it is closely related to the occurrence of cardiovascular diseases and is the main cause of thrombosis, acute coronary syndrome, coronary heart disease and other diseases. Therefore, accurately identifying the presence and severity of TCFA in cardiovascular disease is of great significance in the prevention and diagnosis of cardiovascular diseases. The plaque attenuation index (IPA) is an identification index calculated based on the light attenuation coefficient. It can well distinguish between thin-cap fibrous atherosclerotic plaques (TCFA) (i.e. unstable plaques) and atherosclerotic plaques (FA) (i.e. stable plaques). For example, IPA is used to identify whether TCFA is contained in the intraluminal vascular tissue. When TCFA is contained in the intraluminal blood vessel, the calculation result of IPA will be high (higher than the given threshold), and it can be determined that TCFA is contained in the intraluminal blood vessel; when TCFA is not contained in the intraluminal blood vessel, the calculation result of IPA will be low (lower than the given threshold), and it can be determined that TCFA is not contained in the intraluminal blood vessel. However, when the intraluminal vascular tissue contains calcified plaques, it will make the calculation result of IPA higher (higher than a given threshold). At this time, the calculation result of IPA cannot accurately reflect whether TCFA is contained in the intraluminal blood vessels.

[0004] Therefore, how to improve the accuracy of IPA is an urgent problem that needs to be solved. Summary of the invention

[0005] The present application provides a method for calculating the IPA of an intracavitary OCT image, which can improve the accuracy of the IPA.

[0006] In a first aspect, a method for calculating the IPA of an in-vivo OCT image is provided, including: obtaining an in-vivo OCT image; determining a calcified plaque region of the in-vivo OCT image; determining the light attenuation coefficient of the in-vivo OCT image, where the light attenuation coefficient of the in-vivo OCT image does not include the light attenuation coefficient of the calcified plaque region; and determining the IPA of the in-vivo OCT image according to the light attenuation coefficient of the in-vivo OCT image.

[0007] The above method can be executed by a terminal device or a chip in the terminal device. Taking the vascular tissue in the body as an example, when vascular calcification occurs, the vascular OCT image collected by an OCT machine of the calcified blood vessel will contain a calcified plaque region. If the calcified plaque region in the vascular OCT image is not removed, and the IPA value corresponding to the light attenuation coefficient image of the vascular OCT image is directly calculated, the calculated result of the IPA will be too high. However, at this time, it cannot be determined that there must be a TCFA in the vascular OCT image based on the calculated result of the IPA. The reason is that regardless of whether there is a TCFA in the vascular OCT image, as long as there is a calcified plaque region in the vascular OCT image, the light attenuation coefficient of the calcified plaque region in the light attenuation coefficient image corresponding to the vascular OCT image will become larger, resulting in a too high IPA value calculated based on the light attenuation coefficient. If the calcified plaque region in the vascular OCT image is removed, for example, the light attenuation coefficient corresponding to the calcified plaque region in the vascular OCT image is set to 0, and then the IPA value of the light attenuation coefficient image after removing the calcified plaque region is calculated. If the calculated result of the IPA value is higher than a preset value, it can be determined that there is a TCFA in the vascular OCT image. If the calculated result of the IPA value of the light attenuation coefficient image is lower than the preset value, it can be determined that there is no TCFA in the vascular OCT image. Thus, it can be seen that only after removing the calcified plaque region in the vascular OCT image can the IPA value of the light attenuation coefficient image corresponding to the vascular OCT image be accurately calculated, and then it can be determined whether there is a TCFA in the vascular OCT image based on the IPA value.

[0008] Optionally, the determining the calcified plaque region of the in-vivo OCT image includes: processing the in-vivo OCT image through a target convolutional neural network to determine the calcified plaque region of the in-vivo OCT image. Compared with medical experts using professional software to mark the calcified plaque region of the in-vivo OCT image, the method of using a target neural network in this application to identify the calcified plaque region of the in-vivo OCT image can quickly and accurately identify the calcified plaque region.

[0009] Optionally, the target convolutional neural network is trained by the following method: processing the in-vivo OCT training images through the convolutional neural network to be trained to generate a first feature map; obtaining the texture feature matrix of the calcified plaque region in the in-vivo OCT training images; generating a prediction mask according to the first feature map and the texture feature matrix; obtaining the region of interest of the in-vivo OCT training images, where the region of interest is used to characterize the calcified plaque region in the in-vivo OCT training images; training the convolutional neural network to be trained according to the prediction mask, the region of interest and the standard mask to generate the target convolutional neural network, where the prediction mask is a predicted value, the standard mask is a true value, and the region of interest is used to improve the learning ability of the loss function of the convolutional neural network to be trained for the edge structure information of the calcified plaque region.

[0010] Stitch the first feature map generated by processing the in-vivo OCT training images through the convolutional neural network to be trained and the texture feature matrix of the calcified plaque region in the in-vivo OCT training images to generate a prediction mask; train the convolutional neural network to be trained by combining the prediction mask, the region of interest and the standard mask to generate the target convolutional neural network. The above-mentioned region of interest is used to improve the learning ability of the loss function of the convolutional neural network to be trained for the edge structure information of the calcified plaque region. In addition, the recognition accuracy of the edge structure information of the calcified plaque region is improved by increasing the weight of the region of interest in the loss function. Since there is a problem of low recognition accuracy for the irregular edges of the calcified plaque region when only using a deep learning model to recognize the calcified plaque region in the in-vivo OCT images, therefore, the present application proposes to use the region of interest and the texture features of the in-vivo OCT images to assist in training the convolutional neural network to be trained, so that the target neural network can accurately recognize the calcified plaque region in the in-vivo OCT images.

[0011] Optionally, the obtaining the region of interest of the in-vivo OCT training images includes: obtaining multiple A-lines of the in-vivo OCT training images; determining the region of interest of the in-vivo OCT training images according to the pixel points corresponding to the maximum optical attenuation coefficients on each of the multiple A-lines.

[0012] Optionally, generating a prediction mask according to the first feature map and the texture feature matrix includes: stitching the first feature map and the texture feature matrix to generate a second feature map; performing dimensionality reduction processing on the second feature map to generate the prediction mask.

[0013] Optionally, the performing dimensionality reduction processing on the second feature map includes: performing dimensionality reduction processing on the second feature map through three 1×1 convolutional layers.

[0014] Optionally, obtaining the texture feature matrix of the calcified plaque region in the intracavitary OCT training image includes: determining the spatial gray-level co-occurrence matrix of the intracavitary OCT training image; determining at least one texture feature of the intracavitary OCT training image according to the spatial gray-level co-occurrence matrix; and determining the texture feature matrix according to the texture feature.

[0015] Optionally, the at least one texture feature includes one or more of energy, inertia, entropy, and correlation.

[0016] In a second aspect, a device for calculating the IPA of an intracavitary OCT image is provided. The device includes a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the device executes the method described in any one of the first aspect.

[0017] In a third aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the method described in any one of the first aspect. Description of the Drawings

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a schematic flowchart of the method for calculating the IPA of an intracavitary OCT image in an embodiment of the present invention;

[0020] Figure 2 It is a schematic diagram of the distribution of the A-line optical attenuation coefficients in the calcified region and the non-calcified region provided in an embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram of the calcified region of interest provided in an embodiment of the present invention;

[0022] Figure 4 It is a schematic diagram of the network model provided in an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of calcification recognition provided in an embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of the optical attenuation coefficient image and the IPA value before removing calcification provided in an embodiment of the present application;

[0025] Figure 7It is a schematic diagram of the light attenuation coefficient image and IPA value after removing calcification provided by an embodiment of the present application;

[0026] Figure 8 It is a schematic structural diagram of a device for calculating the IPA of an intracavity OCT image provided by an embodiment of the present application. Detailed implementation manners

[0027] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0028] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0029] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0031] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that specific features, structures, or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0032] The following further elaborates the present application in detail with reference to the accompanying drawings and specific embodiments.

[0033] In intravascular OCT images, different intravascular tissues have different light attenuation coefficients. Therefore, different intravascular tissues can be distinguished by using the light attenuation coefficient. The Index of plaque attenuation (IPA) is an identification index calculated based on the light attenuation coefficient, which can well distinguish Thin-Cap FibroAtheroma (TCFA) (i.e., unstable plaque) and FibroAtheroma (FA) (i.e., stable plaque). For example, IPA is used to identify whether the intravascular tissue in the cavity contains TCFA. When the intravascular tissue in the cavity contains TCFA, the calculation result of IPA will be on the high side (higher than the given threshold), and then it can be determined that the intravascular tissue in the cavity contains TCFA; when the intravascular tissue in the cavity does not contain TCFA, the calculation result of IPA will be on the low side (lower than the given threshold), and then it can be determined that the intravascular tissue in the cavity does not contain TCFA. However, when the intravascular tissue contains calcified plaque, it will make the calculation result of IPA on the high side (higher than the given threshold). At this time, the calculation result of IPA cannot accurately reflect whether the intravascular tissue in the cavity contains TCFA. Therefore, how to improve the accuracy of IPA is an urgent problem to be solved currently.

[0034] A method for calculating IPA of intravascular OCT images provided by this application can improve the accuracy of IPA. As Figure 1 shown, the method includes:

[0035] S101, obtain the intravascular OCT image.

[0036] Exemplarily, taking the intravascular tissue as an example, an OCT machine can be used to obtain the vascular OCT image. Specifically, when using the OCT machine to scan a section of blood vessel in the human body, a set of OCT pullback data will be obtained. This set of OCT pullback data contains 300 temporally adjacent vascular OCT images. For example, when using the OCT machine to scan a section of calcified blood vessel (i.e., a blood vessel with calcification lesions), a set of calcification data will be obtained. This set of calcification data contains 300 vascular OCT images. If at least 50 sets of the above calcification data are collected to construct a training data set, then this training data set contains at least 15,000 (i.e., 50×300) vascular OCT training images, where each vascular OCT training image contains a calcified plaque area.

[0037] S102, determine the calcified plaque area of the intravascular OCT image.

[0038] Exemplarily, taking intravascular tissue as an example, and taking the training dataset constructed with 50 groups of the above-mentioned calcification data (i.e., containing 15,000 vascular OCT training images) as an example, by inviting multiple medical experts from multiple centers, such as 30 medical experts, to use professional software (such as Labelme software) to manually mark the calcified plaque areas of 15,000 vascular OCT training images respectively. The specific marking method is as follows: If the A position range of a certain vascular OCT training image is jointly marked as a calcified plaque area by more than half of the medical experts, then the A position range of this vascular OCT training image is considered as a calcified plaque area, and thus the calcified plaque area marked by the medical experts of this vascular OCT training image is used as the gold standard for the calcified plaque area of this vascular OCT training image; each vascular OCT training image in this training dataset is marked in this way. Finally, the gold standard (i.e., the standard mask) of the calcified plaque area of this training dataset is obtained.

[0039] Exemplarily, after the blood vessel is calcified, the imaging characteristics of the calcified plaque in the blood vessel in the vascular OCT image are manifested as sharp boundaries of the calcified plaque area, uneven overall distribution of the calcified plaque area, and irregularly distributed dark areas in the calcified plaque area. If only a deep learning model is used to identify the calcified plaque area in the vascular OCT image, it is very difficult to accurately identify the irregular edges of the calcified plaque area; in addition, in the vascular OCT image, there are certain differences in the imaging characteristics between deep calcification and superficial calcification, which easily lead to false detection when the deep learning model detects the edge of the calcified plaque area. Therefore, the present application proposes a technical solution of using the region of interest (i.e., the calcification region of interest map) and the texture features of the vascular OCT training image to assist in training the convolutional neural network to be trained to generate a target convolutional neural network, and then using the target convolutional neural network to identify the calcified plaque area in the vascular OCT image.

[0040] First, introduce the calculation of the region of interest (i.e., the calcification region of interest map) using the light attenuation model.

[0041] Exemplarily, obtaining the region of interest of the above-mentioned intravascular OCT training image includes: obtaining multiple A-lines of the intravascular OCT training image; determining the region of interest of the intravascular OCT training image according to the pixel points corresponding to the maximum light attenuation coefficient on each of the multiple A-lines.

[0042] Taking intravascular tissue as an example, since the vascular OCT images (i.e., vascular OCT training images) collected by the OCT machine are vascular OCT images in polar coordinates, therefore, the light attenuation model in polar coordinates can be used to calculate the light attenuation coefficient of each pixel of a single vascular OCT training image, and use the light attenuation coefficient value corresponding to each pixel to replace the value of each pixel of the vascular OCT training image, so as to obtain a single light attenuation coefficient training image corresponding to the single vascular OCT training image in polar coordinates. The calculation formula of the above light attenuation model is as follows:

[0043]

[0044]

[0045]

[0046] Among them, I 0 is the scale factor, r represents the image depth, T(r) is the longitudinal point spread function, z 0 、z R 、z c and z w respectively represent the beam waist position, Rayleigh length, scanning center point and half-width of the roll-off function, and the values are 0, 3mm, 0 and 10um respectively. u t is the light attenuation coefficient (i.e., the variable to be solved). Take the logarithm of both sides of formula (1), and then use the least squares method to calculate the light attenuation coefficient u t .

[0047] Taking a single vascular OCT image (i.e., vascular OCT training image) collected by the OCT machine as an example, it is explained how to obtain the corresponding single light attenuation coefficient training image of a single vascular OCT training image. Since a single vascular OCT training image collected by the OCT machine is 642×500, where 500 means that a total of 500 A-lines are scanned when the catheter scans 360° in the blood vessel, and 642 means that 642 pixels are scanned on each A-line. Use the light attenuation model to calculate the light attenuation coefficient of each pixel of the single vascular OCT training image to obtain the single light attenuation coefficient training image corresponding to the vascular OCT training image. This single light attenuation coefficient training image is also 642×500, where 500 means that a total of 500 A-lines are scanned when the catheter scans 360° in the blood vessel, and 642 means that 642 pixels are scanned on each A-line (i.e., the light attenuation coefficient value corresponding to each pixel in the single vascular OCT training image, that is, 642 means that 642 light attenuation coefficient values are scanned on each A-line).

[0048] For the above single light attenuation coefficient training image, there are 500 A-lines (i.e., multiple A-lines of the vascular OCT training image). The 642 light attenuation coefficient values (i.e., 642 pixel points) on each of the 500 A-lines can be used to draw a light attenuation coefficient distribution curve. When the light attenuation coefficient values on the light attenuation coefficient distribution curve show a trend of suddenly increasing from low to high and then decreasing, and the light attenuation coefficient value at the highest point of the light attenuation coefficient distribution curve (i.e., the peak) is greater than a given threshold (for example, the given threshold is 8), it indicates that there is calcification on this A-line, and the boundary of the calcification appears near the peak. Therefore, the pixel point where the peak is located is the calcification region of interest. The above given threshold is used to characterize that the light attenuation coefficient value on the A-line reaches the light attenuation coefficient value of the calcification region. As Figure 2 shown, where Figure (a) is a vascular OCT image, and the light attenuation coefficient distribution curve of the A-line at position 201 in Figure (a) is shown in Figure (b). Figure (b) is the light attenuation coefficient distribution map of the A-line where calcification is located. Among them, the abscissa represents the 1st to 500th A-lines, and the ordinate represents the light attenuation coefficient value. It can be seen from Figure (b) that there is an obvious peak in the interval [20, 30], and the peak value at this point is significantly greater than the given threshold (for example, the given threshold is 8). Therefore, there is calcification on the A-line where position 201 is located, that is, the pixel point where position 201 is located is the calcification region of interest. The light attenuation coefficient distribution curve of the A-line at position 202 in Figure (a) is shown in Figure (c). Figure (c) is the light attenuation coefficient distribution map of the A-line where there is no calcification. Among them, the abscissa represents the 1st to 500th A-lines, and the ordinate represents the light attenuation coefficient value. It can be seen from Figure (c) that there is no obvious peak in the interval [0, 80], and the maximum light attenuation coefficient value in the interval [0, 80] is also less than the given threshold (for example, the given threshold is 8). Therefore, there is no calcification on the A-line where position 202 is located, that is, the pixel point where position 202 is located is the non-calcification region of interest.

[0049] Exemplarily, the region of interest of the intracavitary OCT training image (i.e., the calcification region of interest map) is determined according to the pixel points corresponding to the maximum light attenuation coefficient on each of the multiple A-lines. Taking the intracavitary vascular tissue as an example, a single vascular OCT image has 500 A-lines, and one calcification region of interest is determined for each A-line. 500 A-lines can determine 500 calcification regions of interest. Connecting these 500 calcification regions of interest can obtain the calcification region of interest map, as Figure 3 shown, where (a) is an OCT image (i.e., a vascular OCT image); (b) is the calcification region of interest map. In the calcification region of interest map, the white curve is the calcification region of interest, and the pixel value on this white curve is 1, and the pixel values at positions other than the white curve in the calcification region of interest map are 0).

[0050] Secondly, introduce the extraction of the texture features of the vascular OCT image using the texture feature extraction algorithm.

[0051] Exemplarily, obtaining the texture feature matrix of the calcified plaque region in the intracavitary OCT training image includes: determining the spatial gray-level co-occurrence matrix of the intracavitary OCT training image; determining at least one texture feature of the intracavitary OCT training image according to the spatial gray-level co-occurrence matrix; and determining the texture feature matrix according to the texture feature. For example, taking the intracavitary vascular tissue as an example, the spatial gray-level co-occurrence matrix (i.e., the correlation matrix) of the vascular OCT image (i.e., the vascular OCT training image) is calculated by using the texture feature statistical analysis method, and at least one texture feature of the vascular OCT training image is further extracted according to the spatial gray-level co-occurrence matrix. The at least one texture feature includes one or more of energy, contrast, entropy, and correlation.

[0052] For example, calculate the energy of the vascular OCT training image, the contrast of the OCT training image, the entropy of the OCT training image, and the correlation of the OCT training image according to the spatial gray-level co-occurrence matrix. Among them, the energy of the vascular OCT training image is the sum of the squares of the element values of the spatial gray-level co-occurrence matrix, which reflects the uniformity of the gray-level distribution and the thickness of the texture of the vascular OCT training image; the contrast of the vascular OCT training image reflects the clarity of the image and the depth of the texture grooves. The deeper the texture grooves, the greater the contrast and the clearer the visual effect; the entropy of the vascular OCT training image is a measure of the amount of information in the image, which is used to represent the non-uniformity or complexity of the texture in the image; the correlation of the vascular OCT training image is a measure of the similarity degree of the elements of the spatial gray-level co-occurrence matrix in the row or column direction, which is used to reflect the local gray-level correlation in the image. Extracting the texture features of the vascular OCT training image can not only be carried out in the spatial domain, but also use the discrete cosine transform and the local Fourier transform to extract the texture features of the vascular OCT training image in the transform domain. In addition, the average pixel intensity (i.e., the mean value, also called the first-order statistic) and variance (i.e., the second-order statistic) of the pixel points of the vascular OCT training image can be calculated to characterize the texture features of the vascular OCT image.

[0053] The texture features of the vascular OCT training images are extracted by the above method. For each pixel point in the vascular OCT training images, a multi-dimensional vector, that is, a 1xK multi-dimensional texture feature vector can be obtained, where K refers to the number of texture features extracted from the vascular OCT training images. For example, when four texture features, namely, the energy of the vascular OCT training images, the inertia of the vascular OCT training images, the entropy of the vascular OCT training images, and the correlation of the vascular OCT training images, are extracted, at this time, K takes the value of 4. For the vascular OCT training images with a size of 642x500, the texture features of the vascular OCT training images are extracted by using the texture feature statistical analysis method, and finally, a texture feature matrix of 642x500xK (i.e., the OCT texture feature matrix) can be obtained. This texture feature matrix is used to assist in training the convolutional neural network to be trained.

[0054] Finally, it is introduced that the region of interest (i.e., the calcification region of interest map) and the texture features of the vascular OCT training images are used to assist in training the convolutional neural network to be trained to generate a target convolutional neural network, and then the target convolutional neural network is used to accurately identify the calcified plaque region in the vascular OCT images.

[0055] Exemplarily, taking the convolutional neural network to be trained as the U-net model as an example, as Figure 4As shown, the convolutional neural network to be trained has a total of 5 convolutional modules (i.e., downsampling modules) and 5 deconvolutional modules (i.e., upsampling modules). Each module contains three layers of convolution, and there are a pooling layer, a non-linear activation function, and a normalization layer after each convolution. Among them, the convolution kernel of each layer of convolution is 3x3 in size, the internal non-linear activation function uses the Relu function, and the pooling layer uses the average pooling method. During the training process of the convolutional neural network to be trained, the learning rate adopts the dynamic learning rate method, and the initial learning rate is 0.1. If the loss function does not decrease significantly during each training process of the convolutional neural network to be trained, then the learning rate is reduced by 10 times (i.e., 0.01). The reason is that when the learning rate is at a certain value and the loss function has not changed, it means that the parameters of the convolutional neural network to be trained may oscillate around a certain value. At this time, the learning rate can be adjusted (such as reducing the learning rate to make the convolutional neural network to be trained learn slower) to further observe whether the loss function of the convolutional neural network to be trained decreases. If the loss function does not decrease, it means that the convolutional neural network to be trained has reached the optimal network. If the loss function decreases, it means that the convolutional neural network to be trained still needs to be trained. For example, using the aforementioned training dataset, which contains 15,000 vascular OCT training images, the convolutional neural network to be trained is trained 80 times (i.e., the epoch is set to 80, where epoch represents the number of times the training dataset is cyclically trained). Completing the training of 15,000 vascular OCT images by the convolutional neural network to be trained is considered as one training. The above 15,000 vascular OCT training images are input into the convolutional neural network to be trained in groups of 8 vascular OCT training images each (i.e., the batch size is set to 8) until all 15,000 vascular OCT training images are trained, and this training is considered to end.

[0056] Exemplarily, the lumen OCT image is processed by the target convolutional neural network to determine the calcified plaque region of the lumen OCT image. For example, taking the lumen vascular tissue as an example, any vascular OCT image is input into the trained target neural network (such as the trained U-net model). After the target neural network processes the vascular OCT image, the calcified plaque region in the vascular OCT image is output.

[0057] Exemplarily, the above-mentioned target convolutional neural network can be trained by the following method: processing the in-vivo OCT training images through the convolutional neural network to be trained to generate a first feature map; obtaining the texture feature matrix of the calcified plaque area in the in-vivo OCT training images; generating a prediction mask according to the first feature map and the texture feature matrix; obtaining the region of interest of the in-vivo OCT training images, where the region of interest is used to characterize the calcified plaque area in the in-vivo OCT training images; training the convolutional neural network to be trained according to the prediction mask, the region of interest and the standard mask to generate the target convolutional neural network, where the prediction mask is the predicted value, the standard mask is the true value, and the region of interest is used to optimize the loss function of the convolutional neural network to be trained.

[0058] For example, taking the in-vivo vascular tissue as an example, the convolutional neural network to be trained is a U-net model. As Figure 4 shown, a group of vascular OCT training images are input into the U-net model. The size of each vascular OCT training image is 642x500. The process of the U-net model processing the vascular OCT training images is as follows: First, the vascular OCT training image with a size of 642x500 is input into the downsampling module. Specifically, after the vascular OCT training image is convolved by the first convolutional module, the image data with a size of 320x250x32 is output, where 32 is the number of channels. Subsequently, the 32-channel image data is successively convolved into 64-channel image data, 128-channel image data, 256-channel image data, 512-channel image data, and 1024-channel image data, where the size of the 1024-channel image is 20x16x1024. Then, the image data with a size of 20x16 of the 1024-channel image is input into the upsampling module. The image data with a size of 20x16 is deconvolved by the first upsampling module to output image data with a size of 40x30x512, where 512 is the number of channels. Subsequently, the 512-channel image data is successively deconvolved into 256-channel image data, 128-channel image data, 64-channel image data, and 32-channel image data, where the image data with a size of 642x500 of the 32-channel image data (i.e., the image data of 642x500x32) is the image data output by the last upsampling module (i.e., the first feature map).

[0059] Exemplarily, generating a prediction mask according to the first feature map and the texture feature matrix (i.e., the texture feature matrix of the vascular OCT training image) includes: splicing the first feature map and the texture feature matrix to generate a second feature map; performing dimensionality reduction processing on the second feature map to generate a prediction mask. The above-mentioned texture feature matrix of the vascular OCT training image is a texture feature matrix obtained by extracting the texture features of the calcified plaque region in the vascular OCT training image using a texture feature statistical analysis method. Specifically, since the texture feature matrix of the above-mentioned K-dimensional vascular OCT training image is the same as the image size of 32 channels output by the U-net neural network (i.e., 642x500), in order to effectively utilize the texture feature matrix of the vascular OCT training image, the matrix of 642x500x32 (i.e., the first feature map) output by the last upsampling module is spliced with the OCT texture feature matrix (i.e., the texture feature matrix of the vascular OCT training image), as Figure 4 shown, to obtain a second feature map 401, and three 1×1 convolutional layers are used to perform dimensionality reduction processing on the second feature map 401. After the second feature map 401 is processed by the first 1x1 convolutional layer, a third feature map of 642x500x16 is output. After the third feature map is processed by the second 1x1 convolutional layer, a fourth feature map of 642x500x8 is output. After the fourth feature map is processed by the third 1x1 convolutional layer, a fifth feature map of 642x500 is output, and the fifth feature map corresponds Figure 4 to the output mask (i.e., the prediction mask) in

[0060] Exemplarily, training a convolutional neural network to be trained according to the above prediction mask, region of interest, and standard mask to generate a target convolutional neural network. According to Figure 2 (b) shown in the calcified region of interest map of the calcified region of interest and non-calcified region to construct the loss function (i.e., the Loss function) of the U-net model, and the Loss function is as follows:

[0061] Loss = |P mask -G mask |·*(ω*M ROI +ε)

[0062] where P mask represents the predicted calcified mask (i.e., the prediction mask), and the prediction mask is the predicted value. G mask represents the gold standard calcified mask (i.e., the standard mask), and the standard mask is the true value. M ROIIndicates the calcification region of interest map (i.e., the region of interest), ω represents the weight coefficient of the region of interest, usually ω>1, ε is the weight coefficient of the background region (non-calcification region), usually 0<ε<1. The above-mentioned gold standard calcification mask is a calcification mask generated after binarizing the vascular OCT training image in which the medical expert has marked the calcification plaque region. This calcification mask is a binary image of 0 and 1, where 0 represents the non-calcification plaque region and 1 represents the calcification plaque region marked by the expert. As can be seen from the above Loss function, this Loss function contains two terms. The first term is the loss function of the calcification region of interest: Loss = |P mask -G mask |·*(ω*M ROI ); The second term is the loss function of the non-calcification region of interest: Loss = |P mask -G mask |·*ε. During the process of training the U-net model, the first loss function is optimized with emphasis. The reason is that the weight coefficient of the first loss function is large (for example, ω>1). Therefore, the magnitude of the first loss function determines the change trend of the entire Loss function. For example, when the first loss function shows a decreasing trend, the Loss function also shows a decreasing trend. According to the Loss function, the network parameters of the convolutional neural network to be trained are continuously adjusted until the Loss function reaches a preset value, indicating that the convolutional neural network to be trained has been trained into the target convolutional neural network. During the process of training the convolutional neural network to be trained, the learning ability of the loss function of the convolutional neural network to be trained for the edge structure information of the calcification plaque region can be improved by using the calcification region of interest map. In addition, the recognition accuracy of the edge structure information of the calcification plaque region is improved by increasing the weight of the region of interest (i.e., the first loss function) in the loss function.

[0063] For example, as Figure 5 shown, where Figure (a) is the result of identifying the calcification plaque region only using the deep learning model without combining other aspect features, and Figure (b) is the result of the above technical solution provided by the present application for identifying the calcification plaque region. It can be seen from Figure (a) and Figure (b) that the range of the calcification plaque region identified by the technical solution provided by the present application is larger than the range of the calcification plaque region identified only using the deep learning model, indicating that the technical solution provided by the present application can identify some calcification plaque regions that cannot be identified only using the deep learning model. Therefore, the technical solution provided by the present application has higher accuracy in identifying the calcification plaque region.

[0064] S103. Determine the optical attenuation coefficient of the in-vivo OCT image. The optical attenuation coefficient of the in-vivo OCT image does not include the optical attenuation coefficient of the calcification plaque region.

[0065] Exemplarily, the above-mentioned target convolutional neural network is used to determine the calcified plaque region in the intracavitary OCT image, and the optical attenuation coefficient of the calcified plaque region is set to 0, so as to obtain an optical attenuation coefficient image after removing the calcification. Of course, the calcified plaque region can also be determined by medical experts using professional software for marking, or by using a neural network model to determine the calcified plaque region, etc. The present application does not make any limitation on the method for determining the calcified plaque region in the intracavitary OCT image.

[0066] For example, taking the intracavitary vascular tissue as an example, since the vascular OCT image collected by the OCT machine is a vascular OCT image in polar coordinates and the size of this vascular OCT image is 642x500, therefore, the optical attenuation model in polar coordinates can be used to calculate the optical attenuation coefficient of each pixel point of the vascular OCT image, and the value of each pixel point in the vascular OCT image is replaced by the corresponding optical attenuation coefficient value, so as to obtain the optical attenuation coefficient image corresponding to the vascular OCT image in polar coordinates, and the size of this optical attenuation coefficient image is 642x500.

[0067] The above-mentioned target convolutional neural network is used to identify the calcified plaque region in the vascular OCT image, and then the pixel value (i.e., the optical attenuation coefficient value) of the calcified plaque region in the optical attenuation coefficient image corresponding to this vascular OCT image is set to 0, so as to obtain an optical attenuation coefficient image after removing the calcified plaque region. Since each single optical attenuation coefficient image has 500 A-lines, and there are 642 optical attenuation coefficient values on each A-line, the maximum optical attenuation coefficient value on each A-line is calculated. There are 500 maximum optical attenuation coefficient values for the 500 A-lines, and these 500 maximum optical attenuation coefficient values form a 1×500 maximum optical attenuation coefficient vector, that is, a single optical attenuation coefficient image can obtain a 1×500 maximum optical attenuation coefficient vector. If a group of OCT pullback data contains 300 temporally adjacent vascular OCT images, then 300 optical attenuation coefficient images will be obtained, and further 300 1×500 maximum optical attenuation coefficient vectors will be obtained, and these 300 1×500 maximum optical attenuation coefficient vectors form a 300×500 maximum optical attenuation coefficient matrix.

[0068] Exemplarily, the plaque attenuation index (IPA) is the proportion of the optical attenuation coefficient values greater than the threshold x. Among them, the optical attenuation coefficient represents the attenuation degree of light by different tissues during the OCT imaging process. It can be seen from the above analysis that each row of data in the maximum optical attenuation coefficient matrix is a 1×500 maximum optical attenuation coefficient vector (that is, each row of data represents an optical attenuation coefficient image). Among them, this maximum optical attenuation coefficient vector has 500 elements (that is, 500 maximum optical attenuation coefficient values μ t) For the IPA value of a single optical attenuation coefficient image, it can be calculated using the following formula:

[0069]

[0070] Where N(μ t >x) means comparing these 500 elements with the threshold x respectively and counting the number of elements greater than the threshold x among these 500 elements. Since N total represents the total number of A-lines in the optical attenuation coefficient image, and according to the foregoing analysis, the 500 largest optical attenuation coefficient values μ t mean there are 500 A-lines. Therefore, N total takes the value of 500. For example, when N(μ t >x) is 400, N total takes the value of 500, and IPA is 800.

[0071] Exemplarily, since a single vascular OCT image collected by an OCT machine is 642×500 (i.e., the vascular OCT image histogram), as shown in Figure 6 (a), where 601 represents the indication line and 602 represents the calibration cursor. The above 500 means that a total of 500 A-lines are scanned when the catheter performs a 360° scan in the blood vessel, and the above 642 means that 642 pixel points are scanned on each A-line. The optical attenuation coefficient of each pixel point of this single vascular OCT image is calculated using the optical attenuation model to obtain the corresponding single optical attenuation coefficient image of this vascular OCT image. This single optical attenuation coefficient image is also 642×500 (i.e., the optical attenuation coefficient image histogram), where 500 means that a total of 500 A-lines are scanned when the catheter performs a 360° scan in the blood vessel, and 642 means that 642 pixel points are scanned on each A-line. The vascular OCT image histogram is converted into a vascular OCT image circular diagram. Specifically, since a total of 500 A-lines are scanned when the catheter performs a 360° scan in the blood vessel, and one A-line is scanned every 0.72° (i.e., 360° divided by 500 equals 0.72°), then these 500 A-lines are arranged at equal intervals of 0.72° in a circular shape to obtain the vascular OCT image circular diagram; the method of converting the optical attenuation coefficient image histogram into the optical attenuation coefficient image circular diagram is the same as the method of converting the vascular OCT image histogram into the vascular OCT image circular diagram, which will not be elaborated here. This optical attenuation coefficient image circular diagram is shown in Figure 6 (b), where 603 represents the blood vessel wall.

[0072] In order to better display the vascular OCT image circle and the corresponding light attenuation coefficient image circle on the software interface for easy observation. A bilinear interpolation algorithm is now used to perform linear interpolation on each pixel point on the vascular OCT image circle. Specifically, the vascular OCT image circle is linearly interpolated in the x and y directions using the surrounding 4 neighborhood pixels to obtain the interpolated vascular OCT image circle. The method of linearly interpolating the light attenuation coefficient image circle corresponding to the vascular OCT image circle is similar to the method of linearly interpolating the vascular OCT image circle, which will not be repeated here. After linear interpolation, the interpolated vascular OCT image circle and the corresponding interpolated light attenuation coefficient image circle are obtained, as shown in Figure 6 (b) and Figure 7 (b) as shown.

[0073] For example, Figure 6 As shown in the figure, when the calcified plaque area in the vascular OCT image is not removed, the IPA value is directly calculated based on the light attenuation coefficient image corresponding to the vascular OCT image. At this time, the IPA value (i.e., IPA=136) is very high. Due to the presence of the calcified plaque area, the light attenuation coefficient of the calcified plaque area in the vascular OCT image is very high, resulting in a high IPA calculation result. However, the IPA calculation result at this time cannot indicate that there is TCFA in the blood vessel. Therefore, the calcified plaque area in the vascular OCT image must be removed before an accurate IPA value can be calculated.

[0074] S104, determining the IPA of the intracavity OCT image according to the light attenuation coefficient of the intracavity OCT image.

[0075] For example, taking the intraluminal vascular tissue as an example, Figure 7 As shown, the above-mentioned target convolutional neural network is used to identify the calcified plaque area in the vascular OCT image, and then the pixel value of the calcified plaque area in the vascular OCT image is set to 0, so that the pixel value (i.e., the light attenuation coefficient value) corresponding to the calcified plaque area in the light attenuation coefficient image is 0, so as to obtain the light attenuation coefficient image after removing the calcified plaque area. The IPA value of the light attenuation coefficient image after removing the calcified plaque area is calculated, and it is found that the IPA value of the light attenuation coefficient image (i.e., IPA = 20) is very low. It can be seen that by comparing the vascular OCT image before and after removing the calcified plaque area ( Figure 6 (b) Light attenuation coefficient diagram ( Figure 6 (b) The corresponding IPA value and the light attenuation coefficient after removing the calcified plaque area ( Figure 7It is found that the IPA value corresponding to the optical attenuation coefficient map after removing the calcified plaque area is lower. The reason is that the optical attenuation coefficient of the calcified area in the optical attenuation coefficient map after removing the calcified plaque area is lower, and there is no interference from the calcified plaque area in the vascular OCT image on the IPA calculation result, thereby improving the accuracy of IPA for TCFA recognition.

[0076] Figure 8 The schematic structural diagram of a device for calculating the IPA of an intracavity OCT image provided by the present application is shown. Figure 8 The dashed line in indicates that the unit or the module is optional. The device 800 can be used to implement the method described in the above method embodiments. The device 800 can be a terminal device, a server, or a chip.

[0077] The device 800 includes one or more processors 801, and the one or more processors 801 can support the device 800 to implement Figure 1 the method in the corresponding method embodiment. The processor 801 can be a general-purpose processor or a special-purpose processor. For example, the processor 801 can be a central processing unit (CPU). The CPU can be used to control the device 800, execute software programs, and process the data of the software programs. The device 800 can also include a communication unit 805 for implementing signal input (reception) and output (transmission).

[0078] For example, the device 800 can be a chip, and the communication unit 805 can be the input and / or output circuit of the chip, or the communication unit 805 can be the communication interface of the chip, and the chip can be a component of the terminal device.

[0079] Again, for example, the device 800 can be a terminal device, and the communication unit 805 can be the transceiver of the terminal device, or the communication unit 805 can be the transceiver circuit of the terminal device.

[0080] The device 800 may include one or more memories 802, on which there is a program 804. The program 804 can be run by the processor 801 to generate an instruction 803, so that the processor 801 executes the method described in the above method embodiment according to the instruction 803. Optionally, data (such as the ID of the chip to be tested) can also be stored in the memory 802. Optionally, the processor 801 can also read the data stored in the memory 802. The data can be stored at the same storage address as the program 804, or the data can be stored at a different storage address from the program 804.

[0081] The processor 801 and the memory 802 can be set separately or integrated together. For example, they can be integrated on a system on chip (SOC) of a terminal device.

[0082] For the specific manner in which the processor 801 executes the method for IPA of OCT images in the calculation cavity, reference can be made to the relevant descriptions in the method embodiments.

[0083] It should be understood that the steps of the above method embodiments can be completed by a logic circuit in hardware form or instructions in software form in the processor 801. The processor 801 can be a CPU, a digital signal processor (DSP), a field programmable gate array (FPGA), or other programmable logic devices. For example, discrete gates, transistor logic devices, or discrete hardware components.

[0084] The present application also provides a computer program product, which, when executed by the processor 801, implements the method described in any one of the method embodiments of the present application.

[0085] This computer program product can be stored in the memory 802, for example, it is the program 804. After processes such as preprocessing, compilation, assembly, and linking, the program 804 is finally converted into an executable target file that can be executed by the processor 801.

[0086] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computer, it implements the method described in any one of the method embodiments of the present application. This computer program can be a high-level language program or an executable target program.

[0087] The computer-readable storage medium is, for example, the memory 802. The memory 802 can be a volatile memory or a non-volatile memory, or the memory 802 can include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0088] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes and the technical effects generated by the above-described devices and apparatuses can refer to the corresponding processes and technical effects in the foregoing method embodiments, and will not be elaborated herein again.

[0089] In several embodiments provided in this application, the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, some features of the above-described method embodiments can be ignored or not executed. The device embodiments described above are merely illustrative. The division of units is only a logical function division, and there can be other division methods in actual implementation. Multiple units or components can be combined or integrated into another system. In addition, the coupling between units or the coupling between each component can be a direct coupling or an indirect coupling. The above couplings include electrical, mechanical, or other forms of connection.

[0090] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A method for calculating the IPA of an intracavity OCT image, characterized in that, the method includes: Obtain an intracavity OCT image; Determine the calcified plaque area of the intracavity OCT image; Determine the light attenuation coefficient of the intracavity OCT image, where the light attenuation coefficient of the intracavity OCT image does not include the light attenuation coefficient of the calcified plaque area; Determine the IPA of the intracavity OCT image according to the light attenuation coefficient of the intracavity OCT image.

2. The method according to claim 1, characterized in that, the determining the calcified plaque area of the intracavity OCT image includes: Process the intracavity OCT image through a target convolutional neural network to determine the calcified plaque area of the intracavity OCT image.

3. The method according to claim 2, characterized in that, the target convolutional neural network is trained by the following method: Process an intracavity OCT training image through a convolutional neural network to be trained to generate a first feature map; Obtain the texture feature matrix of the calcified plaque area in the intracavity OCT training image; Generate a prediction mask according to the first feature map and the texture feature matrix; Obtain the region of interest of the intracavity OCT training image, where the region of interest is used to characterize the calcified plaque area in the intracavity OCT training image; Train the convolutional neural network to be trained according to the prediction mask, the region of interest and a standard mask to generate the target convolutional neural network, where the prediction mask is a predicted value, the standard mask is a true value, and the region of interest is used to improve the learning ability of the loss function of the convolutional neural network to be trained for the edge structure information of the calcified plaque area.

4. The method according to claim 3, characterized in that, the obtaining the region of interest of the intracavity OCT training image includes: Obtain multiple A-lines of the intracavity OCT training image; Determine the region of interest of the intracavity OCT training image according to the pixel points corresponding to the maximum light attenuation coefficient on each of the multiple A-lines.

5. The method according to claim 3 or 4, characterized in that, generating a prediction mask according to the first feature map and the texture feature matrix includes: Stitch the first feature map and the texture feature matrix to generate a second feature map; Perform dimensionality reduction processing on the second feature map to generate the prediction mask.

6. The method according to claim 5, characterized in that, the performing dimensionality reduction processing on the second feature map includes: Perform dimensionality reduction processing on the second feature map through 3 1×1 convolutional layers.

7. The method according to claim 3 or 4, characterized in that, the obtaining the texture feature matrix of the calcified plaque area in the intracavity OCT training image includes: Determine the spatial gray level co-occurrence matrix of the intracavity OCT training image; Determine at least one texture feature of the intracavity OCT training image according to the spatial gray level co-occurrence matrix; Determine the texture feature matrix according to the texture feature.

8. The method according to claim 7, characterized in that, the at least one texture feature includes: One or more of energy, inertia, entropy, and correlation.

9. An apparatus for calculating the IPA of an in-vivo OCT image, characterized in that, the apparatus includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the apparatus executes the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 8.

Images