Vessel plaque detection apparatus and method based on segmentation network

Abstract

The application relates to a blood vessel plaque detection device and method based on a segmentation network, which comprises a preprocessing module for preprocessing a blood vessel image; a feature extraction module comprising an encoder and a decoder, the encoder being used for down-sampling the preprocessed blood vessel image, and the decoder being used for up-sampling the down-sampled blood vessel image, so as to make up for the information lost in the down-sampling process; and a classification network module for outputting a result of whether the blood vessel image has a plaque based on the up-sampled blood vessel image. According to the scheme of the application, 3D convolution kernels are used in the segmentation network, so that the inter-layer information can be well learned; the up-sampling makes up for the information lost in the down-sampling process, which is conducive to reducing the probability of plaque missed detection; there is a cross-layer link between each layer of down-sampling and the corresponding layer of up-sampling, so that the blood vessel can be more accurately detected; in addition, a segmentation branch is added in the process of training the segmentation network, so that the detection precision is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image analysis, and in particular to a device and method for detecting a blood vessel plaque based on a segmentation network. BACKGROUND

[0002] Atherosclerotic plaque is the pathophysiological basis of the occurrence and development of coronary heart disease. At present, cardiac CTA (CT Angiography) images have been widely used in the screening of blood vessel plaques. Various data show that plaques of different properties and different sizes have a large difference in their impact on the human body. To determine the size and other quantitative indicators of plaque properties, it is necessary to first accurately detect the plaque. Moreover, early detection of the presence of plaque can lead to timely treatment, thereby effectively controlling the deterioration of the disease. Therefore, it is of great significance to accurately and effectively detect plaques from CTA images.

[0003] At present, most blood vessel plaque detection algorithms use a classification network-based framework, for example, using a 2D classification convolutional neural network to classify each layer of the straightened blood vessel, or using a 2D convolutional neural network plus LSTM (Long Short-Term Memory) to classify each layer. SUMMARY

[0004] The above-mentioned prior art method has the following problems:

[0005] 1. The 2D classification convolutional neural network uses a 2D convolution kernel and only extracts features in the xy direction of a blood vessel layer, without acting on the z direction. Therefore, it cannot well extract inter-layer information. Even if an LSTM network is added, it can only supplement inter-layer information in a high dimension, and low-dimensional information cannot be fully utilized.

[0006] 2. The 3D classification convolutional neural network uses a 3D convolution kernel and can extract features in the x, y, and z directions simultaneously. Although it can better extract inter-layer information, there is downsampling in the z direction. Short plaques may be lost in the downsampling process, resulting in missed detection of short plaques.

[0007] In view of the above problems, the present application provides a blood vessel plaque detection scheme based on a segmentation network, which can improve the problem of missed detection of plaques, especially short plaques.

[0008] According to a first aspect of the present application, a blood vessel plaque detection device based on a segmentation network is provided, comprising:

[0009] a preprocessing module configured to preprocess a blood vessel image, the blood vessel image comprising n layers, wherein n is greater than or equal to 1;

[0010] a feature extraction module comprising an encoder configured to down-sample the pre-processed vessel image and a decoder configured to up-sample the down-sampled vessel image; and

[0011] a classification network module configured to output results of whether the n layers of vessel images have plaques based on the up-sampled vessel image.

[0012] According to a second aspect of the present application, a method for detecting vessel plaques based on a segmentation network is provided, comprising:

[0013] pre-processing a vessel image by a pre-processing module, the vessel image comprising n layers, wherein n is greater than or equal to 1;

[0014] down-sampling the pre-processed vessel image by an encoder of a feature extraction module,

[0015] up-sampling the down-sampled vessel image by a decoder of the feature extraction module; and

[0016] outputting results of whether the n layers of vessel images have plaques based on the up-sampled vessel image by a classification network module.

[0017] According to a third aspect of the present application, an electronic device is provided, comprising:

[0018] a processor; and

[0019] a memory storing computer instructions, when the computer instructions are executed by the processor, causing the processor to perform the method of the second aspect.

[0020] According to a fourth aspect of the present application, a non-transitory computer storage medium storing a computer program is provided, when the computer program is executed by a plurality of processors, causing the processors to perform the method of the second aspect.

[0021] According to the blood vessel plaque detection scheme based on the segmentation network provided in the application, a 3D convolution kernel is used in the segmentation network, and feature extraction is performed in x, y and z directions, so that the interlayer information can be well learned. After down-sampling, the original image size is restored through up-sampling, the information lost in the down-sampling process is compensated through up-sampling, and the probability of plaque missed detection is reduced. In addition, there is a cross-layer link between each layer of down-sampling and the corresponding layer of up-sampling. The cross-layer link can further integrate the information between the corresponding layers of up-sampling and down-sampling, more accurately detect the blood vessels, and further reduce the probability of plaque missed detection. In addition, during the training of the segmentation network, a segmentation branch can be added, and the mask image information (mask information) of the lesion can be used to help the network better learn and locate the lesion area during the training process, so as to improve the detection accuracy of the blood vessel plaque detection device based on the segmentation network. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without departing from the scope of the application.

[0023] Figure 1 is a schematic diagram of a blood vessel plaque detection device based on a segmentation network according to the first embodiment of the application.

[0024] Figure 2 is a schematic diagram of a blood vessel plaque detection device based on a segmentation network according to the second embodiment of the application.

[0025] Figure 3 is a schematic diagram of a blood vessel plaque detection device based on a segmentation network according to the third embodiment of the application.

[0026] Figure 4 is a training schematic diagram of a blood vessel plaque detection device based on a segmentation network according to the embodiment of the application.

[0027] Figure 5 is another training schematic diagram of a blood vessel plaque detection device based on a segmentation network according to the embodiment of the application.

[0028] Figure 6 is a flowchart of a blood vessel plaque detection method based on a segmentation network according to the embodiment of the application.

[0029] Figure 7 is a structural diagram of an electronic device provided by the application. DETAILED DESCRIPTION

[0030] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts are within the scope of the present application.

[0031] The terms “first”, “second”, “third”, and “fourth” and the like in the description and claims of the present application and the drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or device.

[0032] Reference herein to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood that the embodiments described herein can be combined with other embodiments.

[0033] In the present application, the term “vessel” includes any one of a coronary artery, a carotid artery, an abdominal aorta, a cerebral vessel, an ocular vessel, and a femoral artery. The “z-direction” of the vessel is the longitudinal direction of the columnar vessel, and the “xy-direction” is the cross-sectional direction of the vessel perpendicular or substantially perpendicular to the z-direction.

[0034] Figure 1 is a schematic diagram of a blood vessel plaque detection device based on a segmentation network according to the first embodiment of the present application. As shown in Figure 1 The device includes a preprocessing module, a feature extraction module, and a classification network module.

[0035] The blood vessel plaque detection device based on a segmentation network provided by the present application can be applied to, for example, a coronary straightening image based on a cardiac CTA image. On the straightened blood vessel, a fixed-size section of the blood vessel is sequentially taken from the starting position as the origin on the point on the blood vessel center line with a certain step size and fed into the detection network. The network outputs a prediction result of the same length, and the overlapping part can use the mean value or maximum value method.

[0036] The image of a blood vessel is input into the blood vessel plaque detection device, and after detection by the blood vessel plaque detection device, whether there is plaque in the blood vessel can be obtained. The blood vessel image input into the blood vessel plaque detection device includes n layers, where n is greater than or equal to 1, and after detection by the blood vessel plaque detection device, the results of whether there is plaque in each of the n layers can be output.

[0037] In Figure 1 , the preprocessing module is used for preprocessing the blood vessel image, wherein the preprocessing includes window frame and window level adjustment and mean variance normalization, and other preprocessing methods can also be used as needed.

[0038] In Figure 1 , the feature extraction module includes an encoder and a decoder, the encoder performs down-sampling on the pre-processed blood vessel image, and the decoder performs up-sampling on the down-sampled blood vessel image. After down-sampling, it is back to the original image size through up-sampling. For example, the image input into the blood vessel plaque detection device is 64*64*n (i.e. the pixels of each layer in the n-layer image are 64*64), and the result output by the decoder is 64*64*n*m (where m is the dimension of the decoder, which can be any reasonable value, such as 2048). In this way, the information lost in the down-sampling process is compensated by up-sampling, which helps to reduce the probability of plaque missed detection.

[0039] In Figure 1 , the classification network module outputs the results of whether there is plaque in the n layers of blood vessel images based on the up-sampled blood vessel images. In one specific embodiment, the classification network module includes a global maximum value acquisition unit and a fully connected layer unit, wherein the global maximum value acquisition unit is used to take the global maximum value in each layer of the n layers of blood vessel images, and the fully connected layer unit calculates and outputs the results of whether there is plaque in the n layers based on the results output by the global maximum value acquisition unit. For example, the global maximum value acquisition unit of the classification network module takes the global maximum value of the 64*64 values in each of the n layers of images output by the decoder, obtaining a global maximum value. In this way, the global maximum value acquisition unit obtains 1*n*m values, which are then input into the fully connected layer unit. After calculation, the fully connected layer unit can obtain the probability of whether there is plaque in each of the n layers of blood vessels. A threshold value can be set, for example, the threshold value can be 0.5. When the probability of plaque in the blood vessel of this layer calculated by the fully connected layer unit is greater than 0.5, it is determined that there is plaque, otherwise it is determined that there is no plaque.

[0040] Figure 2 is a schematic diagram of a blood vessel plaque detection device based on a segmentation network according to the second embodiment of the present application. As Figure 2 shown, the encoder and decoder of the feature extraction module each have one or more convolutional blocks Figure 2As shown, there are 4 convolutional blocks, and the encoder and decoder contain the same number of convolutional blocks. Each convolutional block of the encoder and decoder includes one or more 3D convolutional layers. The convolutional blocks contained in the encoder and decoder can be exactly the same, partially the same, or completely different. This application does not require the form and setting of the convolutional blocks of the encoder and decoder.

[0041] like Figure 2 As shown, each convolutional block of the encoder in the feature extraction module downsamples the input image data by a factor of 2, and each convolutional block of the decoder in the feature extraction module upsamples the downsampled image data by a factor of 2. The image is then upsampled back to its original size after downsampling. The upsampling and downsampling factors can also be other suitable values, such as 4x or 8x, and this application does not impose any restrictions on them.

[0042] Figure 3 This is a schematic diagram of a vascular plaque detection device based on a segmentation network according to a third embodiment of this application. Figure 3 As shown, there is a cross-layer link between a convolutional block of the encoder and the corresponding convolutional block of the decoder in the feature extraction module. This allows the decoder to perform upsampling on the blood vessel image by one convolutional block, and then jointly process the upsampled result with the unsampled data from the encoder's corresponding convolutional block to obtain a joint processing result. This joint processing is performed after upsampling in one convolutional layer of the decoder. The joint processing result will then be passed to the next convolutional block of the decoder, or, if there is no next convolutional block, it will be output as the final downsampled result of the decoder.

[0043] For example, such as Figure 3 As shown, after downsampling by the last convolutional block of the encoder, the 64 pixels of the n-layer image are reduced to 32 pixels and output to the decoder. After upsampling by the first convolutional block of the decoder, the 32 pixels are increased to 64 pixels. Then, the values ​​of the 64 pixels after upsampling by the first convolutional block of the decoder are combined with the values ​​of the 64 pixels before downsampling by the corresponding convolutional block (i.e., the last convolutional block) of the encoder to obtain the combined processing result.

[0044] pass Figure 3 The cross-layer links shown indicate that there are cross-layer links between each downsampling layer and the corresponding upsampling layer. These cross-layer links can further integrate the information between the corresponding upsampling and downsampling layers, enabling more accurate detection of blood vessels and helping to further reduce the probability of missed plaque detection.

[0045] Figure 4 This is a training schematic diagram of a segmentation network-based vascular plaque detection device according to an embodiment of this application. Figure 4As shown, the blood vessel plaque detection device includes a preprocessing module, a feature extraction module, and a classification network module. The full connection layer unit of the classification network module outputs the probability of the n layers of blood vessel images having or not having plaques. The loss of the output of the full connection layer unit is calculated and recorded as a first loss. In one specific embodiment, cross-entropy loss can be used for optimization during training, and the loss is recorded as L 分类 = -(q*logp + (1-q)*log(1-p)), where q is the sample label, indicating the samples labeled as "0" or "1" in the training samples (where "1" indicates having plaques and "0" indicates not having plaques), and p is the probability of the network predicting the sample as having plaques (label "1").

[0046] Figure 4 The dashed line indicates the overall optimization training process of the blood vessel plaque detection device. According to the output of the classification network, the loss L 分类 of the classification network is calculated. 分类 The derivative is taken, the gradient is back-propagated, and the model parameters, including the related parameters in the feature extraction module and the classification network module, are updated. The updated model parameters include the weights of the convolution kernels and the weights of the full connection layer, etc. Then the above process is repeated until the model is stable.

[0047] Figure 5 is another training schematic diagram of the blood vessel plaque detection device based on the segmentation network according to the embodiments of the present application. As shown, Figure 5 the blood vessel plaque detection device includes a preprocessing module, a feature extraction module, a classification network module, a segmentation network module, and a total loss calculation module. The segmentation network module is used in the training process to form a blood vessel plaque detection device with a multi-branch segmentation network.

[0048] In Figure 5 , the full connection layer unit of the classification network module outputs the probability of the n layers of blood vessel images having or not having plaques. The loss of the output of the full connection layer unit is calculated. In one embodiment, cross-entropy loss can be used for optimization during training, and the loss is recorded as L 分类 = -(q*logp + (1-q)*log(1-p)), where q is the sample label, indicating the samples labeled as "0" or "1" in the training samples (where "1" indicates having plaques and "0" indicates not having plaques), and p is the probability of the network predicting the sample as having plaques (label "1").

[0049] In Figure 5 , the segmentation network module includes a convolution layer and an activation function layer (such as a Sigmoid function layer) to output the segmentation result of the plaque region (such as Figure 5As shown, the white part is the plaque area, and the black part is the non-plaque area. During the training process, the segmentation network module uses the labeled mask image information (mask information) to process the up-sampling result output by the decoder to obtain the probability of each pixel in the n-layer image having or not having a plaque, and then calculates the loss based on the predicted probability of each pixel in the n-layer image having or not having a plaque and the labeled mask image information, denoted as a second loss.

[0050] In one specific embodiment, the dice loss is used for optimization during training, and the segmentation loss is denoted as L 分割 = 2 * sum (c * gt) / (sum (c) + sum (gt)), where c is the probability of each pixel in the n-layer image predicted by the segmentation network having or not having a plaque, and gt is the actual labeled mask information. The dashed line in the figure represents the overall optimization training process of the network. According to the outputs of the segmentation network and the classification network, the respective losses L 分割 and L 分类 are obtained in the above manner. The total loss calculation module calculates the overall loss of the network L = γL 分类 + βL 分割 , where γ and β are the weights of the two losses, and in one embodiment, γ = β = 1, which can be adjusted according to the actual situation. Then, L is derived, the gradient is returned, and the model parameters are updated, including the related parameters in the feature extraction module, the classification network module, and the segmentation network module. The updated model parameters include the weights of the convolution kernel and the weights of the fully connected layer. Then, the above process is repeated until the model is stable.

[0051] In Figure 5 the embodiment shown, a segmentation branch is added during the training of the network, and strong labeled lesion mask image information is used during the training process to help the network better learn and locate the lesion area, thereby improving the detection accuracy of the blood vessel plaque detection device based on the segmentation network.

[0052] In the above embodiments, the algorithms and calculation formulas for calculating the classification loss and the segmentation loss are described respectively. However, it should be noted that this is only one specific implementation to facilitate the explanation of the scheme of the present application. Those skilled in the art can also use other algorithms and formulas to calculate the classification loss and the segmentation loss, and the present application does not limit this.

[0053] Based on Figures 1 to 5 the blood vessel plaque detection device based on the segmentation network shown, the present application also provides a blood vessel plaque detection method based on the segmentation network. As Figure 6 shown, the method includes the following steps.

[0054] Step S601, pre-processing the blood vessel image by the pre-processing module.

[0055] The blood vessel image input into the blood vessel plaque detection device includes n layers, where n is greater than or equal to 1. The preprocessing module is configured to preprocess the blood vessel image, where the preprocessing includes window frame and window level adjustment and mean-variance normalization, and other preprocessing methods can also be used as needed.

[0056] At step S602, the preprocessed blood vessel image is down-sampled by an encoder of the feature extraction module.

[0057] At step S603, the down-sampled blood vessel image is up-sampled by a decoder of the feature extraction module.

[0058] The feature extraction module includes an encoder and a decoder. The encoder down-samples the preprocessed blood vessel image, and the decoder up-samples the down-sampled blood vessel image. After down-sampling, the blood vessel image is up-sampled to the original size. In this way, the up-sampling compensates for the information lost during down-sampling, which helps to reduce the probability of plaque missed detection.

[0059] In one embodiment, the encoder and the decoder of the feature extraction module each have one or more convolution blocks, and the number of convolution blocks included in the encoder and the decoder is equal. Each convolution block of the encoder and the decoder includes one or more 3D convolution layers. The convolution blocks included in the encoder and the decoder can be completely the same, partially the same, or completely different. The present application does not require the form and setting of the convolution blocks of the encoder and the decoder.

[0060] In one embodiment, the convolution block of the encoder of the feature extraction module and the corresponding convolution block of the decoder have a cross-layer link. After one convolution block of the decoder up-samples the blood vessel image, the decoder jointly processes the result of the up-sampling of the convolution block and the data before the corresponding convolution block of the encoder down-samples, to obtain a joint processing result. The joint processing is performed after the up-sampling of one convolution block of the decoder. The joint processing result enters the next convolution block of the decoder, or is output as the final down-sampling result of the decoder when there is no next convolution block of the decoder.

[0061] In this way, step S603 can include: after one convolution block of the decoder up-samples the blood vessel image, the decoder jointly processes the result of the up-sampling of the convolution block and the data before the corresponding convolution block of the encoder down-samples, to obtain a joint processing result.

[0062] At step S604, the classification network module outputs a result of whether the blood vessel image has a plaque based on the up-sampled blood vessel image.

[0063] In one embodiment, the classification network module comprises a global maximum value obtaining unit and a full connection layer unit, wherein the global maximum value obtaining unit is configured to obtain a global maximum value in each layer of the n layers of the blood vessel image, and the full connection layer unit is configured to calculate and output the result of whether the n layers have plaques based on the result output by the global maximum value obtaining unit. For example, the global maximum value obtaining unit of the classification network module obtains a global maximum value from 64*64 numerical values in each layer of the n layers of the image output by the decoder, so that the global maximum value obtaining unit obtains 1*n*m numerical values, which are then input into the full connection layer unit. After calculation, the full connection layer unit can obtain the probability of whether the n layers of the blood vessel have plaques. A threshold value can be set, for example, the threshold value can be 0.5. When the probability of whether the blood vessel has plaques in the layer calculated by the full connection layer unit is greater than 0.5, it is determined that there is a plaque, otherwise it is determined that there is no plaque.

[0064] Thus, step S604 comprises:

[0065] Sub-step S6041, the global maximum value obtaining unit is configured to obtain a global maximum value in each layer of the n layers of the blood vessel image; and

[0066] Sub-step S6042, the full connection layer unit is configured to calculate and output the result of whether the n layers have plaques based on the result output by the global maximum value obtaining unit.

[0067] Before testing the blood vessel plaque detection device based on the segmentation network, the neural network needs to be trained. Figure 6 The method also comprises:

[0068] Step S605, the neural network is trained based on the feature extraction module and the classification network module.

[0069] In one embodiment, step S605 comprises:

[0070] Sub-step S6051, the classification network module is configured to calculate a first loss based on the sample label and the probability of whether the n layers have plaques predicted by the classification network module.

[0071] The full connection layer unit of the classification network module outputs the probability of whether the n layers of the blood vessel image have plaques, and calculates the loss of the result output by the full connection layer unit, which is recorded as a first loss. In one embodiment, the cross-entropy loss can be used for optimization during training, and the loss is recorded as L 分类= -(q*logp + (1-q)*log(1-p)), where q is the sample label, representing the sample labeled as "0" or "1" in the training sample (where "1" represents the presence of plaque, and "0" represents the absence of plaque), and p is the probability of the network predicting that the sample has plaque (label "1").

[0072] According to the output of the classification network, the loss L of the classification network is calculated 分类 , and L 分类 is derived, gradient backpropagation is performed, and the model parameters, including the relevant parameters in the feature extraction module and the classification network module, are updated; the updated model parameters include the weights of the convolution kernel and the weights of the fully connected layer, etc.; then the above process is repeated until the model is stable.

[0073] In one embodiment, as shown in Figure 5 , the segmentation network module is used in the training process. Thus, step S605 further includes:

[0074] Sub-step S6052, processing the up-sampling result output by the decoder through the segmentation network module to obtain the probability of each pixel in the n-layer image having or not having plaque; and

[0075] Sub-step S6053, calculating a second loss based on the predicted probability of each pixel in the n-layer image having or not having plaque and the labeled mask image information through the segmentation network module.

[0076] The segmentation network module includes a convolution layer and an activation function layer (such as a Sigmoid function layer), and outputs the segmentation result of the plaque region (as shown in Figure 5 , the white part is the plaque region, and the black part is the non-plaque region). In the training process, the segmentation network module uses the labeled mask image information (mask information) to process the up-sampling result output by the decoder to obtain the probability of each pixel in the n-layer image having or not having plaque, and then calculates the loss based on the predicted probability of each pixel in the n-layer image having or not having plaque and the labeled mask image information, denoted as the second loss.

[0077] In one specific embodiment, the dice loss is used for optimization during training, and the segmentation loss is denoted as L 分割 = 2*sum(c*gt) / (sum(c)+sum(gt)), where c is the predicted probability of each pixel in the n-layer image of the segmentation network having or not having plaque, and gt is the actual labeled mask information.

[0078] After obtaining the first loss and the second loss, the total loss can be calculated, and the model parameters are updated based on the total loss. Thus, step S605 further includes:

[0079] Sub-step S6054: The total loss is calculated by the total loss calculation module based on the first loss and the second loss.

[0080] Based on the outputs of the segmentation network and the classification network, their respective losses L are obtained in the manner described above. 分割 and L 分类 The total loss calculation module calculates the overall network loss L = γL. 分类 +βL 分割 , where γ and β are the weights of the two parts of the loss. In one embodiment, γ = β = 1, which can be adjusted according to the actual situation. Then, the derivative of L is taken, gradient backpropagation is performed, and the model parameters are updated, including the relevant parameters in the feature extraction module, classification network module and segmentation network module. The updated model parameters include the weights of the convolutional kernels and the weights of the fully connected layers, etc. Then the above process is repeated until the model is stable.

[0081] According to the segmentation network-based vascular plaque detection scheme provided in this application, a 3D convolutional kernel is used in the segmentation network, and features are extracted simultaneously in the x, y, and z directions, which can effectively learn inter-layer information. After downsampling, upsampling is performed to return the image to its original size. Upsampling compensates for the information lost during downsampling, which helps reduce the probability of missed plaque detection. In addition, there are cross-layer links between each downsampling layer and the corresponding upsampling layer. These cross-layer links can further integrate the information between the corresponding upsampling and downsampling layers, resulting in more accurate vascular detection and further reducing the probability of missed plaque detection. Furthermore, during the training of the segmentation network, segmentation branches can be added, and strongly labeled lesion mask image information can be used during training to help the network better learn and locate lesion regions, thereby improving the detection accuracy of the segmentation network-based vascular plaque detection device.

[0082] See Figure 7 , Figure 7 An electronic device is provided, including a processor; and a memory storing computer instructions that, when executed by the processor, cause the processor to perform the computer instructions as follows: Figure 6 The method and its detailed scheme are shown.

[0083] It should be understood that the above-described device embodiments are merely illustrative, and the device disclosed in this application can be implemented in other ways. For example, the division of units / modules described in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, integrated into another system, or some features may be ignored or not executed.

[0084] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0085] If the integrated unit / module is implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor or chip can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the on-chip cache, off-chip memory, and storage can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0086] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0087] This application embodiment also provides a non-transitory computer storage medium storing a computer program, which, when executed by multiple processors, causes the processors to perform actions such as... Figure 6 The method and its detailed scheme are shown.

[0088] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0089] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0091] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0093] If the integrated unit is realized in the form of a software program module and sold or used as an independent product, it can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of algorithms for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0094] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing related hardware, and the program can be stored in a computer readable memory, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM) or an optical disk, etc.

[0095] The embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, the changes or deformations made by the person of ordinary skill in the art according to the idea of the present application, based on the specific implementation manners and application scope of the present application, all belong to the scope of protection of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A vascular plaque detection device based on a segmentation network, comprising: The preprocessing module is used to preprocess the straightened blood vessel image, which includes n layers, where n is greater than 1, and each of the n layers corresponds to a blood vessel cross-section. The feature extraction module includes an encoder and a decoder, wherein the encoder is used to downsample the preprocessed vascular image and the decoder is used to upsample the downsampled vascular image. A classification network module is used to output the presence or absence of plaques in the n layers of the blood vessel image based on the upsampled blood vessel image; wherein, both the feature extraction module and the classification network module are trained neural network modules, and the classification network module calculates a first loss L based on the sample labels and the probability of presence or absence of plaques in the n layers predicted by the classification network module. 分类 =-(q*logp+(1-q)*log(1-p)), where q represents the sample label, and p represents the probability of plaque presence or absence in the n layers predicted by the classification network module; wherein, the classification network module includes a global maximum value acquisition unit and a fully connected layer unit, wherein the global maximum value acquisition unit is used to take the global maximum value in each layer of the n-layer vascular image, and the fully connected layer unit calculates and outputs the results of plaque presence or absence in the n layers based on the output of the global maximum value acquisition unit; and The segmentation network module is used to process the upsampling results output by the decoder during the neural network training process of the feature extraction module and the classification network module, to obtain the probability of whether each pixel in the n-layer image has patches, and to calculate the second loss Lsegmentation = 2*sum(c*gt) / (sum(c)+sum(gt)) based on the predicted probability of whether each pixel in the n-layer image has patches and the labeled mask image information, where c represents the probability of whether each pixel in the n-layer image has patches predicted by the segmentation network module, and gt represents the labeled mask image information.

2. The vascular plaque detection device as described in claim 1, wherein, The preprocessing includes window frame and window position adjustment and mean and variance normalization.

3. The vascular plaque detection device as described in claim 1, wherein, Both the encoder and the decoder include one or more convolutional blocks, the number of convolutional blocks in the encoder and the decoder is equal, and the convolutional block includes one or more 3D convolutional layers.

4. The vascular plaque detection device as described in claim 3, wherein, There is a cross-layer link between a convolutional block of the encoder and the corresponding convolutional block of the decoder, so that after the convolutional block of the decoder performs upsampling processing on the blood vessel image, the decoder performs joint processing on the upsampled result of the convolutional block and the data before downsampling processing of the corresponding convolutional block of the encoder to obtain a joint processing result.

5. The vascular plaque detection device as described in claim 1, further comprising: The total loss calculation module is used to calculate the total loss based on the first loss and the second loss.

6. A method for detecting vascular plaques based on segmentation networks, comprising: The straightened blood vessel image is preprocessed by a preprocessing module. The blood vessel image includes n layers, where n is greater than 1, and each of the n layers corresponds to a blood vessel cross-section. The encoder in the feature extraction module downsamples the preprocessed blood vessel image. The downsampled blood vessel image is upsampled using the decoder of the feature extraction module; The classification network module outputs the results of whether or not there are plaques in the blood vessel images of the n layers based on the upsampled blood vessel images; as well as The neural network training for the feature extraction module and the classification network module includes: The first loss L is calculated by the classification network module based on the sample labels and the probability of the presence or absence of patches in the n layers predicted by the classification network module. 分类 =-(q*logp+(1-q)*log(1-p)), where q represents the sample label and p represents the probability of the n layers having or not having patches as predicted by the classification network module; The upsampling results output by the decoder are processed by the segmentation network module to obtain the probability of whether each pixel in the n-layer image has patches or not; and The segmentation network module calculates the second loss L based on the predicted probability of each pixel having or not having patches in the n layers of images and the labeled mask image information. 分割 =2*sum(c*gt) / (sum(c)+sum(gt)), where c represents the probability of each pixel in the n-layer image having or not having patches as predicted by the segmentation network module, and gt represents the labeled mask image information; The classification network module includes a global maximum value acquisition unit and a fully connected layer unit. The step of outputting the results of the presence or absence of plaques in the n layers of vascular images based on the upsampled vascular images through the classification network module includes: The global maximum value acquisition unit obtains the global maximum value in each of the n layers of the vascular image; and The results of obtaining the global maximum value from the fully connected layer unit are used to calculate and output the results of whether there are patches in the n layers.

7. The method for detecting vascular plaques as described in claim 6, wherein, The preprocessing includes window frame and window position adjustment and mean and variance normalization.

8. The method for detecting vascular plaques as described in claim 6, wherein, Both the encoder and the decoder include one or more convolutional blocks, the number of convolutional blocks in the encoder and the decoder is equal, and the convolutional block includes one or more 3D convolutional layers.

9. The method for detecting vascular plaques as described in claim 8, wherein, There is a cross-layer link between a convolutional block of the encoder and the corresponding convolutional block of the decoder. The upsampling of the downsampled blood vessel image by the decoder of the feature extraction module includes: After the blood vessel image is upsampled by a convolutional block of the decoder, the upsampled result of the convolutional block is combined with the data before downsampling by the corresponding convolutional block of the encoder to obtain a combined processing result.

10. The vascular plaque detection method as described in claim 6, wherein the neural network training of the feature extraction module and the classification network module further includes: The total loss is calculated by the total loss calculation module based on the first loss and the second loss.

11. An electronic device, comprising: processor; as well as A memory storing computer instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 6-10.

12. A non-transitory computer storage medium storing a computer program that, when executed by a plurality of processors, causes the processors to perform the method of any one of claims 6-10.

Images