A method, system and electronic device for ultrasonic quantification of carotid artery plaque

Abstract

The present invention discloses a method, system and electronic device for ultrasonic quantification of carotid plaques, and relates to the field of ultrasonic quantification of carotid plaques. The method for ultrasonic quantification of carotid plaques comprises: constructing a target ResNet network based on a circular convolution unit, and reconstructing an encoder and a decoder of a U‑Net network respectively through the target ResNet network and a plurality of residual decoding units to obtain a ResNet‑U‑Net combined network; constructing a quantification model through the ResNet‑U‑Net combined network and a CNN network; constructing a training set including a plurality of training samples; the training samples include a carotid ultrasound image and a corresponding pathological image thereof; the labels of the training samples are quantification information of plaques in the corresponding pathological image; obtaining a carotid plaque ultrasonic quantification model through training the quantification model with the training set; obtaining a carotid ultrasound image to be predicted and inputting it into the carotid plaque ultrasonic quantification model, and outputting the quantification information of the plaques in the carotid ultrasound image to be predicted through the model, thereby realizing non-invasive and accurate carotid plaque ultrasonic quantification.

Description

Technical Field

[0001] The present invention relates to the field of ultrasonic quantification of carotid artery plaques, and in particular to a method, system and electronic equipment for ultrasonic quantification of carotid artery plaques. Background Art

[0002] Cardiovascular and cerebrovascular diseases are the leading causes of death and disability worldwide. Atherosclerosis is its pathological basis, and the characteristics of carotid plaques play an important role in risk assessment. Existing plaque assessment mainly relies on pathological histology, but the method of resecting specimens is not suitable for the evaluation of early plaques. Therefore, there is an urgent need for a non-invasive and accurate assessment method. Carotid ultrasound examination has the advantages of being non-invasive, simple, and capable of dynamic monitoring, but it is still challenging in parsing subtle differences in ultrasound images. Although some automatic analysis techniques have emerged, such as grayscale median (GSM), their accuracy is controversial. Therefore, how to improve the accuracy of extracting atherosclerotic plaque information in ultrasound images and achieve results that are highly consistent with pathology has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] In order to improve the analysis accuracy of ultrasound images, the present invention proposes a method for ultrasonic quantification of carotid artery plaques, comprising:

[0004] A target ResNet network is constructed based on the circular convolution unit, and the encoder and decoder of the U-Net network are reconstructed through the target ResNet network and multiple residual decoding units to obtain a ResNet-U-Net combined network; a quantization model is constructed through the ResNet-U-Net combined network and the CNN network;

[0005] Constructing a training set including a plurality of training samples; the training samples include a carotid artery ultrasound image and a corresponding pathological image; the labels of the training samples are quantitative information of plaques in the corresponding pathological image; and obtaining a carotid artery plaque ultrasound quantitative model by training a quantitative model with the training set;

[0006] The carotid artery ultrasound image to be predicted is acquired and input into a carotid artery plaque ultrasound quantification model, and the quantification information of the plaque in the carotid artery ultrasound image to be predicted is output through the model.

[0007] Furthermore, the quantitative information of the plaque includes: the area, volume, density of the plaque, and the morphology of the plaque and an indication of whether the plaque is bleeding; the morphology of the plaque includes: an indication of whether the plaque is regular, an indication of whether the plaque has ulcers, and an indication of whether the plaque is eroded.

[0008] Furthermore, the target ResNet network is constructed based on the circular convolution unit, specifically:

[0009] A ResNet network including multiple residual layers is constructed, and the convolutional unit in each residual layer is replaced by a circular convolutional unit to obtain a target ResNet network.

[0010] Furthermore, the quantization model is constructed by combining the ResNet-U-Net network with the CNN network, specifically:

[0011] The first channel is constructed by combining the ResNet-U-Net network, and the second channel is constructed by the CNN network; a third channel connected to the outputs of the first channel and the second channel is constructed, and a quantization model is formed by the first channel, the second channel and the third channel;

[0012] The first channel is used to extract a first feature vector through a ResNet-U-Net combined network; the first feature vector is a feature vector reflecting plaque information in a carotid artery ultrasound image;

[0013] The second channel is used to extract a second feature vector through a CNN network; the second feature vector is a feature vector that reflects plaque information in a pathological image;

[0014] The third channel is used to obtain a fusion feature by concatenating the first feature vector and the second feature vector, and to obtain quantitative information corresponding to the plaque through the fusion feature.

[0015] Furthermore, the third channel includes:

[0016] A splicing module, used for splicing the first feature vector and the second feature vector to obtain a splicing feature;

[0017] The activation module is used to perform nonlinear transformation on the splicing features through the Rulu activation function to obtain the fusion features;

[0018] The quantification module is used to perform quantitative analysis on the plaques using fusion features to obtain quantitative information corresponding to the plaques.

[0019] Furthermore, the encoder of the ResNet-U-Net combined network specifically includes:

[0020] An input module, used for inputting a carotid artery ultrasound image;

[0021] A first convolution module, used for performing a convolution operation on the carotid artery ultrasound image to obtain a first convolution feature;

[0022] A target ResNet network is used to perform multiple residual operations on the first convolution feature output by the first convolution module to obtain the first residual feature; multiple residual layers in the target ResNet network are connected in sequence, and the input of the first residual layer is the output of the first convolution module;

[0023] The residual block is used to perform a residual operation on the first residual feature to obtain a second residual feature, and add the second residual feature to the first residual feature to obtain a target residual feature.

[0024] Furthermore, the decoder of the ResNet-U-Net combined network includes:

[0025] A plurality of residual decoding units connected in sequence;

[0026] The input of the first residual decoding unit is the output of the residual block in the encoder, and the remaining residual decoding units are connected to the inputs of each residual layer in the target ResNet network in a one-to-one correspondence; the residual decoding unit is used to perform residual decoding operations on its input data or its input data and the input of the corresponding residual layer;

[0027] The output module is used to output the result obtained after the last residual decoding unit performs the residual decoding operation, that is, the first eigenvector.

[0028] The embodiment of the present invention further provides a carotid artery plaque ultrasonic quantification system, comprising:

[0029] A model building module is used to build a target ResNet network based on a recurrent convolution unit, and reconstruct the encoder and decoder of the U-Net network through the target ResNet network and multiple residual decoding units to obtain a ResNet-U-Net combined network; a quantization model is built through the ResNet-U-Net combined network and the CNN network;

[0030] A training module is used to construct a training set including a plurality of training samples; the training samples include a carotid artery ultrasound image and a corresponding pathological image; the labels of the training samples are quantitative information of plaques in the corresponding pathological image; and a carotid artery plaque ultrasound quantitative model is obtained by training a quantitative model with the training set;

[0031] The quantification module is used to obtain the carotid artery ultrasound image to be predicted and input the carotid artery plaque ultrasound quantification model, and output the quantification information of the plaque in the carotid artery ultrasound image to be predicted through the model.

[0032] Furthermore, the quantitative information of the plaque includes: the area, volume, density of the plaque, and the morphology of the plaque and an indication of whether the plaque is bleeding; the morphology of the plaque includes: an indication of whether the plaque is regular, an indication of whether the plaque has ulcers, and an indication of whether the plaque is eroded.

[0033] An embodiment of the present invention further provides an electronic device, including: a processor, and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the method described above.

[0034] The embodiment of the present invention further provides a non-transitory machine-readable medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described above.

[0035] Compared with the prior art, the present invention has at least the following beneficial effects:

[0036] (1) The present invention introduces a recurrent convolution unit to construct a target ResNet network, and combines multiple residual decoding units to reconstruct the encoder and decoder of the U-Net network, ultimately forming a ResNet-U-Net combined network, and jointly constructs a quantization model with a CNN network, thereby solving the technical problem of how to improve the extraction accuracy of atherosclerotic plaque information in ultrasound images without removing the specimen, and achieve results that are highly consistent with pathology. Specifically, by adding carotid ultrasound images and their corresponding pathological images as training samples in the training set, not only can the size (area, volume) of the plaque be obtained, but also further in-depth analysis can be performed on details such as whether the plaque will cause bleeding, the area and volume of the lipid necrosis core, and whether there is a fibrous cap. This is due to the correction effect of the previous pathological image data on the accuracy of the extracted information, ensuring the effectiveness and reliability of the model training. The constructed model can achieve non-invasive and accurate carotid plaque quantification without obtaining pathological samples again in future applications, providing strong support for early clinical diagnosis and treatment.

[0037] (2) In the present invention, the convolutional unit in each residual layer is replaced by a circular convolutional unit to obtain a target ResNet network, thereby enhancing the feature extraction capability and improving the richness and accuracy of features;

[0038] (3) In the present invention, in the decoder of the ResNet-U-Net combined network, the input of the first residual decoding unit is the output of the residual block in the encoder, and the remaining residual decoding units are connected to the inputs of each residual layer in the target ResNet network in a one-to-one correspondence; the residual decoding unit is used to perform residual decoding operations on its input data or its input data and the input of the corresponding residual layer; that is, the residual decoding unit in the present invention further improves the accuracy of feature extraction by fusing the input from the residual layer in the encoder and gradually restoring the feature map, i.e., the spatial resolution of the input data;

[0039] (4) The present invention improves the encoder and decoder of the U-Net network by using a target ResNet network and multiple residual decoding units, thereby improving the ability and accuracy of extracting features of atherosclerotic plaques in carotid artery ultrasound images;

[0040] (5) The present invention is suitable for dynamic monitoring and early evaluation, does not require invasive operations, can be performed frequently, and provides strong support for the early detection and intervention of cardiovascular and cerebrovascular diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flow chart of a method for ultrasound quantification of carotid artery plaque;

[0042] Figure 2 This is a schematic diagram of the structure of the ResNet-U-Net combined network;

[0043] Figure 3 It is a structural diagram of the quantitative model;

[0044] Figure 4 It is a schematic diagram of the structure of the residual layer in the target ResNet network;

[0045] Figure 5 Schematic diagram of the structure of the residual block in the encoder of the ResNet-U-Net combined network;

[0046] Figure 6 It is a schematic diagram of the structure of the residual decoding unit in the decoder of the ResNet-U-Net combined network;

[0047] Figure 7 The expanded structure diagram of the recurrent convolution unit with respect to the time step;

[0048] Figure 8 A module diagram of a carotid artery plaque ultrasound quantification system;

[0049] Fig. 9 FIG. 4 is a schematic diagram of the structure of the electronic device of this embodiment. DETAILED DESCRIPTION

[0050] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solution of the present invention, but the present invention is not limited to these embodiments.

[0051] Embodiment 1

[0052] Existing plaque assessment mainly relies on pathological histology, but the method of excising specimens is not suitable for the evaluation of early plaques because this method is highly invasive, high-risk, and not suitable for frequent monitoring. Carotid artery ultrasound has the advantages of being non-invasive, simple, and capable of dynamic monitoring, but it is still challenging to interpret subtle differences in ultrasound images. Although some automatic analysis techniques have emerged, such as grayscale median (GSM), their accuracy is controversial and cannot achieve results that are highly consistent with pathology. Therefore, in order to make the quantitative results of ultrasound highly consistent with the results of pathological tests, such as Figure 1 As shown, the embodiment of the present invention provides a method for ultrasonic quantification of carotid artery plaque, comprising:

[0053] A target ResNet network is constructed based on the circular convolution unit, and the encoder and decoder of the U-Net network are reconstructed through the target ResNet network and multiple residual decoding units to obtain a ResNet-U-Net combined network; a quantization model is constructed through the ResNet-U-Net combined network and the CNN network;

[0054] The target ResNet network is constructed based on the circular convolution unit, specifically:

[0055] A ResNet network including multiple residual layers is constructed, and the convolutional unit in each residual layer is replaced by a circular convolutional unit to obtain a target ResNet network.

[0056] In this embodiment, the structure of the residual layer is as follows: Figure 4 As shown, specifically including:

[0057] The first path and the second path; wherein:

[0058] The first path is connected in sequence: input unit, batch normalization unit 1, activation unit 1, circular convolution unit 1, batch normalization unit 2, activation unit 2, circular convolution unit 2 and fusion unit 1;

[0059] The second path is: a jump path from the input unit to the fusion unit 1;

[0060] In this embodiment, if the current residual layer is the first residual layer (i.e. Figure 2 The residual layer 1 in , then the input data of the input unit is Figure 2 The first convolution feature output by the first convolution module in ; if the current residual layer is not the first residual layer, the input data of the input unit is the output data of the previous residual layer; in addition, the fusion unit 1 in the residual layer is used to fuse the output data of the circular convolution unit 2 and the input unit, and output it.

[0061] The U-Net network has achieved remarkable results in the field of medical image segmentation. Its original version uses a method of randomly initializing parameters from a Gaussian distribution during training. The core idea of ​​U-Net is to splice low-order features with corresponding high-order features, which essentially provides an additional path for the transmission of information, which not only allows information to flow between low-level and high-level layers in a more direct way, but also facilitates the addition of low-level edge features to high-level semantic features. This design concept is similar to the residual neural network (ResNet). In view of this, the present invention combines the advantages of the residual unit and introduces the ResNet network into the U-Net architecture, which can further enhance the learning ability and generalization performance of the model.

[0062] It should also be noted that the expansion structure of the circular convolution unit in this embodiment with respect to the time step is as follows: Figure 7 As shown, it shows the structure of a recurrent convolution unit (RCU) containing multiple time steps. Specifically, the RCU can be expanded into three convolution operations (the first convolution operation, the second convolution operation, and the third convolution operation), where:

[0063] 1. Time step:

[0064] t=0: the input feature map undergoes the first convolution operation; the input feature map represents the input data of the circular convolution unit;

[0065] t=1 and t=2: In the subsequent circular convolution operation, the output of each time step t will be used as the input of the next time step, forming a circular structure.

[0066] 2. Expanded form:

[0067] When there are t time steps in RCU, unfolding RCU will generate a feed-forward subnetwork with a depth of t+1; for example, when t=2, RCU will be unfolded into a subnetwork with 3 convolutional layers (or 3 convolution operations, i.e. Figure 7 The first convolution operation, the second convolution operation and the third convolution operation in the sub-network (t=0,t=1,t=2).

[0068] 3. Input and output:

[0069] Feedforward input: remains the same in all time steps, that is, the input feature map is used as input in each time step;

[0070] Cyclic input: As the number of iterations changes, the output of each time step becomes the input of the next time step.

[0071] 4. Path:

[0072] The longest path: passing through all the expanded loop connections, with a length of t+1;

[0073] The shortest path: only one feedforward connection, with a length of 1;

[0074] In the recurrent convolutional unit (RCU), the feedforward input (i.e., the input feature map) remains unchanged in each time step as one of the inputs of each time step. The specific explanation is as follows:

[0075] Feedforward input refers to the input feature map, which is the input data of the recurrent convolution unit. This input feature map will be used in each time step of the RCU. The specific examples are as follows:

[0076] 1. t=0:

[0077] Initial time step t=0: Input feature map x undergoes the first convolution operation to generate output feature map y 0 ; This output feature map y 0 It will be used as the input for the next loop at time step t=1.

[0078] 2. t=1:

[0079] At time step t=1, the input data consists of two parts:

[0080] 1) Feedforward input: input feature map x;

[0081] 2) Cycle input: the output feature map y of the previous time step t=0 0 ;

[0082] These two parts of the feature map will be added or concatenated, and then undergo a second convolution operation to generate the output feature map y 1 , and the output feature map y 1 It will be used as the loop input for the next time step t=2.

[0083] 3. t=2:

[0084] At time step t=2, the input feature map consists of two parts:

[0085] 1) Feedforward input: input feature map x;

[0086] 2) Cycle input: the output feature map y of the previous time step t=1 1 ;

[0087] These two parts of the feature map will be added or concatenated, and then undergo a third convolution operation to generate the output feature map y 2 .

[0088] The circular convolution unit in this embodiment has the following advantages:

[0089] 1. The existence of longer paths enables the model to learn high-order and complex features;

[0090] 2. The existence of shorter paths helps the gradient direction propagate during training and avoids the gradient vanishing problem;

[0091] 3. Larger receptive field: RCU can obtain a larger receptive field than ordinary convolutional layers. In conventional convolutional neural networks, the receptive field of a fixed convolutional layer is fixed, and only the top network layer can have a larger receptive field. RCU can merge context information into any area in the current layer as the time step increases, thereby learning stronger feature expressions.

[0092] The encoder of the ResNet-U-Net combined network specifically includes:

[0093] An input module, used for inputting a carotid artery ultrasound image;

[0094] A first convolution module, used for performing a convolution operation on the carotid artery ultrasound image to obtain a first convolution feature;

[0095] A target ResNet network is used to perform multiple residual operations on the first convolution feature output by the first convolution module to obtain the first residual feature; multiple residual layers in the target ResNet network are connected in sequence, and the input of the first residual layer is the output of the first convolution module;

[0096] The residual block is used to perform a residual operation on the first residual feature to obtain a second residual feature, and add the second residual feature to the first residual feature to obtain a target residual feature.

[0097] In this embodiment, the structure of the residual block is as follows: Figure 5 As shown, including:

[0098] The first calculation channel: the batch normalization unit 3, the activation unit 3, the weight calculation unit 1, the batch normalization unit 4, the activation unit 4, the weight calculation unit 2 and the fusion unit 2 connected in sequence;

[0099] The second calculation channel is the connection path from the input of the residual block to the fusion unit 2;

[0100] in:

[0101] The first calculation channel is used to perform residual operation on the first residual feature to obtain a second residual feature;

[0102] The fusion unit 2 is used to perform an addition operation on the second residual feature and the first residual feature to obtain a target residual feature.

[0103] The decoder of the ResNet-U-Net combined network includes:

[0104] A plurality of residual decoding units connected in sequence;

[0105] The input of the first residual decoding unit is the output of the residual block in the encoder, and the remaining residual decoding units are connected to the inputs of each residual layer in the target ResNet network in a one-to-one correspondence; the residual decoding unit is used to perform residual decoding operations on its input data or its input data and the input of the corresponding residual layer;

[0106] The output module is used to output the result obtained after the last residual decoding unit performs the residual decoding operation, that is, the first eigenvector.

[0107] In the present invention, in the decoder of the ResNet-U-Net combined network, the input of the first residual decoding unit is the output of the residual block in the encoder (i.e., the target residual feature), and the remaining residual decoding units are connected one-to-one with the input of each residual layer in the target ResNet network; the residual decoding unit is used to perform residual decoding operations on its input data or its input data and the input of the corresponding residual layer; that is, the residual decoding unit in the present invention further improves the accuracy of feature extraction by fusing the input from the residual layer in the encoder and gradually restoring the feature map, i.e., the spatial resolution of the input data. Specifically:

[0108] like Figure 2 The structure diagram of the ResNet-U-Net combined network shown in the figure, where:

[0109] The target ResNet network includes multiple residual layers, specifically: residual layer 1, residual layer 2, residual layer 3 and residual layer 4;

[0110] The multiple residual decoding units included in the decoder are specifically:

[0111] Residual decoding unit 0 (i.e., the first residual decoding unit), residual decoding unit 1, residual decoding unit 2, residual decoding unit 3, residual decoding unit 4;

[0112] in:

[0113] The input of the residual layer 1 is connected to the residual decoding unit 4;

[0114] The input of the residual layer 2 is connected to the residual decoding unit 3;

[0115] The input of the residual layer 3 is connected to the residual decoding unit 2;

[0116] The input of the residual layer 4 is connected to the residual decoding unit 1.

[0117] Specifically, the structure of the residual decoding unit is as follows: Figure 6 As shown, it includes:

[0118] The first connection channel: the fusion unit 3, the convolution unit 1, the activation unit 5, the batch normalization unit 4, the splicing unit 1 and the upsampling unit 1 are connected in sequence;

[0119] The second connection channel is a jump path from the fusion unit 3 to the upsampling unit 1;

[0120] The output of the residual block is transmitted to the residual decoding unit 1 after convolution, activation, batch normalization, splicing, upsampling processing of the first connection channel in the residual decoding unit 0, and upsampling processing of the second connection channel; the fusion unit 3 in the residual decoding unit 1 fuses the output data of the residual decoding unit 0 with the input of the residual layer 4, and the fused data is processed through the first connection channel and the second connection channel respectively and then output to the residual decoding unit 2, and so on, until the residual decoding unit 4 completes the residual decoding operation.

[0121] The ResNet-U-Net combined network and the CNN network are constructed as follows Figure 3 The quantitative model shown is specifically:

[0122] The first channel is constructed by combining the ResNet-U-Net network, and the second channel is constructed by the CNN network; a third channel connected to the outputs of the first channel and the second channel is constructed, and a quantization model is formed by the first channel, the second channel and the third channel;

[0123] The first channel is used to extract information from the carotid artery ultrasound image through a ResNet-U-Net combined network to obtain a first eigenvector; the first eigenvector is a eigenvector reflecting plaque information in the carotid artery ultrasound image;

[0124] The second channel is used to extract information in the pathological image through the CNN network to obtain a second feature vector; the second feature vector is a feature vector that reflects the plaque information in the pathological image;

[0125] The third channel is used to obtain a fusion feature by concatenating the first feature vector and the second feature vector, and to obtain quantitative information corresponding to the plaque through the fusion feature.

[0126] The third channel comprises:

[0127] A splicing module, used for splicing the first feature vector and the second feature vector to obtain a splicing feature;

[0128] The activation module is used to perform nonlinear transformation on the splicing features through the Rulu activation function to obtain the fusion features;

[0129] The quantification module is used to perform quantitative analysis on the plaques using fusion features to obtain quantitative information corresponding to the plaques.

[0130] Constructing a training set including a plurality of training samples; the training samples include a carotid artery ultrasound image and a corresponding pathological image; the labels of the training samples are quantitative information of plaques in the corresponding pathological image; and obtaining a carotid artery plaque ultrasound quantitative model by training a quantitative model with the training set;

[0131] It should be explained that the technical problem to be solved by the present invention is: how to improve the extraction accuracy of atherosclerotic plaque information in ultrasound images without removing the specimen, so as to achieve a result that is highly consistent with pathology. In this regard, if only the carotid ultrasound image and its corresponding label are used to train the model, only the size (area, volume) of the plaque can be obtained. Therefore, the present invention adds pathological images to the training samples. After adding the pathological images, a more detailed analysis can be performed on whether the plaque will cause bleeding, the area and volume of the lipid necrosis core, and whether there is a fibrous cap. The accuracy of the extracted information is corrected by the early pathological image data. After constructing such a model, when using the trained model in the future, there is no need to take pathological samples, thereby achieving non-invasive and accurate plaque assessment.

[0132] The carotid artery ultrasound image to be predicted is acquired and input into a carotid artery plaque ultrasound quantification model, and the quantification information of the plaque in the carotid artery ultrasound image to be predicted is output through the model.

[0133] The quantitative information of the plaque includes: the area, volume, density of the plaque, the morphology of the plaque and an indication of whether the plaque is bleeding; the morphology of the plaque includes: an indication of whether the plaque is regular, an indication of whether the plaque is ulcerated and an indication of whether the plaque is eroded.

[0134] The quantitative information of the plaque also includes: the area and volume of the lipid necrosis core and the identification of whether there is a fibrous cap; wherein:

[0135] Lipid necrosis core: refers to the necrotic area formed by lipid accumulation in the plaque, which usually appears as a low-density area. It is necessary to be able to identify and measure the area and volume of the lipid necrosis core;

[0136] Fibrous cap: refers to the fibrous tissue covering the lipid necrotic core, which usually appears as a high-density area. It is necessary to be able to determine whether a fibrous cap is present.

[0137] The present invention introduces a recurrent convolution unit to construct a target ResNet network, and combines a plurality of residual decoding units to reconstruct the encoder and decoder of the U-Net network, ultimately forming a ResNet-U-Net combined network, and jointly constructs a quantization model with a CNN network, thereby solving the technical problem of how to improve the extraction accuracy of atherosclerotic plaque information in ultrasound images without removing the specimen, and achieving results that are highly consistent with pathology; specifically, by adding carotid ultrasound images and their corresponding pathological images as training samples in the training set, not only can the size (area, volume) of the plaque be obtained, but also details such as whether the plaque will cause bleeding, the area and volume of the lipid necrosis core, and whether there is a fibrous cap can be further analyzed in depth; this is due to the correction effect of the previous pathological image data on the accuracy of the extracted information, thereby ensuring the effectiveness and reliability of the model training; the constructed model does not need to obtain pathological samples again in future applications, and can achieve non-invasive and accurate carotid plaque quantification, providing strong support for early clinical diagnosis and treatment.

[0138] Embodiment 2

[0139] In order to make the quantitative results of ultrasound highly consistent with the results of pathological examination, such as Figure 8 As shown, the embodiment of the present invention also provides a carotid artery plaque ultrasonic quantification system, including:

[0140] A model building module is used to build a target ResNet network based on a recurrent convolution unit, and reconstruct the encoder and decoder of the U-Net network through the target ResNet network and multiple residual decoding units to obtain a ResNet-U-Net combined network; a quantization model is built through the ResNet-U-Net combined network and the CNN network;

[0141] A training module is used to construct a training set including a plurality of training samples; the training samples include a carotid artery ultrasound image and a corresponding pathological image; the labels of the training samples are quantitative information of plaques in the corresponding pathological image; and a carotid artery plaque ultrasound quantitative model is obtained by training a quantitative model with the training set;

[0142] The quantification module is used to obtain the carotid artery ultrasound image to be predicted and input the carotid artery plaque ultrasound quantification model, and output the quantification information of the plaque in the carotid artery ultrasound image to be predicted through the model.

[0143] The quantitative information of the plaque includes: the area, volume, density of the plaque, the morphology of the plaque and an indication of whether the plaque is bleeding; the morphology of the plaque includes: an indication of whether the plaque is regular, an indication of whether the plaque is ulcerated and an indication of whether the plaque is eroded.

[0144] The present invention constructs a target ResNet network based on a circular convolution unit, and reconstructs an encoder and a decoder of a U-Net network through the target ResNet network and a plurality of residual decoding units, respectively, to obtain a ResNet-U-Net combined network; constructs a quantization model through the ResNet-U-Net combined network and a CNN network; constructs a training set including a plurality of training samples; the training samples include a carotid ultrasound image and its corresponding pathological image; the labels of the training samples are quantization information of plaques in the corresponding pathological image; obtains a carotid plaque ultrasonic quantization model through training the quantization model with the training set; obtains a carotid ultrasound image to be predicted and inputs it into the carotid plaque ultrasonic quantization model, and outputs the quantization information of the plaques in the carotid ultrasound image to be predicted through the model, thereby realizing non-invasive and accurate carotid plaque ultrasonic quantization.

[0145] Embodiment 3

[0146] An embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device executes the method of the embodiment of the present invention.

[0147] An embodiment of the present invention further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present invention.

[0148] An embodiment of the present invention further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to enable the computer to execute the method of the embodiment of the present invention.

[0149] The relevant devices in the present invention are the electronic device, non-transitory machine-readable medium and computer program product in this embodiment.

[0150] refer to Fig. 9 , a block diagram of an electronic device that can be used as a server or client of an embodiment of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0151] like Fig. 9 As shown, the electronic device includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 to a random access memory (RAM) 403. In RAM 403, various programs and data required for the operation of the electronic device can also be stored. The computing unit 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0152] Multiple components in the electronic device are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device that can input information to the electronic device, and the input unit 406 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 407 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 can include but is not limited to a disk, an optical disk. The communication unit 409 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0153] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the methods and processes described above. For example, in some embodiments, the method embodiments of the present invention may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0154] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0155] In the context of an embodiment of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0157] In addition, in the present invention, descriptions such as "first", "second", "one", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0158] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0159] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

Claims

1. A method for ultrasonic quantification of carotid artery plaque, characterized in that: include: A target ResNet network is constructed based on the circular convolution unit, and the encoder and decoder of the U-Net network are reconstructed through the target ResNet network and multiple residual decoding units to obtain a ResNet-U-Net combined network; a quantization model is constructed through the ResNet-U-Net combined network and the CNN network; The encoder of the ResNet-U-Net combined network specifically includes: An input module, used for inputting a carotid artery ultrasound image; A first convolution module, used for performing a convolution operation on the carotid artery ultrasound image to obtain a first convolution feature; A target ResNet network is used to perform multiple residual operations on the first convolution feature output by the first convolution module to obtain the first residual feature; multiple residual layers in the target ResNet network are connected in sequence, and the input of the first residual layer is the output of the first convolution module; A residual block, used for performing a residual operation on the first residual feature to obtain a second residual feature, and adding the second residual feature to the first residual feature to obtain a target residual feature; The decoder of the ResNet-U-Net combined network includes: A plurality of residual decoding units connected in sequence; The input of the first residual decoding unit is the output of the residual block in the encoder, and the remaining residual decoding units are connected to the inputs of each residual layer in the target ResNet network in a one-to-one correspondence; the residual decoding unit is used to perform residual decoding operations on its input data or its input data and the input of the corresponding residual layer; An output module, used to output the result obtained after the last residual decoding unit performs residual decoding operation, that is, the first eigenvector; The quantization model is constructed by combining the ResNet-U-Net network with the CNN network, specifically: The first channel is constructed by combining the ResNet-U-Net network, and the second channel is constructed by the CNN network; a third channel connected to the outputs of the first channel and the second channel is constructed, and a quantization model is formed by the first channel, the second channel and the third channel; The first channel is used to extract a first feature vector through a ResNet-U-Net combined network; the first feature vector is a feature vector reflecting plaque information in a carotid artery ultrasound image; The second channel is used to extract a second feature vector through a CNN network; the second feature vector is a feature vector that reflects plaque information in a pathological image; The third channel is used to obtain a fusion feature by concatenating the first feature vector and the second feature vector, and to obtain quantitative information corresponding to the plaque through the fusion feature; Constructing a training set including a plurality of training samples; the training samples include a carotid artery ultrasound image and a corresponding pathological image; the labels of the training samples are quantitative information of plaques in the corresponding pathological image; and obtaining a carotid artery plaque ultrasound quantitative model by training a quantitative model with the training set; The carotid artery ultrasound image to be predicted is acquired and input into a carotid artery plaque ultrasound quantification model, and the quantification information of the plaque in the carotid artery ultrasound image to be predicted is output through the model.

2. The method for ultrasonic quantification of carotid plaque according to claim 1, characterized in that: The quantitative information of the plaque includes: the area, volume, density of the plaque, the morphology of the plaque and an indication of whether the plaque is bleeding; the morphology of the plaque includes: an indication of whether the plaque is regular, an indication of whether the plaque is ulcerated and an indication of whether the plaque is eroded.

3. The method for ultrasonic quantification of carotid plaque according to claim 2, characterized in that: The target ResNet network is constructed based on the circular convolution unit, specifically: A ResNet network including multiple residual layers is constructed, and the convolutional unit in each residual layer is replaced by a circular convolutional unit to obtain a target ResNet network.

4. The method for ultrasonic quantification of carotid plaque according to claim 3, characterized in that: The third channel comprises: A splicing module, used for splicing the first feature vector and the second feature vector to obtain a splicing feature; The activation module is used to perform nonlinear transformation on the splicing features through the Rulu activation function to obtain the fusion features; The quantification module is used to perform quantitative analysis on the plaques using fusion features to obtain quantitative information corresponding to the plaques.

5. A carotid artery plaque ultrasound quantification system, characterized in that: include: A model building module is used to build a target ResNet network based on a recurrent convolution unit, and reconstruct the encoder and decoder of the U-Net network through the target ResNet network and multiple residual decoding units to obtain a ResNet-U-Net combined network; a quantization model is built through the ResNet-U-Net combined network and the CNN network; The encoder of the ResNet-U-Net combined network specifically includes: An input module, used for inputting a carotid artery ultrasound image; A first convolution module, used for performing a convolution operation on the carotid artery ultrasound image to obtain a first convolution feature; A target ResNet network is used to perform multiple residual operations on the first convolution feature output by the first convolution module to obtain the first residual feature; multiple residual layers in the target ResNet network are connected in sequence, and the input of the first residual layer is the output of the first convolution module; A residual block, used for performing a residual operation on the first residual feature to obtain a second residual feature, and adding the second residual feature to the first residual feature to obtain a target residual feature; The decoder of the ResNet-U-Net combined network includes: A plurality of residual decoding units connected in sequence; The input of the first residual decoding unit is the output of the residual block in the encoder, and the remaining residual decoding units are connected to the inputs of each residual layer in the target ResNet network in a one-to-one correspondence; the residual decoding unit is used to perform residual decoding operations on its input data or its input data and the input of the corresponding residual layer; An output module, used to output the result obtained after the last residual decoding unit performs residual decoding operation, that is, the first eigenvector; The quantization model is constructed by combining the ResNet-U-Net network with the CNN network, specifically: The first channel is constructed by combining the ResNet-U-Net network, and the second channel is constructed by the CNN network; a third channel connected to the outputs of the first channel and the second channel is constructed, and a quantization model is formed by the first channel, the second channel and the third channel; The first channel is used to extract a first feature vector through a ResNet-U-Net combined network; the first feature vector is a feature vector reflecting plaque information in a carotid artery ultrasound image; The second channel is used to extract a second feature vector through a CNN network; the second feature vector is a feature vector that reflects plaque information in a pathological image; The third channel is used to obtain a fusion feature by concatenating the first feature vector and the second feature vector, and to obtain quantitative information corresponding to the plaque through the fusion feature; A training module is used to construct a training set including a plurality of training samples; the training samples include a carotid artery ultrasound image and a corresponding pathological image; the labels of the training samples are quantitative information of plaques in the corresponding pathological image; and a carotid artery plaque ultrasound quantitative model is obtained by training a quantitative model with the training set; The quantification module is used to obtain the carotid artery ultrasound image to be predicted and input the carotid artery plaque ultrasound quantification model, and output the quantification information of the plaque in the carotid artery ultrasound image to be predicted through the model.

6. The carotid artery plaque ultrasonic quantification system according to claim 5, characterized in that: The quantitative information of the plaque includes: the area, volume, density of the plaque, the morphology of the plaque and an indication of whether the plaque is bleeding; the morphology of the plaque includes: an indication of whether the plaque is regular, an indication of whether the plaque is ulcerated and an indication of whether the plaque is eroded.

7. An electronic device comprising: A processor and a memory storing a program, wherein the program comprises instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 4.

Images