A carotid ultrasound plaque multi-view classification method based on segmentation guidance

Abstract

The present application relates to the technical field of artificial intelligence, in particular to a carotid ultrasound plaque multi-view classification method based on segmentation guidance, comprising: adding a plaque prediction mask for plaque prediction to an ultrasound image group to be predicted, and then establishing a plaque local view based on the plaque prediction mask; performing feature extraction and aggregation on the plaque local view and the corresponding original input in the ultrasound image group to obtain aggregated features; and inputting the aggregated features into a fusion classification model to obtain a prediction result. The plaque recognition scheme in the prior art usually only focuses on one type of input data, resulting in an undesirable recognition accuracy. By introducing conventional ultrasound images and contrast ultrasound images and constructing corresponding plaque local views as modal input supplements, better input modal acquisition is achieved. In subsequent recognition, more accurate classification and prediction effects can be achieved based on the aggregated features of each modal, and the segmentation results and confidence corresponding to the plaque are output.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a segmentation-guided multi-view classification method for carotid artery ultrasound plaques. Background Technology

[0002] Carotid artery plaques are a significant factor contributing to carotid artery stenosis and related cerebrovascular events, making accurate identification and classification crucial. Ultrasound imaging, characterized by its non-invasiveness, ease of operation, and repeatability, is a commonly used method for carotid artery plaque detection and assessment. Conventional ultrasound can display plaque morphology, boundaries, and echogenicity, while angiography can further reflect microvascular perfusion within the plaque. These two imaging modalities describe plaque characteristics from different perspectives. With the development of computer vision and deep learning technologies, automatic carotid artery plaque classification methods based on ultrasound images are increasingly being applied. Existing methods typically input the complete ultrasound image into a feature extraction network and obtain classification results based on the extracted image features.

[0003] For example, patent application CN202111556641.1 discloses a method and system for arterial plaque identification based on ultrasound scan video, relating to the field of medical image processing technology. The method includes: acquiring an ultrasound scan video; extracting multiple consecutive frames of ultrasound scan images from the ultrasound scan video; performing arterial plaque identification on each of the ultrasound scan images to obtain corresponding image recognition results; and processing the image recognition results to obtain the arterial plaque identification result of the ultrasound scan video. The beneficial effect is that by performing arterial plaque identification based on multiple consecutive frames of ultrasound scan images from the ultrasound scan video, dynamic image information of consecutive frames can be captured, thereby enabling the detection of missing arterial plaques and arterial plaques that are difficult to identify in a single ultrasound scan image, improving the recognition rate and reducing the false positive detection rate.

[0004] For example, patent application CN202110826201.7 discloses a carotid artery ultrasound automatic scanning and plaque identification system and method, belonging to the field of ultrasound image processing. It includes: a probe for acquiring carotid artery images; a sensing unit for simultaneously acquiring the probe's position information and transmitting it along with the carotid artery image to a three-dimensional imaging ultrasound system; a three-dimensional ultrasound imaging system performing three-dimensional reconstruction processing on the carotid artery image based on the position information; an identification unit for identifying the three-dimensional image; and a feedback unit for outputting feedback information when the identification result indicates the presence of plaque in the carotid artery. The beneficial effect is that, based on a three-dimensional ultrasound imaging system and combined with intelligent ultrasound AI technology, an integrated system for automatic carotid artery ultrasound scanning and plaque identification, suitable for community deployment to assist elderly patients in self-health management, is established. This effectively avoids the interference of traditional handheld ultrasound on the operator's skills and experience, greatly reducing the technical threshold for operators.

[0005] However, in actual implementation, the inventors found that such technical solutions usually use a single image as the classification object, or simply average the prediction results of multiple images, making it difficult to fully utilize the complementary information between multiple images of the same patient. Summary of the Invention

[0006] To address the aforementioned problems in existing technologies, a segmentation-guided multi-view classification method for carotid artery ultrasound plaques is provided.

[0007] The specific technical solution is as follows: A segmentation-guided multi-view classification method for carotid artery plaques via ultrasound includes: Step S1: For the ultrasound image group to be predicted, patch prediction is performed by adding a patch prediction mask, and then local views of the patches are established based on the patch prediction mask. The ultrasound image set includes multiple conventional ultrasound images and / or contrast-enhanced ultrasound images; Step S2: Extract and aggregate features from the local view of the patch and the corresponding original input in the ultrasound image group to obtain aggregated features; Step S3: Input the aggregated features into the fusion classification model to obtain the prediction result.

[0008] On the other hand, when the ultrasound image group includes conventional ultrasound images and contrast ultrasound images, the patch local view consists of a conventional patch local view and a contrast patch local view.

[0009] On the other hand, step S1 includes: Step S11: The ultrasound image group is received sequentially using a pre-trained patch segmentation model; Step S12: Use the patch segmentation model to output the patch prediction mask and regress it to the corresponding original input; Step S13: Dilate the patch prediction mask to extract the surrounding local area and establish a local view of the patch.

[0010] On the other hand, the patch segmentation model is implemented using the Medical SAM3 model; In step S11, the ultrasound impact group is combined with the patch text prompt and input into the patch segmentation model; The visual encoder of the patch segmentation model has a low-rank adapter in its attention Q-projection and V-projection.

[0011] On the other hand, step S2 includes: Step S21: Use an image feature extraction network to extract features from the local view of the patch and the original input to obtain feature maps; Step S22: Aggregate the feature map to obtain the aggregated feature.

[0012] On the other hand, in step S22, the feature map is aggregated using a feature aggregation formula: ; In the formula, Indicates the first i The patient in v Aggregated features under each view This indicates a feature concatenation operation. For the corresponding to the first i The patient in v The first view under the first view j Feature maps extracted from images.

[0013] On the other hand, step S3 includes: Step S31: Input the aggregated features sequentially into the feature mapping network and the evidence output network to obtain the corresponding evidence sources and add them to the evidence source set; Step S32: Calculate the fusion weight for each evidence source, and then fuse the first evidence source corresponding to the patch local view and the second evidence source corresponding to the original input to obtain the view hiding representation; Step S33: Fuse the view-hiding representation input modal evidence network and construct a cross-modal evidence representation; Step S34: Input the cross-modal evidence representation into the joint evidence network to obtain the final classification evidence source; Step S35: Obtain the prediction result based on the final classification evidence source prediction.

[0014] On the other hand, the generation methods of the fusion weights include: ; In the formula, The fusion weights of the evidence sources s for the i-th patient are given. For the set of evidence sources, The credibility of the evidence source s, Indicates whether the source of evidence s exists. The uncertainty of the evidence source s The total strength of evidence from the aforementioned source s. This represents the k-th component of the Dirichlet parameter of the evidence source s, based on Perform calculations. The evidence output network outputs non-negative class evidence for the evidence source s.

[0015] On the other hand, during the training process, the following loss function is used for optimization: , In the formula, Indicates the final classification loss. Indicates view-assisted loss, Indicates modal-assisted loss, Indicates cross-modal alignment loss. , and This represents the corresponding equilibrium parameter.

[0016] A storage medium includes computer instructions that, when executed by a computer device, perform the aforementioned carotid artery ultrasound plaque multi-view classification method.

[0017] The above technical solution has the following advantages or beneficial effects: To address the issue that existing patch recognition schemes typically focus on only one modality of input data, leading to unsatisfactory recognition accuracy, this paper introduces conventional ultrasound images and contrast-enhanced ultrasound images, constructing corresponding local views of the patches as supplementary modal inputs. This achieves better acquisition of the input modalities. In subsequent recognition, more accurate classification and prediction results can be achieved based on the aggregated features of each modality, outputting the patch segmentation results and confidence scores. Attached Figure Description

[0018] Embodiments of the invention will be described more fully with reference to the accompanying drawings. However, the drawings are for illustration and explanation only and do not constitute a limitation on the scope of the invention.

[0019] Figure 1 This is an overall schematic diagram of an embodiment of the present invention; Figure 2 This is a schematic diagram of step S1 in an embodiment of the present invention; Figure 3 This is a schematic diagram of step S2 in an embodiment of the present invention; Figure 4 This is a schematic diagram of step S3 in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0022] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0023] This invention includes: A segmentation-guided multi-view classification method for carotid artery ultrasound plaques, such as Figure 1 As shown, it includes: Step S1: For the ultrasound image group to be predicted, patch prediction is performed by adding a patch prediction mask, and then local views of the patches are created based on the patch prediction mask. The ultrasound imaging group includes multiple conventional ultrasound images and / or contrast-enhanced ultrasound images; When an ultrasound imaging set includes conventional ultrasound images and contrast-enhanced ultrasound images, the localized plaque view consists of a conventional localized plaque view and a contrast-enhanced localized plaque view.

[0024] Step S2: Extract and aggregate features from the original inputs corresponding to the local view of the patch and the ultrasound image group to obtain aggregated features; The input to the feature extraction network here includes: conventional ultrasound images, conventional patch local views, contrast-enhanced ultrasound images, and contrast-enhanced patch local views; Step S3: Input the aggregated features into the fusion classification model to obtain the prediction results.

[0025] Specifically, addressing the issue that existing patch recognition schemes typically focus on only one modality of input data, leading to unsatisfactory recognition accuracy, this paper introduces conventional ultrasound images and contrast-enhanced ultrasound images, constructing corresponding local views of the patches as supplementary modal inputs. This achieves better acquisition of the input modalities. In subsequent recognition, more accurate classification and prediction results can be achieved based on the aggregated features of each modality, outputting the corresponding patch segmentation results and confidence levels.

[0026] In the actual implementation process, the above-mentioned multi-view classification method for carotid ultrasound plaques is mainly configured as a software embodiment in a workstation, local computer or server supporting ultrasound diagnostic equipment, which is configured to receive an input ultrasound image group, process it sequentially, and output the recognition and segmentation result of the patient's carotid ultrasound plaque, including the plaque position annotation, confidence level and preliminary classification result actually recognized in the image, so as to provide auxiliary film reading for doctors and reduce the workload of identifying plaque positions. The ultrasound image group includes multiple conventional ultrasound images and / or contrast-enhanced ultrasound images of the same patient, wherein certain data missing may occur according to acquisition issues. For the missing part, in the subsequent feature fusion process, by adjusting the running processing branch and setting an existence mark for the view and the evidence source, the missing modality or missing view is prevented from participating in the corresponding evidence fusion, thereby ensuring the normal execution of the fusion classification process in the scenario of missing imaging modalities.

[0027] In one embodiment, as shown in Figure 2 , step S1 includes: Step S11: sequentially receive the ultrasound image group by using a pre-trained plaque segmentation model; Step S12: output a plaque prediction mask by using the plaque segmentation model and regress it to the corresponding original input; Step S13: perform dilation based on the plaque prediction mask to intercept the surrounding local region and establish a local view of the plaque.

[0028] Specifically, in order to realize multi-modal evidence supplementation for the local view, in this embodiment, a pre-trained plaque segmentation model is first used to sequentially receive the ultrasound image group, perform preprocessing such as size normalization on each input image, and distribute the image to the processing branch of the corresponding modality according to the imaging modality to which the image belongs. Wherein, multiple conventional ultrasound images of the same patient constitute the conventional ultrasound image set of the patient, and multiple contrast-enhanced ultrasound images constitute the contrast-enhanced ultrasound image set of the patient.

[0029] Wherein, the plaque segmentation model is implemented by adopting the Medical SAM3 model; In step S11, the ultrasound image group is input into the plaque segmentation model in combination with a plaque text prompt; Low-rank adapters are arranged in the attention Q projection and V projection of the visual encoder of the plaque segmentation model.

[0030] Specifically, the trained plaque segmentation model is integrated inside the software product, and the conventional ultrasound image and the contrast-enhanced ultrasound image respectively correspond to the plaque segmentation model trained under the respective imaging modality. The plaque segmentation model is implemented by adopting the MedicalSAM3 model, and during operation, the carotid plaque text prompt is combined with the image frame and input into the plaque segmentation model together.

[0031] The patch segmentation model has been pre-trained, and most of the pre-trained parameters are frozen during use. To better adapt to the subsequent evidence extraction and aggregation processes, low-rank adapters are set in the attention Q-projection and V-projection of the Transformer block at the rear of its visual encoder.

[0032] The original linear mapping for the input The adapted linear mapping is represented as: , in, and Represents a low-rank learnable matrix. Represents a low-rank dimension. This represents the scaling factor.

[0033] For each input image frame, the patch segmentation model generates multiple candidate segmentation results and their corresponding confidence scores. The candidate segmentation result with the highest confidence score is used as the actual output patch prediction mask, which is then regressed into the original input, i.e., the original conventional ultrasound image or contrast ultrasound image.

[0034] Based on the regression-based patch prediction mask, a dilation algorithm is used to obtain the surrounding image region in the original input, thereby capturing the surrounding local region, and combining it with the patch prediction mask to generate a patch local view.

[0035] In one embodiment, such as Figure 3 As shown, step S2 includes: Step S21: Use an image feature extraction network to extract features from the local view of the patch and the original input to obtain feature maps; Step S22: Aggregate the feature maps to obtain aggregated features.

[0036] Specifically, to achieve better feature extraction results, this embodiment employs a pre-trained image feature extraction network to extract features from both the local patch view and the original input, obtaining feature maps. The image feature extraction network is implemented using a ConvNeXt-Tiny network pre-trained on a large image dataset, with its end classification header removed. Input images are uniformly resized before being fed into the image feature extraction network, and the global pooling features output from the network backbone are used as image features. Depending on the embodiment, other convolutional neural networks or visual Transformers such as ResNet or Vision Transformer can also be used. Subsequently, the feature maps are aggregated to obtain aggregated features. Each feature map includes feature maps corresponding to regular ultrasound images, regular local patch views, contrast-enhanced ultrasound images, and contrast-enhanced local patch views, which are treated as features of a separate view in subsequent processing.

[0037] In one embodiment, in step S22, the feature map is aggregated using a feature aggregation formula: ; In the formula, Indicates the first i The patient in v Aggregated features under each view This indicates a feature concatenation operation. For the corresponding to the first i The patient in v The first view under the first view j Feature maps extracted from images.

[0038] Specifically, let the first The patient in There are a total of views. The images are processed using the formula described above to achieve feature stitching and aggregation. This essentially means performing dimension-wise average aggregation and dimension-wise maximum aggregation on the image features of multiple images under the same view, and then stitching the two aggregation results together. Simultaneously, the existence of each view is recorded and an existence marker is maintained to indicate whether each view exists.

[0039] In one embodiment, such as Figure 4 As shown, step S3 includes: Step S31: Input the aggregated features into the feature mapping network and the evidence output network in sequence to obtain the corresponding evidence sources and add them to the evidence source set; Step S32: Calculate the fusion weight for each evidence source, and then fuse the first evidence source corresponding to the patch local view and the second evidence source corresponding to the original input to obtain the view hiding representation; Step S33: Fuse the view-hidden representations into the input modal evidence network and construct a cross-modal evidence representation; Step S34: Input the cross-modal evidence representation into the joint evidence network to obtain the final classification evidence source; Step S35: Obtain the prediction result based on the final classification evidence source.

[0040] In one embodiment, the generation of the fusion weights includes: ; In the formula, Let s be the fusion weight of evidence sources s for the i-th patient. For the collection of evidence sources, To assess the credibility of evidence source s, Indicates whether the source of evidence s exists. Due to the uncertainty of the source of evidence s, The total strength of evidence from source s. This represents the k-th component of the Dirichlet parameter of the evidence source s. The evidence output network outputs non-negative class evidence for evidence source s.

[0041] Specifically, to achieve better evidence credibility fusion results, this embodiment introduces a hierarchical evidence credibility fusion classification model, including: a view feature mapping network and a view evidence output network corresponding to each view, a modal evidence network corresponding to each imaging modality, a joint evidence network, and a credibility fusion module.

[0042] First, aggregated features are used to generate view-level evidence.

[0043] The first The patient in Aggregation features under each view Input the view feature mapping network and the view evidence output network corresponding to this view: ; in, Indicates the first Feature mapping network corresponding to each view This indicates that the view is hidden. This indicates the output network of the view evidence. This indicates evidence of the non-negative category.

[0044] In specific implementation, feature mapping network This can be achieved using a two-layer fully connected network: the first fully connected layer maps the 1536-dimensional view aggregation features to 512 dimensions, which, after processing with the GELU activation function and Dropout, are mapped by the second fully connected layer to obtain a 256-dimensional view hidden representation; the evidence output network... Using a fully connected layer to map the view-hidden representation to dimensional vector, Indicates the number of carotid artery plaque types, then... The activation function output ensures that the obtained categorical evidence is non-negative. The four views correspond to their respective feature mapping networks and evidence output networks, and their network parameters are not shared.

[0045] For any source of evidence Its Dirichlet parameter Overall strength of evidence Represented as: ; Its category probability and uncertainty Represented as: ； Let the set of evidence sources participating in the current layer fusion be denoted as . ,by This indicates any one of the sources of evidence. Credibility of each source of evidence and fusion weight They are represented as follows: ; in, This indicates whether the corresponding source of evidence exists. These are pre-defined constants used to avoid zero denominators. The fused evidence is represented as follows: .

[0046] When a source of evidence is missing, its presence marker, credibility, and fusion weight are all 0, and it does not participate in the evidence fusion at the current level. When the uncertainty of a source of evidence is high, its credibility and fusion weight are reduced accordingly, thereby reducing the impact of low-credibility evidence on the fusion result.

[0047] The view sets corresponding to the conventional ultrasound modality and the contrast ultrasound modality are respectively and For any imaging modality Its existence is indicated by: ； in, This indicates the indicator function. It applies when at least one valid view exists in this imaging modality. Select 1 if the value is 1, otherwise select 0.

[0048] Following the aforementioned reliable fusion method, the original view evidence and local patch view evidence under the same imaging modality are fused, and the corresponding view hiding representations are aggregated using the same weights: ; in, This indicates preliminary modal evidence. This represents modal hiding representation.

[0049] Subsequently, the modal hiding representation is input into the corresponding modal evidence network: ; in, Represents imaging mode The corresponding modal evidence network, This represents modal embedding evidence. The modal evidence network is implemented using a fully connected layer structure with a softplus activation function, mapping the 256-dimensional modal hidden representation to K-dimensional non-negative class evidence. Finally, following the aforementioned trusted fusion method, the preliminary modal evidence is... With modal embedding evidence Re-fusion yields the first source of evidence for conventional ultrasound modalities. Second source of evidence: contrast-enhanced ultrasound modality .

[0050] It should be noted that in this embodiment, the superscript is used. Marked view-level variables (such as view evidence) Uncertainty Existence marker ) and the above marks Labeled source variables of evidence (such as) , , The objects referred to are essentially the same: views That is, the source of evidence at the view level. This is a generic notation for any source of evidence at any fusion level. Specifically refers to the source of evidence at the view level.

[0051] Finally, joint evidence is generated and fused.

[0052] Joint evidence The existence marker is represented as: .

[0053] Marking the existence of the primary source of evidence. This serves as a marker for the existence of a second source of evidence.

[0054] When the presence flag of joint evidence is set to 1, meaning both imaging modalities exist, a cross-modal interactive representation is constructed based on the hidden representations of the two imaging modalities: ; in, This represents element-wise multiplication, i.e., hiding the conventional ultrasonic modal representation. Contrast-enhanced ultrasound modal hiding representation The element-wise difference and element-wise product of the two modalities are concatenated to characterize the interaction between the two imaging modalities. This cross-modal interaction representation is then input into the joint evidence network. ; in, This represents a joint evidence network, whose structure is the same as that of a modal evidence network. When any imaging modality is missing, the presence of joint evidence is marked as 0, and the joint evidence does not participate in subsequent fusion.

[0055] Conventional ultrasound modal evidence Contrast-enhanced ultrasound modal evidence and joint evidence As the final level of evidence source, the final classified evidence is obtained according to the aforementioned credible fusion method: ; Based on the final classification evidence Calculate the corresponding Dirichlet parameters:

[0056] The overall strength of evidence is expressed as:

[0057] No. The classification probability of each category is expressed as:

[0058] The results of carotid artery ultrasound plaque classification are as follows:

[0059] in, Indicates the first The carotid artery ultrasound plaque classification results for each patient are output along with the classification probabilities for each category and displayed on the ultrasound workstation's interactive interface for physician reference.

[0060] In one embodiment, the following loss function is used for optimization during training: , In the formula, Indicates the final classification loss. Indicates view-assisted loss, Indicates modal-assisted loss, Indicates cross-modal alignment loss. , and This indicates the corresponding balance parameters, which are obtained through empirical configuration or pre-calibration.

[0061] Specifically, in order to achieve better training results, the overall training process of the model has been improved in this embodiment.

[0062] First, a patch dataset is established. During the training process of the patch segmentation model, for the imaging modality... The segmentation training data with patch mask annotations is represented as follows:

[0063] in, Indicates the first Zhang ultrasound image, This represents the corresponding true mask for the patch. This text indicates carotid artery plaque. This indicates the number of segmentation training images for the corresponding imaging modality.

[0064] Using Medical SAM3 as the basic plaque segmentation model, ultrasound images and text prompts were combined. A common input model is used. During training, most of the pre-trained parameters of the base segmentation model are frozen, and only the low-rank adapter parameters in the attention Q-projection and V-projection of the Transformer block after the visual encoder are trained, with the adaptation method as shown in the aforementioned formula. The corresponding low-rank adaptation parameters are trained for conventional ultrasound images and contrast-enhanced ultrasound images, respectively, thus obtaining the conventional ultrasound patch segmentation model and the contrast-enhanced ultrasound patch segmentation model.

[0065] For imaging modal The overall loss of the patch segmentation model is expressed as:

[0066] in, Represents imaging mode The loss of the one-to-one main matching branch is as follows. This represents the corresponding one-to-many auxiliary matching branch loss.

[0067] The loss of the one-to-one main matching branch is expressed as:

[0068] The one-to-many auxiliary matching branch loss is expressed as:

[0069] in, This represents the candidate classification loss, used to supervise the matching relationship between candidate queries and real patch targets; This represents the target existence loss, used to enhance the model's response to textual prompts about targets; and This represents the loss for predicting the bounding box location; and This represents the predicted mask loss. Since the one-to-many auxiliary matching branch is mainly used to expand the scope of candidate-level supervision, the image-level target presence loss is not calculated repeatedly.

[0070] When training the hierarchical evidence credibility fusion classification model, the process is as described above. First, the trained patch segmentation model is used to generate a patch prediction mask for each ultrasound image in the training set and crop the local patch view. Then, the view aggregation features and view presence markers of the four views are constructed. Subsequently, the view aggregation features are input into the hierarchical evidence credibility fusion classification model for forward calculation to obtain evidence at each level. Based on the evidence at each level, an overall optimization objective is constructed, and the model parameters are updated through backpropagation.

[0071] During training, class labels are converted to Unique thermal coding For any piece of evidence Its Dirichlet parameter is The overall strength of evidence is The evidence classification fitting loss is expressed as:

[0072] Construct the modified Dirichlet parameters:

[0073] Dirichlet regularization terms are represented as:

[0074] in, Represents the double gamma function. Let Beta be the function. The evidence classification loss for a single evidence source is expressed as:

[0075] in, This represents the balance parameter of the Dirichlet regularization term.

[0076] The final classification loss is expressed as:

[0077] make Represents the collection of valid views. Indicates the first Number of valid patients per view This represents the number of valid views. The view-assisted loss is expressed as:

[0078] Modal-level auxiliary supervision includes conventional ultrasound modal evidence, contrast-enhanced ultrasound modal evidence, and combined evidence. Represents the set of valid modal-level evidence sources. Indicate the source of evidence The number of effective patients This represents the number of valid modal-level evidence sources. The modal-aided loss is expressed as:

[0079] The class probabilities of conventional ultrasound modality and contrast ultrasound modality are denoted as follows: and The average probability distribution of the two is expressed as:

[0080] The JS divergence between the probabilities of the two modal categories is represented as:

[0081] in, This represents the KL divergence. Cross-modal alignment loss is calculated only for patients with both conventional and contrast-enhanced ultrasound modalities.

[0082] The overall optimization objective is expressed as: , in, , and These represent the balance parameters for view-assisted loss, modality-assisted loss, and cross-modal alignment loss, respectively. During training, missing views do not participate in the corresponding evidence fusion and loss calculation; when any imaging modality is missing, no valid joint evidence is generated, and the joint evidence does not participate in the final fusion and corresponding loss calculation.

[0083] A storage medium includes computer instructions that, when executed by a computer device, perform the aforementioned carotid artery ultrasound plaque multi-view classification method.

[0084] Those skilled in the art will understand that various aspects, or possible implementations of various aspects, of the present invention can be embodied as systems, methods, or computer program products. Therefore, various aspects, or possible implementations of various aspects, of the present invention can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, etc.), or embodiments combining software and hardware aspects, all collectively referred to herein as "circuit," "module," or "system." Furthermore, various aspects, or possible implementations of various aspects, of the present invention can take the form of computer program products, which are computer instructions stored in memory.

[0085] The memory can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, and portable read-only memory (CD-ROM).

[0086] A processor in a computer reads computer instructions stored in memory, enabling the processor to execute the functional actions specified in each step or combination of steps in a flowchart; and to generate means for implementing the functional actions specified in each block or combination of blocks in a flowchart.

[0087] It should be understood that a processor in a computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components used to execute the aforementioned computer instructions.

[0088] Computer instructions may be executed entirely on the user's local computer, partially on the user's local computer, as a separate software package, partially on the user's local computer and partially on a remote computer, or entirely on a remote computer or server. It should also be noted that in some alternative implementations, the functions indicated by the steps in the flowchart or the blocks in the block diagram may not occur in the order shown in the diagram. For example, depending on the functions involved, two consecutive steps or blocks may actually be executed approximately simultaneously, or these blocks may sometimes be executed in reverse order.

[0089] Of course, in practical applications, the various components of a computer system are coupled together through a bus system. The bus system is used to enable communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0090] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. A segmentation-guided multi-view classification method for carotid artery plaques, characterized in that, include: Step S1: For the ultrasound image group to be predicted, patch prediction is performed by adding a patch prediction mask, and then local views of the patches are established based on the patch prediction mask. The ultrasound image set includes multiple conventional ultrasound images and / or contrast-enhanced ultrasound images; Step S2: Extract and aggregate features from the local view of the patch and the corresponding original input in the ultrasound image group to obtain aggregated features; Step S3: Input the aggregated features into the fusion classification model to obtain the prediction result.

2. The carotid artery ultrasound plaque multi-view classification method according to claim 1, characterized in that, When the ultrasound image group includes conventional ultrasound images and contrast-enhanced ultrasound images, the patch local view consists of a conventional patch local view and a contrast-enhanced patch local view.

3. The carotid artery ultrasound plaque multi-view classification method according to claim 1, characterized in that, Step S1 includes: Step S11: The ultrasound image group is received sequentially using a pre-trained patch segmentation model; Step S12: Use the patch segmentation model to output the patch prediction mask and regress it to the corresponding original input; Step S13: Dilate the patch prediction mask to extract the surrounding local area and establish a local view of the patch.

4. The carotid artery ultrasound plaque multi-view classification method according to claim 3, characterized in that, The patch segmentation model is implemented using the Medical SAM3 model. In step S11, the ultrasound impact group is combined with the patch text prompt and input into the patch segmentation model; The visual encoder of the patch segmentation model has a low-rank adapter in its attention Q-projection and V-projection.

5. The carotid artery ultrasound plaque multi-view classification method according to claim 1, characterized in that, Step S2 includes: Step S21: Use an image feature extraction network to extract features from the local view of the patch and the original input to obtain feature maps; Step S22: Aggregate the feature map to obtain the aggregated feature.

6. The carotid artery ultrasound plaque multi-view classification method according to claim 5, characterized in that, In step S22, the feature map is aggregated using a feature aggregation formula: ； In the formula, This represents the aggregated features of the i-th patient in the v-th view. This indicates a feature concatenation operation. This is the feature map extracted from the j-th image corresponding to the i-th patient in the v-th view.

7. The carotid artery ultrasound plaque multi-view classification method according to claim 1, characterized in that, Step S3 includes: Step S31: Input the aggregated features sequentially into the feature mapping network and the evidence output network to obtain the corresponding evidence sources and add them to the evidence source set; Step S32: Calculate the fusion weight for each evidence source, and then fuse the first evidence source corresponding to the patch local view and the second evidence source corresponding to the original input to obtain the view hiding representation; Step S33: Fuse the view-hiding representation input modal evidence network and construct a cross-modal evidence representation; Step S34: Input the cross-modal evidence representation into the joint evidence network to obtain the final classification evidence source; Step S35: Obtain the prediction result based on the final classification evidence source prediction.

8. The carotid artery ultrasound plaque multi-view classification method according to claim 7, characterized in that, The generation methods for the fusion weights include: ； In the formula, The fusion weights of the evidence sources s for the i-th patient are given. For the set of evidence sources, The credibility of the evidence source s, Indicates whether the source of evidence s exists. The uncertainty of the evidence source s The total strength of evidence from the aforementioned source s. This represents the k-th component of the Dirichlet parameter of the evidence source s, based on Perform calculations. The evidence output network outputs non-negative class evidence for the evidence source s.

9. The carotid artery ultrasound plaque multi-view classification method according to claim 7, characterized in that, During training, the following loss function is used for optimization: , In the formula, Indicates the final classification loss. Indicates view-assisted loss, Indicates modal-assisted loss, Indicates cross-modal alignment loss. , and This represents the corresponding equilibrium parameter.

10. A storage medium comprising computer instructions, characterized in that, When the computer device executes the computer instructions, it performs the carotid artery ultrasound plaque multi-view classification method as described in any one of claims 1-9.

Images