A multi-contrast learning coronary artery high-risk plaque detection method, system and terminal

Abstract

The application belongs to the technical field of medical image processing and computer-aided diagnosis, and discloses a multi-contrast learning coronary artery high-risk plaque detection method, system and terminal. A two-dimensional image sequence is extracted as a sample along a coronary artery center line point marked by a doctor on three-dimensional medical scanning data, and a sample data set is divided. A multi-contrast learning coronary artery high-risk plaque detection network based on a Transformer is constructed by taking a coronary artery two-dimensional image sequence sample as input and whether an image contains a high-risk plaque as output, so that coronary artery high-risk plaque detection is realized. The coronary artery high-risk plaque detection method does not need to label the extracted data sample, and thus labeling errors are avoided. A multi-path twin network based on the Transformer is trained using a large amount of unlabeled data, which is helpful to improve the ability of the network to generate feature representation. Patient-level feature representation is generated through feature recoding for final prediction, so that the working time is shortened.

Description

Technical Field

[0001] This invention belongs to the fields of medical image processing and computer-aided diagnosis technology, and in particular relates to a multi-contrast learning method, system and terminal for detecting high-risk plaques in coronary arteries. Background Technology

[0002] Currently, high-risk / vulnerable plaques in coronary atherosclerosis typically exist in the early stages of plaque formation and are a major contributing factor to acute cardiovascular events. Therefore, if plaque characteristics can be analyzed in the early stages of plaque formation, doctors can develop appropriate treatment strategies based on plaques of different risk levels, thereby preventing acute cardiovascular events. Most existing studies use traditional machine learning methods to detect high-risk plaques in the coronary arteries. This involves experienced experts manually designing features, which are then fed into a machine learning classifier to predict plaque type. This process heavily relies on the operator's experience and extensive expertise and is time-consuming and labor-intensive. Therefore, using traditional machine learning methods to detect high-risk coronary plaques still has certain limitations.

[0003] Currently, deep learning has been widely applied in medical imaging fields such as brain tumor detection and cardiovascular plaque recognition. Furthermore, numerous studies have confirmed the feasibility of applying deep learning in medical imaging. Deep learning models often require a large number of labeled samples to fully extract information; however, obtaining labeled samples is extremely difficult for medical image data, and the annotation process is time-consuming. Moreover, due to the characteristics of vulnerable plaques in coronary arteries—that is, plaques containing two or more vulnerable plaque features in CCTA scans are defined as vulnerable plaques—the single-image, single-label deep learning training method is unsuitable for high-risk plaque detection tasks, as plaques typically span multiple CCTA scans. Given the diversity of data sample features, machine learning methods struggle to manually design suitable, highly discriminative features, a cumbersome process; data annotation requires experience, and deep learning methods face difficulties in data acquisition and annotation, being time-consuming and labor-intensive, and prone to errors in labeling individual images. Therefore, there is an urgent need to design a new method for detecting high-risk coronary artery plaques.

[0004] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0005] (1) Given the diversity of coronary plaque sample characteristics, machine learning methods have difficulty designing specific features for vulnerable plaques and rely heavily on the operator's experience and extensive professional knowledge, which is time-consuming and laborious.

[0006] (2) Existing deep learning model training requires a large amount of labeled data. However, it is very difficult to obtain labeled samples for medical image data, and the labeling process of the samples takes a lot of time.

[0007] (3) Due to the characteristics of vulnerable plaques in the coronary arteries, a plaque usually spans multiple CCTA scans. The deep learning training method of a single image corresponding to a single label has errors and is not suitable for high-risk plaque detection tasks. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention provides a method, system, and terminal for detecting high-risk coronary artery plaques through multi-contrast learning, and particularly relates to a method, system, medium, device, and terminal for detecting high-risk coronary artery plaques based on Transformer through multi-contrast learning.

[0009] This invention is implemented as follows: a multi-contrast learning method for detecting high-risk coronary artery plaques. The method includes: extracting two-dimensional image sequences as samples from three-dimensional medical scan data along the coronary artery centerline points marked by the doctor, and dividing the sample dataset; using the two-dimensional coronary artery image sequence samples as input and whether the image contains high-risk plaques as output, constructing a Transformer-based multi-contrast learning network for detecting high-risk coronary artery plaques, thereby achieving the detection of high-risk coronary artery plaques.

[0010] The Transformer-based multi-contrast learning coronary high-risk plaque detection network consists of two parts: the first part is a multi-path Siamese network for image sequence feature extraction, and the second part is a feature recoding and high-risk plaque detection network. Training data is fed into the multi-path Siamese network, and the loss function is calculated to train the network model. One branch of the trained multi-path Siamese network model is used as an image sequence feature extractor to extract features from all image sequences. The new features generated from each image sequence are recoded to generate feature vectors belonging to each patient. The patient-level feature vectors are fed into the machine learning model for training, and the trained network model makes predictions on the test set to complete the coronary high-risk plaque detection task.

[0011] Furthermore, comparative learning of high-risk coronary plaque detection methods includes the following steps:

[0012] Step 1: Preprocess the original medical images, including cropping the target lesion vessel segment and exporting it as a medical digital imaging and communication dataset; adjusting the window level and window width of the images; manually annotating the coronary artery centerline on the preprocessed data; extracting image sequences of length N along the coronary artery centerline points as samples; the two-dimensional image sequences of all extracted medical images constitute the sample dataset, and are divided into training set, validation set and test set;

[0013] Step 2: Using M coronary artery 2D image sequences as input and low-dimensional feature representations of different image sequences as output, construct a Transformer-based multi-path Siamese network. The network structure consists of M branches with identical structures and shared weights. Each branch contains N parallel 2D convolutional neural networks and a Transformer structure. Construct N parallel 2D convolutional neural networks to learn the feature representation of each 2D image in the image sequence and extract local information from the image sequence. Construct a Transformer structure to fuse the local feature information extracted by the N parallel 2D convolutional neural networks to obtain the global feature representation of the image sequence.

[0014] Step 3: Feed the training dataset into the Transformer-based multi-path Siamese network, traverse the training dataset and train the network, using cosine similarity loss as the network's loss function; use the backpropagation algorithm to backpropagate the error to the top convolutional layer, and use gradient descent to update the network parameters until the network converges.

[0015] Step 4: Use one branch of the trained Transformer-based multi-path Siamese network as a feature extractor for the image sequence. Input all the patient's image sequences into the branch and extract the feature representation of the corresponding image sequence.

[0016] Step 5: Feature recoding. Perform cluster analysis on all feature representations in the training set using the K-Means clustering method, and save the generated K cluster centers and the cluster category to which each feature representation belongs. Create a unique feature vector for each patient, and count the number of times each patient's feature representation belongs to each cluster category. Use this count as the weight of the corresponding position in the created feature vector. The feature vector is the patient's new feature representation.

[0017] Step 6: Use the generated patient-level feature representation as input and whether the patient has high-risk plaque disease as output to train the XGBoost classification model; feed the validation set dataset into the training model to evaluate the model performance and select the best-performing model parameters.

[0018] Step 7: Feed the test data into the trained model, and extract feature representations using a Transformer-based multi-path Siamese network; calculate the nearest cluster center for each feature representation to generate feature vectors for the patients in the test set; obtain prediction results using an XGBoost classifier to evaluate network performance.

[0019] Furthermore, step one involves preprocessing the original medical images, adjusting the window level and width, marking the coronary artery centerline, extracting two-dimensional images, and assembling them into sequences, including:

[0020] (1) Reconstruct maximum intensity projection images of coronary CCTA scans on a Syngo workstation to optimize visualization of the target lesion;

[0021] (2) In a direction roughly perpendicular to the long axis of the target lesion, slice the image from the proximal end to the distal end of the target lesion, and minimize the slice thickness and the distance between slices.

[0022] (3) Extract slice images and export them as a medical digital imaging and communication dataset;

[0023] (4) Read the CT data header information, obtain the window level and window width; adjust the pixel values ​​to the window width range and perform normalization processing to obtain the best display of blood vessels and surrounding tissues;

[0024] (5) Take the N / 2+1th point near the coronary artery centerline as the origin of the image sequence, cut out a two-dimensional image of size L×H, and cut out images of the same size from the two-dimensional slices of the N / 2 center points before and after, retain the position order of the images, and merge them into an image sequence of length N;

[0025] (6) Starting from the N / 2+1th center point of the nearest segment, and taking the N / 2+1th center point as the origin, extract the image sequence as samples;

[0026] (7) The two-dimensional image sequence of all medical images extracted along the coronary artery centerline constitutes the sample dataset;

[0027] (8) In the sample dataset, all data are divided into training set, validation set and test set in a ratio of 7:1:2.

[0028] Furthermore, in step two, the construction of a Transformer-based multi-path Siamese network, using M coronary artery two-dimensional image sequences as input and low-dimensional feature representations of different image sequences as output, includes:

[0029] (1) Using M coronary artery two-dimensional image sequences as input and the low-dimensional feature representations of different image sequences as output, a multi-channel Siamese network based on Transformer is constructed; wherein, the multi-channel Siamese network based on Transformer consists of N parallel two-dimensional convolutional neural networks and Transformer structures;

[0030] (2) Construct N parallel two-dimensional convolutional neural networks to learn the feature representation of each two-dimensional image in the image sequence and extract local information of the image sequence; the N parallel two-dimensional convolutional neural networks have the same structure and do not share weights; each two-dimensional convolutional neural network contains 5 cascaded convolutional layers, each convolutional layer uses the ReLU activation function to learn non-linear features, and the convolution compensation of each convolutional layer is set to 2.

[0031] (3) Construct a Transformer structure to fuse the local feature information extracted by N parallel two-dimensional convolutional neural networks to obtain the global feature representation of the image sequence. The Transformer structure contains 8 Transformer encoders. Each encoder in the Transformer structure contains two sub-modules, MSA and FFN. Each MSA module is followed by an FFN module. The MSA module uses multiple self-attention layers to calculate the relevant information between every two embedding block sequences. The FFN module uses two fully connected layers and a GeLU activation function to map the data from low dimension to high dimension and then back to low dimension to improve the feature representation. An additional normalization layer is inserted before each sub-module to maintain the temporal correlation between features. Finally, an additional feature vector added in the Transformer is used as the global feature representation of the image sequence.

[0032] Furthermore, step three, which involves training a Transformer-based multipath Siamese network using M-1 image sequence pairs as input, includes:

[0033] (1) Traverse the training dataset and set the traversed images as anchor images;

[0034] (2) Randomly select M-1 image sequences to form M-1 image pairs. Image sequences belonging to the same disease type are considered similar images and are labeled as 1; otherwise, they are dissimilar images and are labeled as 0.

[0035] (3) The network outputs a cosine similarity loss function between the feature representations of M-1 image pairs to evaluate the similarity between the anchored image and other images, given a feature vector v. a and v b In this case, the cosine similarity comparison loss is:

[0036]

[0037] Among them, y i Let be the label for whether the i-th image pair belongs to the same category, n be the number of samples in a batch, margin is a hyperparameter set to 0.2; cos(·) represents the cosine similarity between vectors.

[0038]

[0039] in and This represents the i-th feature of the feature vector, where n is the length of the feature vector.

[0040] The differences between the anchor image and the other M-1 images are calculated separately, and the average of these differences is used as the loss function to guide the training of the multi-path Siamese network. The final loss function is defined as follows:

[0041]

[0042] Among them l M-1 This represents the calculated loss value for each image pair.

[0043] (4) Use the backpropagation algorithm to backpropagate the error to the top convolutional layer, and use the gradient descent method to update the network parameters until the network converges.

[0044] Furthermore, step five involves using the K-Means clustering method to perform clustering analysis on the feature representations, including:

[0045] (1) Construct a dictionary using the K-Means clustering algorithm and use the k cluster centers generated by the clustering algorithm as dictionary values; calculate the category of all feature vectors in the test set based on the dictionary generated by the feature vectors of the training set, and obtain the cluster center closest to the feature vector;

[0046] (2) Based on the created feature vector dictionary, a unique feature vector is created for each patient. The length of the feature vector is determined by the number of cluster centers. Each cluster center in the created feature vector contains a weight w, where w is the number of feature vectors of the patient belonging to that cluster center category.

[0047] (3) The feature vector created for each patient is a new feature representation.

[0048] Furthermore, in step six, the generated patient-level feature representation is used as input, and the status of having high-risk plaque disease is used as output to train the XGBoost classification model, which includes:

[0049] (1) Use the generated patient-level feature representation as input and whether the patient has high-risk plaque disease as output to train the XGBoost classification model;

[0050] (2) Feed the validation set into the training model, evaluate the model performance, and select the model parameters with the best performance.

[0051] Furthermore, step seven involves feeding the test data into the trained model, where the XGBoost classifier obtains the prediction results to evaluate the network performance.

[0052] (1) The test data is fed into the trained model, and the feature representation is extracted by the Transformer-based multi-path Siamese network;

[0053] (2) Calculate the distance between the feature representation of the test set and the cluster centers generated by the training set. The nearest cluster center is the category of the corresponding feature representation, and generate a new feature vector at the patient level.

[0054] (3) Use the new patient-level feature vectors as input to the trained XGboost model to obtain prediction results and evaluate network performance.

[0055] Another object of the present invention is to provide a multi-contrast learning coronary high-risk plaque detection system that applies the aforementioned multi-contrast learning coronary high-risk plaque detection method. The multi-contrast learning coronary high-risk plaque detection system includes:

[0056] The sample data partitioning module is used to preprocess the original medical images, adjust the window level and window width of the images, extract image sequences as samples along the coronary artery centerline points, and partition the sample dataset.

[0057] The multi-path twin network construction module is used to construct a Transformer-based multi-path twin network by taking a two-dimensional coronary artery image sequence as input and the low-dimensional feature representations of different image sequences as output.

[0058] The network model training module is used to feed the training dataset into a Transformer-based multi-path Siamese network, traverse the training dataset and train the network, and extract feature representations of image sequences.

[0059] The feature recoding module is used to perform cluster analysis on all feature representations in the training set using the K-Means clustering method, and saves the generated K cluster centers and the cluster category to which each feature representation belongs;

[0060] The feature representation extraction module is used to feed the validation set dataset into the training model to evaluate the model performance, and to extract feature representations from the training dataset using a Transformer-based multi-path Siamese network.

[0061] The prediction module is used to calculate the nearest cluster center for each feature representation and generate feature vectors for patients in the test set; the prediction results are obtained by the XGBoost classifier to evaluate the network performance.

[0062] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the multi-contrast learning method for detecting high-risk coronary plaques.

[0063] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the multi-contrast learning method for detecting high-risk coronary plaques.

[0064] Another objective of this invention is to provide an information data processing terminal for implementing the aforementioned multi-contrast learning coronary high-risk plaque detection system.

[0065] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0066] First, addressing the technical problems existing in the prior art and the difficulty of solving them, this paper closely analyzes, in conjunction with the technical solution to be protected by this invention and the results and data obtained during the research and development process, how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about by solving these problems. The specific description is as follows:

[0067] This invention extracts two-dimensional image sequences as samples from three-dimensional medical scan data along the coronary artery centerline points marked by doctors, and divides the sample dataset. Using these two-dimensional coronary artery image sequence samples as input and the presence or absence of high-risk plaques as output, a Transformer-based multi-contrast learning coronary artery high-risk plaque detection network is constructed. The detection network of this invention mainly consists of two parts: a multi-path Siamese network for image sequence feature extraction, and a feature recoding and high-risk plaque detection network. Training data is first fed into the multi-path Siamese network, and a loss function is calculated to train the network model. One branch of the trained multi-path Siamese network model is used as an image sequence feature extractor to extract features from all image sequences. Then, the new features generated from each image sequence are recoded to generate feature vectors belonging to each patient. These patient-level feature vectors are then fed into the machine learning model for training. The trained network model makes predictions on the test set to complete the coronary artery high-risk plaque detection task. The coronary artery high-risk plaque detection method of this invention combines Siamese networks for multi-contrast learning, eliminating the need for pixel-level annotation for sample feature extraction. It relies solely on patient-level annotation to complete high-risk plaque detection, significantly reducing the difficulty of annotating coronary artery data. This invention achieves the detection of high-risk coronary artery plaques, characterized by not requiring pre-segmentation of coronary vessels, not relying on detailed doctor-annotated data, and high accuracy.

[0068] Second, considering the technical solution as a whole or from a product perspective, the technical effects and advantages of the technical solution to be protected by this invention are specifically described as follows:

[0069] The multi-contrast learning method for detecting high-risk coronary plaques provided by this invention eliminates the need for labeling extracted data samples, thus avoiding labeling errors. Training a Transformer-based multi-path Siamese network with a large amount of unlabeled data enhances the network's ability to generate feature representations. Using a Transformer-based multi-path Siamese network for feature extraction avoids the experience-dependent and complex process of manually designing features, automatically extracting appropriate feature representations. Furthermore, this invention generates patient-level feature representations through feature recoding for final prediction, eliminating the need for expert pixel-level labeling; only patient-level labels are required to train the high-risk plaque prediction model, significantly reducing processing time.

[0070] Third, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0071] The technical solution of this invention fills a technological gap in the industry both domestically and internationally:

[0072] Currently, there is little research in the field of automated detection of vulnerable coronary artery plaques. Addressing the difficulty of existing automated methods in distinguishing vulnerable plaques, this invention proposes a Transformer-based multi-contrast learning method for coronary artery vulnerable plaque detection. A Transformer-based multi-contrast learning structure is constructed, which can extract richer image information even with a limited number of samples. Feature recoding is used to encode image-based features into patient-level features, eliminating the need for detailed manual annotation by experts and significantly reducing algorithm construction time. Furthermore, due to the use of the multi-contrast learning structure, the model can achieve good performance without requiring a large number of samples during training. Attached Figure Description

[0073] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 This is a flowchart of the multi-contrast learning method for detecting high-risk coronary plaques provided in this embodiment of the invention;

[0075] Figure 2 This is a schematic diagram of the structure of the coronary artery high-risk plaque detection system provided in an embodiment of the present invention;

[0076] Figure 3 This is a schematic diagram of a multi-path twin network structure based on Transformer provided in an embodiment of the present invention. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0078] To address the problems existing in the prior art, the present invention provides a method, system, and terminal for detecting high-risk coronary plaques through multi-contrast learning. The present invention will be described in detail below with reference to the accompanying drawings.

[0079] like Figure 1 As shown, the multi-contrast learning method for detecting high-risk coronary plaques provided in this embodiment of the invention includes the following steps:

[0080] S101, On the original medical image, extract two-dimensional image sequences as samples along the coronary artery centerline points marked by the doctor, and divide the sample dataset;

[0081] S102 uses two-dimensional coronary artery image sequence sample pairs as input and the mean of the cosine similarity loss between all image pairs as output to construct a Transformer-based multi-path Siamese network.

[0082] S103: Feed the training dataset into the multi-path Siamese network, calculate the loss function to train the network model; use the branches of the multi-path Siamese network as feature extractors to extract feature representations of the image sequence;

[0083] S104, perform feature recoding, cluster all feature representations in the training set, and count the number of all feature representations of each patient belonging to each cluster category, which is used as the new feature representation of the patient;

[0084] S105 takes the generated patient-level feature representation as input and whether it contains high-risk plaques as output, trains the XGBoost classification model, and makes predictions on the test set to complete the detection of high-risk plaques in the coronary arteries.

[0085] As a preferred embodiment, the multi-contrast learning method for detecting high-risk coronary plaques provided in this invention specifically includes the following steps:

[0086] Step S1: Preprocess the raw medical images, including cropping the target lesion vessel segment and exporting it as a Medical Digital Imaging and Communication (DICOM) dataset; adjust the window level and width of the images to optimally display the tissue structures surrounding the vessels. Manually annotate the coronary artery centerline on the preprocessed data. To extract an image sequence of length N, the first image segment proximal to the coronary artery centerline is used as the starting point. The origin of the image sequence is a point, from which a two-dimensional image of size L×H is extracted, and points are added before and after. Images of the same size are extracted from two-dimensional slices at each of the center points, preserving the positional order of the images, and merged into an image sequence of length N. Next, the image sequence is analyzed using the nearest segment... Starting from a center point, each interval Using one center point as the origin, image sequences are extracted as samples; the two-dimensional image sequences of all extracted medical images constitute the sample dataset, which is then divided into training set, validation set, and test set.

[0087] Step S2: Using M coronary artery 2D image sequences as input and low-dimensional feature representations of different image sequences as output, construct a Transformer-based multi-path Siamese network. The network structure consists of M branches with identical structures and shared weights. Each branch contains N parallel 2D convolutional neural networks and Transformer structures (see...). Figure 3 Construct N parallel two-dimensional convolutional neural networks to learn the feature representation of each two-dimensional image in the image sequence and extract local information from the image sequence; construct a Transformer structure to fuse the local feature information extracted by the N parallel two-dimensional convolutional neural networks to obtain the global feature representation of an image sequence.

[0088] Step S3: Feed the training dataset into the constructed Transformer-based multi-path Siamese network, traverse the training dataset, train the network, and use cosine similarity loss as the network's loss function; use the backpropagation algorithm to backpropagate the error to the top convolutional layer, and use gradient descent to update the network parameters until the network converges.

[0089] Step S4: Use one branch of the trained Transformer-based multi-path Siamese network as a feature extractor for the image sequence. Input all the patient's image sequences into this branch to obtain the corresponding feature representations.

[0090] Step S5: Perform cluster analysis on all feature representations in the training set using the K-Means clustering method, and save the generated K cluster centers and the cluster category to which each feature representation belongs; create a unique feature vector for each patient, and count the number of times each patient's feature representation belongs to each cluster category, which is used as the weight of the corresponding position of the created feature vector. This feature vector is the new feature representation of the patient.

[0091] Step S6: Take the generated patient-level feature representation as input and whether the patient has high-risk plaque disease as output, train the XGBoost classification model, feed the validation set dataset into the training model, evaluate the model performance, and select the model parameters with the best performance.

[0092] Step S7: Feed the test data into the trained model, and extract feature representations using a Transformer-based multi-path Siamese network; calculate the nearest cluster center for each feature representation to generate feature vectors for the patients in the test set; obtain prediction results using an XGBoost classifier to evaluate network performance.

[0093] In step S1 of this embodiment of the invention, the original medical image is preprocessed, including cropping the target lesion vessel segment and exporting it as a Medical Digital Imaging and Communication (DICOM) dataset; adjusting the window level and window width of the image. The coronary artery centerline is manually annotated on the preprocessed data, and two-dimensional images are extracted and assembled into sequences, specifically according to the following steps:

[0094] (1) First, reconstruct the maximum intensity projection image of the coronary CCTA scan on the Syngo workstation to optimize the visualization of the target lesion;

[0095] (2) In a direction roughly perpendicular to the long axis of the target lesion, slice the image from the proximal end to the distal end of the target lesion, and minimize the slice thickness and the distance between slices.

[0096] (3) Extract these slice images and export them as a Medical Digital Imaging and Communications (DICOM) dataset;

[0097] (4) Read the CT data header information, obtain the window level and window width, adjust the CT pixel values ​​to the window width range, and perform normalization processing to obtain the best display of blood vessels and surrounding tissues;

[0098] (5) The first one near the coronary artery centerline The origin of the image sequence is a point, from which a two-dimensional image of size L×H is extracted, and points are added before and after. Images of the same size are extracted from two-dimensional slices at each center point, and the positional order of the images is preserved. These images are then merged into an image sequence of length N.

[0099] (6) The first of the recent period Starting from a center point, each interval Using one center point as the origin, extract the image sequence as samples;

[0100] (7) The two-dimensional image sequence of all medical images extracted along the coronary artery centerline constitutes the sample dataset;

[0101] (8) In the sample dataset, all data are divided into training set, validation set and test set in a ratio of 7:1:2.

[0102] In step S2 of this embodiment of the invention, M two-dimensional coronary artery image sequences are used as input, and low-dimensional feature representations of different image sequences are used as output to construct a multi-path Siamese network based on Transformer. The specific steps are as follows:

[0103] (1) Using M coronary artery two-dimensional image sequences as input and the low-dimensional feature representations of different image sequences as output, a multi-path Siamese network based on Transformer is constructed. The network mainly consists of N parallel two-dimensional convolutional neural networks and Transformer structures.

[0104] (2) Construct N parallel two-dimensional convolutional neural networks to learn the feature representation of each two-dimensional image in the image sequence and extract local information of an image sequence; the N parallel two-dimensional convolutional neural networks have the same structure and do not share weights; each two-dimensional convolutional neural network contains 5 cascaded convolutional layers, each convolutional layer uses the ReLU activation function to learn non-linear features, and the convolution compensation of each convolutional layer is set to 2, which can directly perform feature dimensionality reduction and reduce feature loss.

[0105] (3) A Transformer structure is constructed to fuse the local feature information extracted by N parallel two-dimensional convolutional neural networks to obtain a global feature representation of an image sequence. The Transformer structure contains eight Transformer encoders, each with two sub-modules: Multi-head self-attention (MSA) and Feedforward Network (FFN). Each MSA module is followed by an FFN module. The MSA module uses multiple self-attention layers to compute the correlation between every two embedding block sequences. The FFN module improves the feature representation by mapping the data from low dimension to high dimension and then back to low dimension using two fully connected layers and a GeLU activation function. To maintain the temporal correlation between features, an additional normalization (LN) layer is inserted before each sub-module. Finally, an additional feature vector added in the Transformer is used as the global feature representation of the image sequence.

[0106] Step S3 of this embodiment of the invention, which involves using M-1 image sequence pairs as input to train a Transformer-based multipath Siamese network, is specifically performed according to the following steps:

[0107] (1) Traverse the training dataset and set the traversed images as anchor images;

[0108] (2) Randomly select M-1 image sequences to form M-1 image pairs. Image sequences belonging to the same disease type are considered similar images with a label of 1, and otherwise, they are dissimilar images with a label of 0.

[0109] (3) The network outputs a cosine similarity loss function between the feature representations of M-1 image pairs to evaluate the similarity between the anchored image and other images, given a feature vector v. a and v b In this case, the cosine similarity comparison loss is:

[0110]

[0111] Here, margin is a hyperparameter set to 0.2; cos(·) represents the cosine similarity between vectors, expressed as:

[0112]

[0113] The differences between the anchor image and the other M-1 images are calculated separately, and the average of these differences is used as the loss function to guide the training of the multi-channel Siamese network. The final loss function is defined as:

[0114]

[0115] The mean of is used as the final loss function of the network;

[0116] (4) Use the backpropagation algorithm to backpropagate the error to the top convolutional layer, and use the gradient descent method to update the network parameters until the network converges.

[0117] In step S4 of this embodiment of the invention, a branch of the trained Transformer-based multi-path Siamese network is used as a feature extractor for the image sequence to extract the feature representation of the image sequence.

[0118] In step S5 of this embodiment of the invention, the K-Means clustering method is used to perform cluster analysis on all feature representations in the training set, saving the generated K cluster centers and the cluster category to which each feature representation belongs; a unique feature vector is created for each patient, and the number of times each patient's feature representation belongs to each cluster category is counted, which is used as the weight of the corresponding position of the created feature vector. This feature vector is the new feature representation of the patient. Specifically, the following steps are performed:

[0119] (1) Use the K-Means clustering algorithm to construct a dictionary and use the k cluster centers generated by the clustering algorithm as dictionary values; based on the dictionary generated by the feature vectors of the training set, calculate the category of all feature vectors in the test set, that is, the cluster center closest to the feature vector;

[0120] (2) Based on the created feature vector dictionary, create a unique feature vector for each patient. The length of the feature vector is determined by the number of cluster centers. Each cluster center in the created feature vector has a weight w, where w is the number of feature vectors of the patient belonging to that cluster center category.

[0121] (3) The feature vector created for each patient is its new feature representation;

[0122] In step S6 of this embodiment of the invention, the generated patient-level feature representation is used as input, and whether the patient has high-risk plaque disease is used as output to train the XGBoost classification model, which is performed according to the following steps:

[0123] (1) Take the generated patient-level feature representation as input and whether the patient has high-risk plaque disease as output to train the XGBoost classification model;

[0124] (2) Feed the validation set into the training model, evaluate the model performance, and select the model parameters with the best performance.

[0125] In step S7 of this embodiment of the invention, the test data is fed into the trained model, and feature representations are extracted by a Transformer-based multi-path Siamese network; the nearest cluster center for each feature representation is calculated to generate feature vectors for the patients in the test set; and the prediction results are obtained by an XGBoost classifier to evaluate the network performance. Specifically, the following steps are performed:

[0126] (1) The test data is fed into the trained model, and the feature representation is extracted by the Transformer-based multi-path Siamese network;

[0127] (2) Calculate the distance between the feature representation of the test set and the cluster center generated by the training set. The cluster center closest to it is the category of the corresponding feature representation, thereby generating a new feature vector at the patient level.

[0128] (3) Use the new patient-level feature vector as the input of the trained XGboost model to obtain the prediction results and evaluate the network performance.

[0129] like Figure 2 As shown, the multi-contrast learning coronary high-risk plaque detection system provided in this embodiment of the invention includes:

[0130] The sample data partitioning module is used to preprocess the original medical images, adjust the window level and window width of the images, extract image sequences as samples along the coronary artery centerline points, and partition the sample dataset.

[0131] The multi-path twin network construction module is used to construct a Transformer-based multi-path twin network by taking a two-dimensional coronary artery image sequence as input and the low-dimensional feature representations of different image sequences as output.

[0132] The network model training module is used to feed the training dataset into a Transformer-based multi-path Siamese network, traverse the training dataset and train the network, and extract feature representations of image sequences.

[0133] The feature recoding module is used to perform cluster analysis on all feature representations in the training set using the K-Means clustering method, and saves the generated K cluster centers and the cluster category to which each feature representation belongs;

[0134] The feature representation extraction module is used to feed the validation set dataset into the training model to evaluate the model performance, and to extract feature representations from the training dataset using a Transformer-based multi-path Siamese network.

[0135] The prediction module is used to calculate the nearest cluster center for each feature representation and generate feature vectors for patients in the test set; the prediction results are obtained by the XGBoost classifier to evaluate the network performance.

[0136] To demonstrate the inventiveness and technical value of the technical solution of this invention, this section provides specific product or related technology application examples of the technical solution claimed.

[0137] This invention patent, using a proprietary algorithm, is applied to an automated coronary artery vulnerable plaque detection system. The system accepts three-dimensional medical scan data as input, extracts two-dimensional image sequences along the coronary artery centerline, and uses a branch of a Transformer-based multi-contrast learning method to extract the patient's sequence features. Then, all patient features are re-encoded into patient-level features. These patient-level features are then fed into a trained machine learning classification model to obtain prediction input. Finally, the automated coronary artery detection system outputs the patient's predicted category: vulnerable plaque or stable plaque.

[0138] This invention patent, using a proprietary algorithm, is applied to an automated coronary artery vulnerable plaque detection terminal. The terminal accepts three-dimensional medical scan data as input, extracts a two-dimensional image sequence along the coronary artery centerline, and uses a branch of a Transformer-based multi-contrast learning method to extract the patient's sequence features. Then, all patient features are re-encoded into patient-level features. These patient-level features are then fed into a trained machine learning classification model to obtain prediction input. Finally, the automated coronary artery detection terminal outputs the patient's predicted category: vulnerable plaque or stable plaque.

[0139] As a preferred embodiment, the multi-contrast learning method for detecting high-risk coronary plaques provided in this invention specifically includes the following steps:

[0140] (1) On the original medical images, preprocessing is performed, including cropping the target lesion vessel segment and exporting it as a Medical Digital Imaging and Communication (DICOM) dataset; adjusting the window level and window width of the image. The coronary artery centerline is manually annotated on the preprocessed data, and two-dimensional images are extracted and assembled into sequences;

[0141] The specific steps are as follows:

[0142] (1.1) On the Syngo workstation, the maximum intensity projection image of the coronary CCTA scan was first reconstructed to optimize the visualization of the target lesion;

[0143] (1.2) In a direction roughly perpendicular to the long axis of the target lesion, slice the image from the proximal end to the distal end of the target lesion, and minimize the slice thickness and the distance between slices;

[0144] (1.3) Extract these slice images and export them as a Medical Digital Imaging and Communications (DICOM) dataset;

[0145] (1.4) Read the CT data header information, obtain the window level and window width, adjust the CT pixel values ​​to the window width range, and perform normalization processing to obtain the best display of blood vessels and surrounding tissues;

[0146] (1.5) In order to extract an image sequence of length 7, the fourth point near the coronary artery centerline is taken as the origin of the image sequence. A two-dimensional image of size 121×121 is extracted, and images of the same size are extracted from the two-dimensional slices of the three center points in front and behind. The positional order of the images is preserved and they are merged into an image sequence of length 7.

[0147] (1.6) Starting from the 4th center point of the nearest segment, extract the image sequence as samples every 4 center points as the origin;

[0148] (1.7) The two-dimensional image sequence of all medical images extracted along the coronary artery centerline constitutes the sample dataset;

[0149] (1.8) In the sample dataset, all data are divided into training set, validation set and test set in a ratio of 7:1:2.

[0150] (2) Using three coronary artery two-dimensional image sequences as input and low-dimensional feature representations of different image sequences as output, a multi-channel Siamese network based on Transformer is constructed, specifically according to the following steps:

[0151] (2.1) Using three coronary artery two-dimensional image sequences as input and the low-dimensional feature representations of different image sequences as output, a multi-channel Siamese network based on Transformer is constructed. The network mainly consists of seven parallel two-dimensional convolutional neural networks and Transformer structures.

[0152] (2.2) Seven parallel two-dimensional convolutional neural networks are constructed to learn the feature representation of each two-dimensional image in the image sequence and extract the local information of an image sequence. The seven parallel two-dimensional convolutional neural networks have the same structure and do not share weights. Each two-dimensional convolutional neural network contains five cascaded convolutional layers. Each convolutional layer uses the ReLU activation function to learn non-linear features. The kernel size of each convolutional layer is [3, 3], the convolution stride is set to 2, and the number of convolutional kernels are 16, 32, 64, 128, and 256, respectively.

[0153] (2.3) A Transformer structure is constructed to fuse the local feature information extracted by seven parallel two-dimensional convolutional neural networks to obtain a global feature representation of an image sequence. The Transformer structure contains eight Transformer encoders, each with two sub-modules: Multi-head self-attention (MSA) and Feedforward Network (FFN). Each MSA module is followed by an FFN module. The MSA module uses multiple self-attention layers to compute the correlation between every two embedding block sequences. The FFN module improves the feature representation by mapping the data from low dimension to high dimension and then back to low dimension using two fully connected layers and a GeLU activation function. To maintain the temporal correlation between features, an additional normalization (LN) layer is inserted before each sub-module. Finally, an additional feature vector added in the Transformer is used as the global feature representation of the image sequence.

[0154] (3) Use the two image sequence pairs as input to train the Siamese network for the feature recoding module based on Transformer, specifically according to the following steps:

[0155] (3.1) Traverse the training dataset and set the traversed images as anchor images;

[0156] (3.2) Randomly select two image sequences and form two image pairs with the anchor image. Image sequences belonging to the same disease type are considered similar images with a label of 1, and otherwise are considered dissimilar images with a label of 0.

[0157] (3.3) The network outputs a cosine similarity loss function between the feature representations of two image pairs to evaluate the similarity between the anchored image and other images, given a feature vector v.a and v b In this case, the cosine similarity comparison loss is:

[0158]

[0159] Here, margin is a hyperparameter set to 0.2; cos(·) represents the cosine similarity between vectors, expressed as:

[0160]

[0161] The differences between the anchor image and the other two images are calculated separately, and the average of these differences is used as the loss function to guide the training of the Siamese network. The final loss function is defined as follows:

[0162]

[0163] (3.4) The backpropagation algorithm is used to backpropagate the error to the top convolutional layer, and the gradient descent method is used to update the network parameters until the network converges.

[0164] (4) Use a branch of the trained Transformer-based feature recoding module Siamese network as a feature extractor for the image sequence to extract the feature representation of the image sequence.

[0165] (5) Feature recoding: K-Means clustering method is used to cluster all feature representations in the training set, and 40 cluster centers and the cluster category to which each feature representation belongs are saved; a unique feature vector is created for each patient, and the number of feature representations of each patient belonging to each cluster category is counted as the weight of the corresponding position of the created feature vector. This feature vector is the new feature representation of the patient.

[0166] The specific steps are as follows:

[0167] (5.1) Construct a dictionary using the K-Means clustering algorithm and use the 40 cluster centers generated by the clustering algorithm as dictionary values; calculate the category of all feature vectors in the test set based on the dictionary generated from the feature vectors of the training set, i.e. the cluster center closest to the feature vector;

[0168] (5.2) Based on the created feature vector dictionary, create a unique feature vector for each patient. The length of the feature vector is determined by the number of cluster centers. Each cluster center in the created feature vector has a weight w, where w is the number of feature vectors of the patient belonging to that cluster center category.

[0169] (5.3) The feature vector created for each patient is its new feature representation.

[0170] (6) Using the generated patient-level feature representation as input and whether the patient has high-risk plaque disease as output, train the XGBoost classification model, specifically following these steps:

[0171] (6.1) The generated patient-level feature representation is used as input and whether the patient has high-risk plaque disease is used as output to train the XGBoost classification model;

[0172] (6.2) Feed the validation set into the training model, evaluate the model performance, and select the model parameters with the best performance.

[0173] (7) Feed the test data into the trained model, and use the XGBoost classifier to obtain the prediction results. Evaluate the network performance by following these steps:

[0174] (7.1) The test data is fed into the trained model, and the feature representation is extracted by the Transformer-based multi-path Siamese network;

[0175] (7.2) Calculate the distance between the feature representation of the test set and the cluster center generated by the training set. The cluster center with the closest distance is the category of the corresponding feature representation, thereby generating a new feature vector at the patient level.

[0176] (7.3) Use the new patient-level feature vectors as input to the trained XGboost model to obtain prediction results and evaluate network performance.

[0177] The evaluation criteria for the Transformer-based multi-contrast learning coronary high-risk plaque detection method provided in this embodiment of the invention—sensitivity, specificity, area under the ROC curve, and accuracy—are defined as follows:

[0178]

[0179]

[0180]

[0181]

[0182] Where TP represents the number of true positive samples predicted as positive samples, FP represents the number of true negative samples predicted as positive samples, TN represents the number of true negative samples predicted as negative samples, FN represents the number of true positive samples predicted as negative samples, P represents the number of true positive samples, and N represents the number of true negative samples. This represents the index of the i-th sample, sorted by probability score from smallest to largest; This indicates that only the serial numbers of positive samples are summed. The evaluation index results provided in this embodiment of the invention are all between 0 and 1; the closer to 1, the better the detection result. Test results on the test set show an area under the ROC curve of 0.82, an accuracy of 0.77, a sensitivity of 0.83, and a specificity of 0.75.

[0183] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0184] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-contrast learning method for detecting high-risk coronary plaques, characterized in that, The multi-contrast learning method for detecting high-risk coronary plaques includes: extracting two-dimensional image sequences as samples from three-dimensional medical scan data along the coronary artery centerline points marked by doctors, and dividing the sample dataset; using the two-dimensional image sequence samples of the coronary arteries as input and whether the image contains high-risk plaques as output, constructing a Transformer-based multi-contrast learning network for detecting high-risk coronary plaques, and realizing the detection of high-risk coronary plaques; The Transformer-based multi-contrast learning coronary high-risk plaque detection network consists of two parts: the first part is a multi-path Siamese network for image sequence feature extraction, and the second part is a feature recoding and high-risk plaque detection network. Training data is fed into the multi-path Siamese network, and the loss function is calculated to train the network model. One branch of the trained multi-path Siamese network model is used as an image sequence feature extractor to extract features from all image sequences. The new features generated from each image sequence are recoded to generate a feature vector belonging to each patient. The patient-level feature vector is fed into the machine learning model for training, and the trained network model makes predictions on the test set to complete the coronary high-risk plaque detection task. Multi-comparison learning of high-risk coronary plaque detection methods includes the following steps: Step 1: Preprocess the original medical images, including cropping the target lesion vessel segment and exporting it as a medical digital imaging and communication dataset; adjusting the window level and window width of the images; manually annotating the coronary artery centerline on the preprocessed data; extracting image sequences of length N along the coronary artery centerline points as samples; the two-dimensional image sequences of all extracted medical images constitute the sample dataset, and are divided into training set, validation set and test set; Step 2: Using M coronary artery 2D image sequences as input and low-dimensional feature representations of different image sequences as output, construct a Transformer-based multi-path Siamese network. The network structure consists of M branches with identical structures and shared weights. Each branch contains N parallel 2D convolutional neural networks and a Transformer structure. Construct N parallel 2D convolutional neural networks to learn the feature representation of each 2D image in the image sequence and extract local information from the image sequence. Construct a Transformer structure to fuse the local feature information extracted by the N parallel 2D convolutional neural networks to obtain the global feature representation of the image sequence. Step 3: Feed the training dataset into the Transformer-based multi-path Siamese network, traverse the training dataset and train the network, using cosine similarity loss as the network's loss function; use the backpropagation algorithm to backpropagate the error to the top convolutional layer, and use gradient descent to update the network parameters until the network converges. Step 4: Use one branch of the trained Transformer-based multi-path Siamese network as a feature extractor for the image sequence. Input all the patient's image sequences into the branch and extract the feature representation of the corresponding image sequence. Step 5: Feature recoding. Perform cluster analysis on all feature representations in the training set using the K-Means clustering method, and save the generated K cluster centers and the cluster category to which each feature representation belongs. Create a unique feature vector for each patient, and count the number of times each patient's feature representation belongs to each cluster category. Use this count as the weight of the corresponding position in the created feature vector. The feature vector is the patient's new feature representation. Step 6: Use the generated patient-level feature representation as input and whether the patient has high-risk plaque disease as output to train the XGBoost classification model; feed the validation set dataset into the training model to evaluate the model performance and select the best-performing model parameters. Step 7: Feed the test data into the trained model, and extract feature representations using a Transformer-based multi-path Siamese network; calculate the nearest cluster center for each feature representation to generate feature vectors for the patients in the test set; obtain prediction results using an XGBoost classifier to evaluate network performance.

2. The multi-contrast learning method for detecting high-risk coronary plaques as described in claim 1, characterized in that, Step one involves preprocessing the original medical images, adjusting the window level and width, marking the coronary artery centerline, extracting two-dimensional images, and assembling them into sequences. (1) Reconstruct maximum intensity projection images of coronary CCTA scans on a Syngo workstation to optimize visualization of the target lesion; (2) Slice the image from the proximal end to the distal end of the target lesion in a direction approximately perpendicular to the long axis of the target lesion, and minimize the slice thickness and the distance between slices; (3) Extract slice images and export them as a medical digital imaging and communication dataset; (4) Read the CT data header information, obtain the window level and window width; adjust the pixel values ​​to the window width range and perform normalization processing to obtain the best display of blood vessels and surrounding tissues; (5) Take the N / 2+1th point near the coronary artery centerline as the origin of the image sequence, cut out a two-dimensional image of size L×H, and cut out images of the same size from the two-dimensional slices of the N / 2 center points before and after, retain the position order of the images, and merge them into an image sequence of length N; (6) Starting from the N / 2+1th center point of the nearest segment, extract the image sequence as samples every N / 2+1 center points as the origin; (7) The two-dimensional image sequence of all medical images extracted along the coronary artery centerline constitutes the sample dataset; (8) In the sample dataset, all data are divided into training set, validation set and test set in a ratio of 7:1:

2.

3. The multi-contrast learning method for detecting high-risk coronary plaques as described in claim 1, characterized in that, Step two involves using M coronary artery two-dimensional image sequences as input and low-dimensional feature representations of different image sequences as output to construct a Transformer-based multi-path Siamese network, including: (1) Using M coronary artery two-dimensional image sequences as input and the low-dimensional feature representations of different image sequences as output, a multi-channel Siamese network based on Transformer is constructed; wherein, the multi-channel Siamese network based on Transformer consists of N parallel two-dimensional convolutional neural networks and Transformer structures; (2) Construct N parallel two-dimensional convolutional neural networks to learn the feature representation of each two-dimensional image in the image sequence and extract the local information of the image sequence; the N parallel two-dimensional convolutional neural networks have the same structure and do not share weights; each two-dimensional convolutional neural network contains 5 cascaded convolutional layers, each convolutional layer uses the ReLU activation function to learn non-linear features, and the convolution compensation of each convolutional layer is set to 2. (3) Construct a Transformer structure to fuse the local feature information extracted by N parallel two-dimensional convolutional neural networks to obtain the global feature representation of the image sequence. The Transformer structure contains 8 Transformer encoders. Each encoder in the Transformer structure contains two sub-modules, MSA and FFN. Each MSA module is followed by an FFN module. The MSA module uses multiple self-attention layers to calculate the relevant information between each pair of embedding block sequences. The FFN module uses two fully connected layers and a GeLU activation function to map the data from low dimension to high dimension and then back to low dimension to improve the feature representation. An additional normalization layer is inserted before each sub-module to maintain the temporal correlation between features. Finally, an additional feature vector added in the Transformer is used as the global feature representation of the image sequence.

4. The multi-contrast learning method for detecting high-risk coronary plaques as described in claim 1, characterized in that, Step three, which involves training a Transformer-based multi-path Siamese network using M-1 image sequence pairs as input, includes: (1) Traverse the training dataset and set the traversed images as anchor images; (2) Randomly select M-1 image sequences to form M-1 image pairs. Image sequences belonging to the same disease type are considered similar images and are labeled as 1; otherwise, they are dissimilar images and are labeled as 0. (3) The network outputs a cosine similarity loss function between the feature representations of M-1 image pairs to evaluate the similarity between the anchored image and other images, given a feature vector. and In this case, the cosine similarity comparison loss is: ； in, Let be the label for whether the i-th image pair belongs to the same category, n be the number of samples in a batch, margin is a hyperparameter set to 0.2; cos(·) represents the cosine similarity between vectors. ； in and This represents the i-th feature of the feature vector; The differences between the anchor image and the other M-1 images are calculated separately, and the average of these differences is used as the loss function to guide the training of the multi-path Siamese network. The final loss function is defined as follows: ； in This represents the calculated loss value for each image pair; (4) Use the backpropagation algorithm to backpropagate the error to the top convolutional layer, and use the gradient descent method to update the network parameters until the network converges.

5. The multi-contrast learning method for detecting high-risk coronary plaques as described in claim 1, characterized in that, Step five involves using the K-Means clustering method to perform clustering analysis on the feature representations, including: (1) Use the K-Means clustering algorithm to construct a dictionary and use the k cluster centers generated by the clustering algorithm as dictionary values; calculate the category of all feature vectors in the test set based on the dictionary generated by the feature vectors of the training set, and obtain the cluster center closest to the feature vector; (2) Based on the created feature vector dictionary, a unique feature vector is created for each patient. The length of the feature vector is determined by the number of cluster centers. Each cluster center in the created feature vector contains a weight w. (3) The feature vector created for each patient is a new feature representation; Step six, which uses the generated patient-level feature representation as input and whether the patient has high-risk plaque disease as output, to train the XGBoost classification model, includes: using the generated patient-level feature representation as input and whether the patient has high-risk plaque disease as output to train the XGBoost classification model; feeding the validation set dataset into the training model to evaluate the model performance and selecting the best-performing model parameters; Step seven involves feeding the test data into the trained model, where the XGBoost classifier obtains the prediction results to evaluate the network performance. (1) The test data is fed into the trained model, and the feature representation is extracted by the Transformer-based multi-path Siamese network; (2) Calculate the distance between the feature representation of the test set and the cluster centers generated by the training set. The nearest cluster center is the category of the corresponding feature representation, and generate a new feature vector at the patient level. (3) Use the new patient-level feature vector as the input to the trained XGboost model to obtain the prediction results and evaluate the network performance.

6. A multi-contrast learning coronary high-risk plaque detection system applying the multi-contrast learning coronary high-risk plaque detection method as described in any one of claims 1 to 5, characterized in that, Multi-contrast learning coronary artery high-risk plaque detection systems include: The sample data partitioning module is used to preprocess the original medical images, adjust the window level and window width of the images, extract image sequences as samples along the coronary artery centerline points, and partition the sample dataset. The multi-path twin network construction module is used to construct a Transformer-based multi-path twin network by taking a two-dimensional coronary artery image sequence as input and the low-dimensional feature representations of different image sequences as output. The network model training module is used to feed the training dataset into a Transformer-based multi-path Siamese network, traverse the training dataset and train the network, and extract feature representations of image sequences. The feature recoding module is used to perform cluster analysis on all feature representations in the training set using the K-Means clustering method, and saves the generated K cluster centers and the cluster category to which each feature representation belongs; The feature representation extraction module is used to feed the validation set dataset into the training model to evaluate the model performance, and to extract feature representations from the training dataset using a Transformer-based multi-path Siamese network. The prediction module is used to calculate the nearest cluster center for each feature representation and generate feature vectors for patients in the test set; the prediction results are obtained by the XGBoost classifier to evaluate the network performance.

7. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the multi-contrast learning method for detecting high-risk coronary plaques as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the multi-contrast learning method for detecting high-risk coronary plaques as described in any one of claims 1 to 5.

9. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the multi-contrast learning coronary high-risk plaque detection system as described in claim 6.

Images