A Method and System for Knee Meniscus Injury Detection Integrating Multi-Sequence and Contrastive Learning
By integrating multiple sequence and contrastive learning methods, and utilizing mutual information affine transformation and YOLOv8 detection model, knee meniscus injury detection is performed. This solves the problems of scarce labeled data and insufficient utilization of multiple sequence information, and achieves efficient and accurate detection of early meniscus injury.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIANGSU SHIYU INTELLIGENT MEDICAL TECH CO LTD
- Filing Date
- 2026-02-28
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies for diagnosing knee meniscus injuries suffer from a lack of high-quality labeled data, insufficient utilization of multi-sequence information, and difficulty in detecting subtle injuries, resulting in long diagnostic cycles, high subjectivity, and a high risk of missed diagnoses.
A method combining multiple sequences and contrastive learning is adopted. Image registration is performed using the mutual information affine transformation algorithm to construct a standardized training dataset. Feature extraction and fusion are performed using contrastive learning and the YOLOv8 detection model. Combined with an adaptive feature fusion module and a multi-task loss function, accurate localization and classification of meniscus injuries are achieved.
It improves the model's generalization ability and robustness in small sample scenarios, enhances the sensitivity and localization accuracy of early meniscus injuries, significantly reduces the false positive rate, and achieves efficient detection of subtle injuries.
Smart Images
Figure CN121767758B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image processing and computer-aided diagnosis technology, and in particular relates to a method and system for detecting knee meniscus injuries by integrating multiple sequences and contrastive learning. Background Technology
[0002] Currently, meniscus injury is a common musculoskeletal disease, and accurate diagnosis is crucial for treatment selection and patient prognosis. Traditional diagnosis of meniscus injuries relies primarily on clinicians' experience and manual analysis of knee MRI images. However, this method suffers from long diagnostic cycles, high subjectivity, and a high risk of missed diagnoses. In recent years, computer-aided diagnostic techniques based on deep learning have provided new solutions for automated analysis of medical images. However, existing methods largely suffer from the following limitations: First, high-quality annotated medical image data is scarce and expensive to acquire, leading to overfitting and insufficient generalization ability in small sample scenarios. Second, knee MRI typically includes multiple sequences such as T1-weighted, T2-weighted, and proton density, each providing complementary pathophysiological information. Existing methods often use single sequences or simple sequence splicing, failing to achieve intelligent and adaptive fusion of information between different sequences, thus limiting further improvement in model performance. Finally, meniscus injuries, especially early grade I and II signal changes, show subtle differences in imaging manifestations compared to normal tissue or other lesions. Models struggle to learn highly discriminative features, resulting in low detection rates and high false-positive rates for subtle lesions. Therefore, there is an urgent need for an intelligent detection method that can fully utilize multi-sequence MRI information, overcome the limitations of small sample data, and accurately capture subtle pathological features of the meniscus. Summary of the Invention
[0003] In view of this, the present invention aims to propose a method and system for detecting knee meniscus injuries that integrates multiple sequences and contrastive learning, so as to solve at least one of the problems existing in the prior art.
[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows:
[0005] In a first aspect, the present invention provides a method for detecting knee meniscus injury that integrates multiple sequences and contrastive learning, comprising:
[0006] S1. Acquire multi-sequence MRI training images and images to be detected of the knee joint;
[0007] S2. Preprocess and register the multi-sequence MRI training images, and perform registration based on the affine transformation algorithm of mutual information to achieve spatial alignment of different sequences at the same anatomical level and construct a standardized training dataset.
[0008] S3. Input the preprocessed multi-sequence MRI training images into a feature coding network based on contrastive learning for feature learning, obtain pre-trained weights, and optimize the parameters of the feature coding network using contrastive loss.
[0009] S4. Construct a multi-sequence fusion detection model based on YOLOv8 and load pre-trained weights; use labeled data to optimize the parameters of the multi-sequence fusion detection model through the detection loss function;
[0010] S5. Input the multi-sequence MRI images of the knee joint to be detected into the optimized multi-sequence fusion detection model to obtain the localization and classification results of the injury.
[0011] Furthermore, in step S2, the multi-sequence MRI training images are preprocessed and registered. Registration is performed using an affine transformation algorithm based on mutual information to achieve spatial alignment of different sequences at the same anatomical level, thus constructing a standardized training dataset, including:
[0012] The acquired MRI volume data, including sagittal T1-weighted, T2-weighted, and proton density sequences, were used as input. First, the images of all sequences were scaled to a fixed size using bilinear interpolation. Then, pixel values were standardized using Z-Score normalization to ensure that the pixel value distribution conformed to a normal distribution with a mean of 0 and a standard deviation of 1.
[0013] Secondly, the key multi-sequence image registration operation is performed. The PD sequence image with clear anatomical structure is used as the fixed reference image, and the T1 sequence and T2 sequence are used as moving images respectively. The spatial transformation matrix from the moving image to the fixed image is calculated by using the affine transformation algorithm based on mutual information, and the transformation is applied to the entire moving image sequence. Finally, T1, T2 and PD sequence image pairs that are accurately aligned in anatomical space are obtained, which constitute a standardized training dataset.
[0014] Furthermore, in step S3, the preprocessed multi-sequence MRI training images are input into a feature encoding network based on contrastive learning for feature learning to obtain pre-trained weights. The parameters of the feature encoding network are then optimized using contrastive loss, including:
[0015] The acquired training images are used to construct a training set. Random data augmentation is applied to each training image in the training set to generate multiple images with different views. Positive sample pairs from the same image and negative sample pairs from different images are formed based on the generated images with different views. The positive and negative sample pairs are input into a feature encoding network based on contrastive learning to learn image features.
[0016] Data augmentation operations include random color jitter, Gaussian blur, Gaussian noise injection, and elastic deformation.
[0017] Furthermore, in step S3, the feature encoding network based on contrastive learning sequentially includes a YOLOv8 backbone network, a global average pooling layer, and a projection head; random data augmentation is applied to the training images to generate positive sample pairs, and the contrastive loss function drives the model to bring the positive sample pairs closer in the feature space, while pushing them further away from the negative samples.
[0018] Furthermore, in step S4, the YOLOv8-based multi-sequence fusion detection model sequentially includes a multi-branch backbone network, an adaptive feature fusion module, a YOLOv8 neck section, and a YOLOv8 head section; wherein,
[0019] The multi-branch backbone network consists of three YOLOv8 backbone network branches with identical structures and shared parameters, which extract features from the registered T1, T2 and PD sequence input images respectively. When building the model, each branch synchronously loads the weights obtained from the first stage of contrastive learning pre-training to ensure that the network has a robust general feature representation of the knee joint from the beginning.
[0020] An adaptive feature fusion module is used to perform multi-sequence fusion on the features output by the multi-branch backbone network;
[0021] The YOLOv8 neck is used to receive fused features and perform multi-scale fusion, outputting an enhanced multi-scale feature map.
[0022] The YOLOv8 header is used to perform parallel classification and regression on the enhanced multi-scale feature map, and output the bounding box, category and confidence score of the meniscus injury.
[0023] Furthermore, the workflow of the adaptive feature fusion module is as follows: feature maps from different sequences in the same layer are concatenated by channels; the concatenated features are input into a channel attention module to generate an attention weight vector corresponding to the number of sequences; the original feature map is multiplied by the corresponding weights and then summed to output the fused feature map.
[0024] Furthermore, in step S4, the detection loss function is the optimized multi-task loss function, which includes: CIoU loss for bounding box regression. Focal classification loss is used to solve the class imbalance problem. and the distribution focus loss for fine-tuning the box distribution. .
[0025] Furthermore, in step S5, the knee joint multi-sequence MRI images to be detected are input into the optimized multi-sequence fusion detection model to obtain the damage localization and classification results, including:
[0026] The clinically acquired multi-sequence MRI images of the knee joint to be tested are first processed and registered to form a standardized T1, T2, and PD sequence image group.
[0027] Subsequently, the image set is input into the YOLOv8-based multi-sequence fusion detection model trained in step S4; the model performs forward propagation inference and outputs multiple candidate detection boxes, each box including coordinates and confidence scores for each damage level; the candidate boxes are post-processed using a non-maximum suppression algorithm to obtain the final damage detection set.
[0028] Secondly, based on the same concept, the present invention also provides a knee meniscus injury detection system that integrates multi-sequence and contrastive learning, comprising:
[0029] The data preprocessing and registration module is used to acquire multi-sequence MRI training images and images to be detected of the knee joint; it performs preprocessing and registration on the multi-sequence MRI training images to construct a standardized training dataset;
[0030] The contrastive learning module is used to input the preprocessed training image into the feature encoding network based on contrastive learning for feature learning, obtain pre-trained weights, and optimize the parameters of the feature encoding network using contrastive loss.
[0031] The multi-sequence fusion detection model construction and fine-tuning module is used to build a YOLOv8-based multi-sequence fusion detection model and load pre-trained weights; it also optimizes the parameters of the multi-sequence fusion detection model using labeled data and a detection loss function.
[0032] The damage detection and inference module is used to input the multi-sequence MRI images of the knee joint into the optimized multi-sequence fusion detection model, perform forward propagation inference, and obtain the damage localization and classification results.
[0033] Compared with existing technologies, the knee meniscus injury detection method and system integrating multi-sequence and contrastive learning described in this invention has the following advantages:
[0034] (1) The present invention provides a method and system for detecting knee meniscus injury by integrating multiple sequences and contrastive learning. The present invention addresses the core challenges in the detection of knee meniscus injury, such as the scarcity of high-quality labeled data, insufficient utilization of multiple sequence information, and difficulty in detecting subtle injuries, and proposes a systematic solution.
[0035] (2) The method and system for detecting knee meniscus injury that integrates multiple sequences and contrastive learning described in this invention introduces self-supervised contrastive learning. By constructing positive and negative sample pairs through image augmentation, the feature encoding network based on contrastive learning is optimized, thus constructing a feature extraction foundation with strong generalization ability. This effectively reduces the model's dependence on large-scale, high-cost labeled data and enhances the model's generalization ability and robustness when facing images acquired by different devices and scanning protocols.
[0036] (3) The present invention describes a method and system for detecting knee meniscus injuries by integrating multiple sequences and contrastive learning. The present invention constructs a multi-sequence fusion detection model based on YOLOv8. The system is based on the YOLOv8 detection framework and creatively transforms it into a multi-branch input mode to process the registered T1, T2, and PD sequences respectively. Among them, the C2f module and multi-scale fusion strategy adopted by YOLOv8 enhance the model's ability to capture small structures and weak signal features, which helps to improve the sensitivity and localization accuracy of early meniscus injuries. At the same time, YOLOv8 adopts a modular architecture design, which facilitates integration and optimization with the multi-branch backbone network and adaptive fusion module in this scheme. This model not only inherits the advantages of YOLOv8 in detection speed and accuracy, but also introduces Focal classification loss optimized for class imbalance problems, making it suitable for meniscus injury level classification tasks and able to output accurate injury localization and classification results.
[0037] (4) The present invention provides a method and system for detecting knee meniscus injuries by integrating multiple sequences and contrastive learning. The present invention designs an adaptive multi-sequence feature fusion module based on an attention mechanism to integrate complementary information from different MRI sequences. The core of this module lies in its introduced channel attention mechanism, which can dynamically analyze feature maps from different sequences and automatically generate weight vectors representing the importance of each sequence, thereby achieving feature fusion through weighted summation. This module maximizes information utilization and significantly improves the detection performance for subtle and complex injuries. Attached Figure Description
[0038] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:
[0039] Figure 1 This is a schematic diagram of the overall method flow according to an embodiment of the present invention;
[0040] Figure 2 This is a schematic diagram of a feature encoding network based on contrastive learning as described in an embodiment of the present invention;
[0041] Figure 3This is a schematic diagram of the YOLOv8-based multi-sequence fusion detection model described in an embodiment of the present invention;
[0042] Figure 4 This is a schematic diagram of the adaptive feature fusion module described in an embodiment of the present invention. Detailed Implementation
[0043] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.
[0044] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0045] like Figures 1 to 4 As shown, the knee meniscus injury detection method integrating multiple sequences and contrastive learning includes:
[0046] S1. Acquire multi-sequence MRI training images and images to be detected of the knee joint;
[0047] S2. Preprocess and register the multi-sequence MRI training images, and perform registration based on the affine transformation algorithm of mutual information to achieve spatial alignment of different sequences at the same anatomical level and construct a standardized training dataset.
[0048] S3. Input the preprocessed training image into a feature encoding network based on contrastive learning to learn features, obtain pre-trained weights, and optimize the parameters of the feature encoding network using contrastive loss.
[0049] S4. Construct a multi-sequence fusion detection model based on YOLOv8 and load pre-trained weights; use labeled data to optimize the parameters of the fusion detection model through the detection loss function;
[0050] S5. Input the multi-sequence MRI images of the knee joint to be detected into the optimized fusion detection model to obtain the localization and classification results of the injury.
[0051] In a preferred embodiment of the present invention, the feature encoding network based on contrastive learning sequentially includes a YOLOv8 backbone network, a global average pooling layer, and a projection head; random data augmentation is applied to the training images to generate positive sample pairs, and the contrastive loss function drives the model to bring the positive sample pairs closer in the feature space, while pushing away the distance from the negative samples.
[0052] In a preferred embodiment of the present invention, the YOLOv8-based multi-sequence fusion detection model sequentially includes a multi-branch backbone network, an adaptive feature fusion module, a YOLOv8 neck, and a YOLOv8 head. The multi-branch backbone network consists of three structurally identical and parameter-shared YOLOv8 backbone network branches, which respectively extract features from the registered T1, T2, and PD sequence input images. During model construction, each branch synchronously loads the weights obtained from the first-stage contrastive learning pre-training, ensuring that the network initially possesses robust general knee joint feature representations. The adaptive feature fusion module is used to perform multi-sequence fusion on the features output by the multi-branch backbone network. The YOLOv8 neck receives the fused features and performs multi-scale fusion, outputting an enhanced multi-scale feature map. The YOLOv8 head performs parallel classification and regression on the enhanced multi-scale feature map, outputting the bounding box, category, and confidence score of the meniscus injury.
[0053] In a preferred embodiment of the present invention, the workflow of the adaptive feature fusion module is as follows: concatenating feature maps from different sequences in the same layer; inputting the concatenated features into a channel attention module to generate an attention weight vector corresponding to the number of sequences; multiplying the original feature map with the corresponding weights and summing the results to output the fused feature map.
[0054] In a preferred embodiment of the present invention, the detection loss function is an optimized multi-task loss function, which includes: CIoU loss for bounding box regression. Focal classification loss is used to solve the class imbalance problem. and the distribution focus loss for fine-tuning the box distribution. .
[0055] In this embodiment, the method proposed in this invention mainly includes two stages: a feature representation training stage based on contrastive learning and a damage detection and classification fine-tuning stage based on YOLOv8. In the first stage, this invention constructs a feature encoding network based on contrastive learning, employing a self-supervised pre-training strategy using readily available unlabeled knee MRI data to pre-train the model's backbone network. This invention selects the YOLOv8 backbone network. This process, by constructing data-enhanced image-view pairs, enables the model to learn robust knee joint feature representations without manual annotation, thus laying a solid feature foundation for subsequent detection tasks. In the second stage, this invention designs a multi-sequence fusion detection model based on YOLOv8, aiming to fully exploit the complementary information of T1-weighted, T2-weighted, and proton density (PD) sequences to achieve accurate localization and hierarchical classification of meniscus injuries. This model uses a multi-branch backbone network to extract features from the registered T1, T2, and PD sequences respectively, and through an adaptive feature fusion module, dynamically learns the relative importance weights of each sequence feature in the damage detection task, and then performs weighted fusion. Finally, the fused feature maps are input into the YOLOv8 neck and YOLOv8 head to achieve accurate bounding box localization of the meniscus injury area and hierarchical classification of the injury degree.
[0056] Most existing technologies employ single-stage training, primarily due to reliance on large-scale labeled datasets or the traditional approach of directly using pre-trained models from natural images for transfer learning. These methods are not designed to address the scarcity of labeled medical images and the complementary nature of sequence information. Given the typically scarce nature of labeled medical image data, end-to-end training often struggles to learn highly discriminative feature representations. Therefore, this invention proposes a two-stage training method integrating self-supervised contrastive learning and multi-sequence adaptive fusion. By first learning robust features on unlabeled data and then fine-tuning them using multi-sequence information, the model's performance and generalization ability are significantly improved in labeled-scarce scenarios. In this two-stage training scheme, firstly, contrastive learning is used to pre-train a feature extraction network on unlabeled knee MRI images to obtain more discriminative feature representations; this step uses a large number of positive and negative image pairs to improve the separability and generalization ability of the features. Subsequently, in the second step, the multi-sequence fusion detection model is initialized based on the pre-trained features and fine-tuned using limited labeled data, thereby achieving accurate localization and hierarchical classification of meniscus injuries.
[0057] In the meniscus injury detection task of this invention, YOLOv8 was chosen as the core detection framework based on a comprehensive consideration of its high suitability for the task requirements. Compared to YOLOv7, YOLOv8 not only improves general target detection performance, but its C2f module and multi-scale fusion strategy enhance the model's ability to capture small structures and weak signal features, helping to improve the sensitivity and localization accuracy of early meniscus injuries. Furthermore, YOLOv8 is currently the version that achieves a mature balance between accuracy, speed, and architectural stability, facilitating integration and optimization with the multi-branch backbone network and adaptive fusion module in this invention. Therefore, YOLOv8 not only provides stronger basic performance for the detection task, but also further strengthens the overall advantages of the two-stage method of this invention in feature learning and injury discrimination.
[0058] This invention specifically relates to a method and system for automatic localization and injury classification of the meniscus of the knee joint based on the deep learning target detection model YOLOv8, self-supervised learning, and multi-sequence medical image fusion technology.
[0059] This invention also proposes a knee meniscus injury detection system that integrates multiple sequences and contrastive learning, comprising:
[0060] The data preprocessing and registration module is configured to: acquire multi-sequence MRI training images and images to be detected of the knee joint; preprocess and register the multi-sequence MRI training images to construct a standardized training dataset;
[0061] The contrastive learning module is configured to: input the preprocessed training image into a feature encoding network based on contrastive learning for feature learning, obtain pre-trained weights, and optimize the parameters of the feature encoding network using contrastive loss;
[0062] The multi-sequence fusion detection model construction and fine-tuning module is configured to: construct a YOLOv8-based multi-sequence fusion detection model and load the pre-trained weights; and optimize the parameters of the fusion detection model using labeled data and a detection loss function.
[0063] The damage detection and reasoning module is configured to: input the knee joint multi-sequence MRI images to be detected into the optimized fusion detection model, perform forward propagation reasoning, and obtain the damage localization and classification results.
[0064] Advantages of this invention:
[0065] This invention addresses the core challenges in knee joint meniscus injury detection, such as the scarcity of high-quality labeled data, insufficient utilization of multi-sequence information, and difficulty in detecting subtle injuries, and proposes a systematic solution.
[0066] This invention introduces self-supervised contrastive learning, constructs positive and negative sample pairs through image augmentation to optimize the feature encoding network based on contrastive learning, and builds a feature extraction foundation with strong generalization ability. This effectively reduces the model's dependence on large-scale, high-cost labeled data and enhances the model's generalization ability and robustness when facing images acquired by different devices and scanning protocols.
[0067] This invention constructs a multi-sequence fusion detection model based on YOLOv8. The system uses the YOLOv8 detection framework as its basic architecture, creatively modifying it into a multi-branch input mode to process the registered T1, T2, and PD sequences separately. Specifically, the C2f module and multi-scale fusion strategy employed by YOLOv8 enhance the model's ability to capture fine structures and weak signal features, contributing to improved sensitivity and localization accuracy for early meniscus injuries. Simultaneously, YOLOv8's modular architecture facilitates integration and optimization with the multi-branch backbone network and adaptive fusion module in this solution. This model not only inherits the advantages of YOLOv8 in detection speed and accuracy but also introduces a Focal classification loss optimized for class imbalance, making it suitable for meniscus injury level classification tasks and capable of outputting accurate injury localization and classification results.
[0068] This invention designs an adaptive multi-sequence feature fusion module based on an attention mechanism to integrate complementary information from different MRI sequences. The core of this module lies in its introduced channel attention mechanism, which dynamically analyzes feature maps from different sequences and automatically generates weight vectors representing the importance of each sequence, then achieves feature fusion through weighted summation. This module maximizes information utilization and significantly improves the detection performance for subtle and complex lesions.
[0069] Example 1
[0070] like Figure 1 As shown in the figure, this embodiment discloses a method for detecting knee meniscus injuries based on contrastive learning and multi-sequence fusion. The method includes the following steps:
[0071] A1: Preprocess and register the multi-sequence MRI training images to construct a standardized training dataset;
[0072] The acquired MRI volumetric data, including sagittal T1-weighted, T2-weighted, and proton density sequences, were used as input. First, all image sequences were resized using bilinear interpolation to a fixed size. Then, pixel values were standardized using Z-score normalization to ensure a normal distribution with a mean of 0 and a standard deviation of 1.
[0073] Next, the crucial multi-sequence image registration operation is performed. Using PD sequence images with clear anatomical structures as fixed reference images, T1 and T2 sequences are used as moving images. An affine transformation algorithm based on mutual information is used to calculate the spatial transformation matrix from the moving images to the fixed images, and this transformation is applied to the entire moving image sequence. Finally, T1, T2, and PD sequence image pairs that are precisely aligned in anatomical space are obtained, forming a standardized training dataset.
[0074] A2: Input the preprocessed training image into a feature encoding network based on contrastive learning to learn features, obtain pre-training weights, and optimize the parameters of the pre-trained model using contrastive loss;
[0075] like Figure 2 As shown, the YOLOv8 backbone network is used as the encoder backbone network for the feature encoding network based on contrastive learning, aiming to learn a general and robust feature representation of knee MRI.
[0076] The acquired training images are used to construct a training set. Random data augmentation operations are applied to each training image in the training set to generate multiple images with different views. Positive sample pairs from the same image and negative sample pairs from different images are then constructed based on these generated images with different views. The positive and negative sample pairs are input into a feature encoding network based on contrastive learning to learn image features. The data augmentation operations include random color jitter, Gaussian blur, Gaussian noise injection, and elastic deformation, etc., to simulate complex imaging variations in clinical images.
[0077] The feature encoding network based on contrastive learning comprises a YOLOv8 backbone network, a global average pooling layer, and a projection head. Specifically, given an unlabeled MRI image X, two views are generated through random enhancement. , The view is input into the YOLOv8 backbone network, and the output feature map is then input into a global average pooling layer to convert the feature map into a feature vector. , The feature vector is then input into the projection head, and the feature vector is mapped to a lower-dimensional space to obtain a vector. , The projection head includes two fully connected layers and a ReLU activation function between them. This is used to map high-dimensional features to a low-dimensional contrastive learning space.
[0078] A contrastive loss function is calculated based on positive and negative samples. Effective feature representations are learned by bringing positive sample pairs closer together and pushing negative sample pairs further apart, thereby optimizing the encoder parameters. The contrastive loss function uses the InfoNCE loss, defined as follows:
[0079] ;
[0080] in, and These are positive sample pairs. It is a negative sample. Represents the cosine similarity function. It's a temperature parameter. This represents the number of samples in the training batch. , , For sample index, , used to exclude itself.
[0081] Optimizing the network using a contrastive loss function enables the model to learn to extract discriminative features that are insensitive to imaging changes and can distinguish different anatomical structures. After pre-training, the weights of the YOLOv8 backbone network are saved.
[0082] A3: Construct a multi-sequence fusion detection model based on YOLOv8 and load the pre-trained weights; optimize the parameters of the fusion detection model using labeled data and a detection loss function;
[0083] In this embodiment, the constructed YOLOv8-based multi-sequence fusion detection model comprises four core parts: a multi-branch backbone network, an adaptive feature fusion module, a YOLOv8 neck, and a YOLOv8 head, as shown in the following structure. Figure 3 As shown.
[0084] First, three CSPDarknet branches with shared weights are initialized, and the pre-trained weights obtained in step S2 are loaded into these three branches to ensure strong initial feature extraction capabilities. The three branches are used to process the registered T1, T2, and PD sequence inputs, respectively, and each branch outputs features at three scales.
[0085] Then, an adaptive feature fusion module is designed and embedded. This module receives feature maps of the same scale from the three branches. Figure 4 As shown, its workflow is as follows: First, the three feature maps are concatenated along the channel dimension; then, the concatenated features are passed through a single channel attention layer. Specifically, a three-channel attention weight vector is generated through global average pooling and two fully connected layers. This vector dynamically represents the importance of each sequence in the current context; finally, the original three feature maps are multiplied by their corresponding weights and then summed using weighted averages to output the fused feature map.
[0086] Next, the fused multi-sequence feature maps are fed into the standard YOLOv8 neck network for multi-scale feature fusion.
[0087] Finally, the fused multi-scale feature map is fed into the YOLOv8 header to output the bounding box of the damage, confidence score, and classification probability.
[0088] Using the training data from step A1, the fusion model is fine-tuned through a detection loss function. The detection loss function... The optimized multi-task loss function includes: CIoU loss for bounding box regression. Focal classification loss is used to solve the class imbalance problem. and the distribution focus loss for fine-tuning the box distribution. The loss functions are defined as follows:
[0089] ;
[0090] ;
[0091] ;
[0092] ;
[0093] in, , and To balance the weighting coefficients of each loss term, the parameters of the entire fusion detection model are optimized by minimizing the total detection loss. middle The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. For Euclidean distance, The coordinates of the center point of the prediction box. The coordinates of the center point of the true bounding box. This is the diagonal length of the smallest bounding box between the predicted bounding box and the ground truth bounding box. This is a balancing coefficient used to adjust the weights of overlap-related terms. This is a term for aspect ratio consistency. middle The total number of damage categories, For the first The predicted probability of a class. This is a focusing parameter used to adjust the degree of attention given to difficult-to-differentiate samples. One-hot encoding for the real label. middle For discrete intervals, For range index, For the first The weights of each distribution interval For the model to the first The predicted probability for each interval.
[0094] A4: Input the multi-sequence MRI images of the knee joint to be detected into the optimized fusion detection model to obtain the localization and classification results of the injury.
[0095] In this embodiment, clinically acquired multi-sequence MRI images of the knee joint to be tested are first preprocessed and registered to form a standardized set of T1, T2, and PD sequence images. This image set is then input into a YOLOv8-based multi-sequence fusion detection model trained in step A3. The model performs forward propagation inference, outputting a series of candidate bounding boxes, each containing coordinates and a confidence level for each damage grade. Non-maximum suppression is applied to post-process the candidate boxes to remove highly overlapping redundant boxes, resulting in the final damage detection set.
[0096] This invention fully utilizes the complementary information of different sequences by performing precise registration and adaptive feature fusion on multi-sequence MRI of the knee joint; through a comparative learning pre-training strategy, it significantly improves the model's feature extraction ability and generalization in small sample scenarios; finally, through the integrated YOLOv8 detection model, it achieves rapid and accurate localization and grading of meniscus injuries, providing a highly efficient computer-aided diagnostic tool for clinical practice.
[0097] Example 2
[0098] This embodiment discloses a knee joint meniscus injury detection system based on contrastive learning and multi-sequence fusion, including:
[0099] The data preprocessing and registration module is configured to: acquire multi-sequence MRI training images and images to be detected of the knee joint; perform standardization and spatial registration processing on the multi-sequence MRI training images to construct a standardized training dataset;
[0100] The contrastive learning module is configured to: input the preprocessed training image into the feature encoding network based on contrastive learning for training; generate positive sample pairs by applying different random data augmentations to the same image, and optimize the network parameters based on the InfoNCE contrastive loss function, so that the model learns a knee joint feature representation that is robust to imaging differences, and finally outputs the pre-trained network weights.
[0101] The multi-sequence fusion detection model construction and fine-tuning module is configured to: construct a YOLOv8-based multi-sequence fusion detection model including a multi-branch backbone network, an adaptive feature fusion module, a YOLOv8 neck, and a YOLOv8 head; load the pre-trained weights into the multi-branch backbone network; and perform end-to-end training of the fusion detection model using labeled damage data and a detection loss function.
[0102] The damage detection and inference module is configured to: input preprocessed and registered multi-sequence MRI images of the knee joint to be detected into the trained fusion detection model; perform forward propagation calculations and output raw detection results including meniscus injury bounding box coordinates, injury grade category and prediction confidence.
[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for knee meniscus tear detection fusing multi-sequence and contrastive learning, characterized in that: include: S1. Acquire multi-sequence MRI training images and images to be detected of the knee joint; S2. Preprocess and register the multi-sequence MRI training images, and perform registration based on the affine transformation algorithm of mutual information to achieve spatial alignment of different sequences at the same anatomical level and construct a standardized training dataset. The acquired MRI volume data, including sagittal T1-weighted, T2-weighted, and proton density sequences, were used as input. First, the images of all sequences were scaled to a fixed size using bilinear interpolation. Then, pixel values were standardized using Z-Score normalization to ensure that the pixel value distribution conformed to a normal distribution with a mean of 0 and a standard deviation of 1. Secondly, perform the critical multi-sequence image registration operation; Using PD sequence images with clear anatomical structures as fixed reference images, and T1 and T2 sequences as moving images respectively, a spatial transformation matrix from moving images to fixed images is calculated using an affine transformation algorithm based on mutual information. This transformation is then applied to the entire moving image sequence, ultimately obtaining T1, T2, and PD sequence image pairs that are precisely aligned in anatomical space, forming a standardized training dataset. S3. Input the preprocessed multi-sequence MRI training images into a feature coding network based on contrastive learning for feature learning, obtain pre-trained weights, and optimize the parameters of the feature coding network using contrastive loss. The acquired training images are used to construct a training set. Random data augmentation is applied to each training image in the training set to generate multiple images with different views. Positive sample pairs from the same image and negative sample pairs from different images are formed based on the generated images with different views. The positive and negative sample pairs are input into a feature encoding network based on contrastive learning to learn image features. Data augmentation operations include random color jitter, Gaussian blur, Gaussian noise injection, and elastic deformation. S4. Construct a multi-sequence fusion detection model based on YOLOv8 and load pre-trained weights; Using labeled data, the parameters of a multi-sequence fusion detection model are optimized through a detection loss function; S5. Input the multi-sequence MRI images of the knee joint to be detected into the optimized multi-sequence fusion detection model to obtain the localization and classification results of the injury.
2. The method of claim 1, wherein: In step S3, the feature encoding network based on contrastive learning sequentially includes a YOLOv8 backbone network, a global average pooling layer, and a projection head; random data augmentation is applied to the training images to generate positive sample pairs, and the contrastive loss function drives the model to bring the positive sample pairs closer in the feature space, while pushing them further away from the negative samples.
3. The method of claim 1, wherein: In step S4, the YOLOv8-based multi-sequence fusion detection model sequentially includes a multi-branch backbone network, an adaptive feature fusion module, a YOLOv8 neck section, and a YOLOv8 head section; wherein, The multi-branch backbone network consists of three YOLOv8 backbone network branches with identical structures and shared parameters, which extract features from the registered T1, T2 and PD sequence input images respectively. When building the model, each branch synchronously loads the weights obtained from the first stage of contrastive learning pre-training to ensure that the network has a robust general feature representation of the knee joint from the beginning. An adaptive feature fusion module is used to perform multi-sequence fusion on the features output by the multi-branch backbone network; The YOLOv8 neck is used to receive fused features and perform multi-scale fusion, outputting an enhanced multi-scale feature map. The YOLOv8 header is used to perform parallel classification and regression on the enhanced multi-scale feature map, and output the bounding box, category and confidence score of the meniscus injury.
4. The method of claim 3, wherein: The workflow of the adaptive feature fusion module is as follows: feature maps from different sequences in the same layer are concatenated by channel; The concatenated features are input into a channel attention module to generate an attention weight vector corresponding to the number of sequences; the original feature map is multiplied by the corresponding weights and then summed to output the fused feature map.
5. The method of claim 3, wherein: In step S4, the detection loss function is an optimized multi-task loss function, and the detection loss function is an optimized multi-task loss function, including: CIoU loss for bounding box regression , focal classification loss for solving the class imbalance problem , and distribution focal loss for fine-tuning the distribution of boxes .
6. The method of claim 1, wherein: In step S5, the knee joint multi-sequence MRI images to be detected are input into the optimized multi-sequence fusion detection model to obtain the damage localization and classification results, including: The clinically acquired multi-sequence MRI images of the knee joint to be tested are first processed and registered to form a standardized T1, T2, and PD sequence image group. Subsequently, the image set is input into the YOLOv8-based multi-sequence fusion detection model trained in step S4; the model performs forward propagation inference and outputs multiple candidate detection boxes, each box including coordinates and confidence scores for each damage level; the candidate boxes are post-processed using a non-maximum suppression algorithm to obtain the final damage detection set.
7. The knee meniscus injury detection system fusing multi-sequence and contrast learning, applied to the knee meniscus injury detection method fusing multi-sequence and contrast learning according to any one of claims 1-6, characterized in that: include: The data preprocessing and registration module is used to acquire multi-sequence MRI training images and images to be detected of the knee joint; Preprocessing and registration of multi-sequence MRI training images were performed to construct a standardized training dataset; The contrastive learning module is used to input the preprocessed training image into the feature encoding network based on contrastive learning for feature learning, obtain pre-trained weights, and optimize the parameters of the feature encoding network using contrastive loss. The multi-sequence fusion detection model construction and fine-tuning module is used to build a YOLOv8-based multi-sequence fusion detection model and load pre-trained weights. Using labeled data, the parameters of a multi-sequence fusion detection model are optimized through a detection loss function; The damage detection and reasoning module is used to input the multi-sequence MRI images of the knee joint into the optimized multi-sequence fusion detection model, perform forward propagation reasoning, and obtain the damage localization and classification results.