MRI Image Prediction Method and Equipment for Hip Labral Tears Based on SwinT-Hybrid and DINOv2 Models

Abstract

This invention discloses a method and device for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models. The method uses standard MRI images labeled with signs of hip labral tears obtained through arthroscopic examination at preset time intervals as a prediction dataset, and performs data preprocessing. A hip labral tear prediction model is constructed based on the SwinT-Hybrid and DINOv2 models, and trained using the preprocessed prediction dataset. Real-time acquired hip labral MRI images are processed and input into the trained model. The model output determines the presence of a hip labral tear. This invention reduces the need for labeled data, improves model efficiency, and offers significant advantages in terms of prediction sensitivity and specificity.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a method, apparatus, device, and storage medium for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models. Background Technology

[0002] Hip labral tears are a common sports injury of the hip joint, primarily occurring in the anterosuperior region of the acetabulum. They frequently occur during sports injuries, such as football and basketball, where sudden twisting, impacts, or hyperextension can cause labral tears. Without accurate prediction and timely treatment, patients typically face limited joint mobility, increased hip pain, decreased lower limb muscle strength, and reduced hip joint stability, leading to recurrent subluxation and eventual bone loss. These consequences impose a significant burden on patients and incur substantial social costs. In particular, when hip labral tears are misdiagnosed due to their similar clinical presentation to other conditions (including arthritis, fasciitis, and tendinitis), delayed diagnosis can further exacerbate these negative effects.

[0003] The gold standard for predicting labral tears is arthroscopy, which involves direct visualization of the labrum using a camera during minimally invasive surgery. Previous studies evaluating the performance of labral tear prediction tools have relied on this method. However, this is a surgical procedure that typically requires preoperative examination and carries anesthesia risks; therefore, it is generally used for lesion repair rather than simply predicting the disease.

[0004] Magnetic resonance imaging arthrograms (MRAs) are often the preferred imaging modality for preoperative prediction of labral tears. However, MRAs require the injection of contrast agents into the joint, which not only increases patient discomfort but also raises medical and time costs, while also posing risks of infection and other complications.

[0005] While traditional magnetic resonance imaging (MRI) is a non-invasive alternative, its lack of contrast agent contrast results in low resolution of intra-articular details, leading to low accuracy in predicting labral tears. Studies have shown that routine MRI scans have low sensitivity and a high false-negative rate for labral pathology in adolescents and young adults, whether reported by an official radiologist or reviewed by a sports medicine orthopedic surgeon not involved in the patient's treatment. The interpreter's predictive abilities in MRI images largely depend on personal experience and expertise, making them susceptible to subjective influences and increasing the risk of misdiagnosis and missed diagnosis.

[0006] In recent years, the application of deep learning technology in medical image analysis has gradually attracted attention. By automatically extracting high-level image features, deep learning models have shown great potential in improving the efficiency and accuracy of image prediction. However, current research based on deep learning mainly focuses on other fields (such as cancer detection and cardiovascular disease detection), and its application in orthopedic imaging, especially in the prediction of labral tears, remains relatively limited. Existing technologies mostly rely on large-scale manually labeled datasets, leading to significant challenges in developing and deploying such systems.

[0007] Traditional MRI imaging prediction methods often struggle to accurately identify hip labral tears due to the complex anatomy and subtle pathological changes. Standard MRI has low sensitivity and specificity, while MRAs, although improving predictive accuracy, are limited by their invasiveness and high cost. Furthermore, interpreters are susceptible to subjective biases when interpreting images, increasing the risk of misdiagnosis and missed diagnosis. Summary of the Invention

[0008] The present invention aims to solve the technical problems existing in the prediction of hip labral tears using MRI images, improve the accuracy and efficiency of hip labral tear prediction, reduce reliance on invasive examinations, and reduce medical costs and patient burden.

[0009] Therefore, the first objective of this invention is to propose a method for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models, comprising:

[0010] Standard MRI images labeled with hip labral tears based on arthroscopic examination results were collected within a preset time interval to form a hip labral tear prediction dataset, and the hip labral tear prediction dataset was preprocessed.

[0011] A hip labral tear prediction model was constructed based on the SwinT-Hybrid and DINOv2 fusion model, and the model was trained using a preprocessed hip labral tear prediction dataset.

[0012] After processing the real-time acquired hip labral MRI images, the data is input into the trained hip labral tear prediction model. The model output results are then used to determine whether a hip labral tear exists.

[0013] The dataset includes standard MRI images with positive and negative labels for hip labral tears obtained from arthroscopic examinations within a preset time interval. These images are used as the hip labral tear prediction dataset and include:

[0014] Data sets were collected from patients aged 12 to 65 years who underwent standard MRI within one year prior to arthroscopy.

[0015] Based on the results of intraoperative arthroscopy, standard MRI images containing positive and negative labels for hip labral tears were obtained from a standard MRI patient dataset and used as a hip labral tear prediction dataset, and consistency verification was performed.

[0016] The data preprocessing for the hip labral tear prediction dataset includes:

[0017] Standard MRI images were resized to n×224×224 to focus on the region of interest;

[0018] Based on the MRI sequence type and fat saturation status, the standard MRI image intensity values ​​were standardized to the range of 0 to 1.

[0019] Standard MRI images are augmented using random rotation, translation, scaling, flipping, and Gaussian noise.

[0020] Among them, a hip labral tear prediction model was constructed based on the SwinT-Hybrid and DINOv2 fusion model, including:

[0021] The model parameters of the SwinT-Hybrid and DINOv2 fusion model were initialized using the ImageNet dataset to obtain an initial hip labral tear prediction model.

[0022] The initial hip labral tear prediction model was pre-trained using axial, coronal, and sagittal views from the MRNet knee MRI dataset. It was then fine-tuned by transferring the model to a standard hip MRI dataset to obtain the final hip labral tear prediction model.

[0023] The model was trained using a preprocessed hip labral tear prediction dataset, including:

[0024] Each frame slice in a single sequence of MRI images from the preprocessed hip labral tear prediction dataset is repeatedly expanded to the RGB channels and divided into non-overlapping 4×4 image patches. These patches are then converted into high-dimensional vectors through a fully connected layer and used as the model input for the hip labral tear prediction model.

[0025] The input image patch vector is used for feature extraction through multiple SwinT-Hybrid Blocks; each SwinT-Hybrid Block includes a window multi-head self-attention module, a hybrid shift window multi-head self-attention module, a fully connected layer, a residual-based multilayer perceptron, and residual connections; the feature extraction process downsamples layer by layer and increases the number of feature channels through patch merging layers;

[0026] The DINOv2 model extracts global features, reduces their dimensionality, and then fuses them with SwinT-Hybrid features. The fused features are then aggregated using max pooling, and the aggregated global feature vector is input into the classification head, where a multilayer perceptron completes the classification task.

[0027] The model training uses the cross-entropy loss function, and the weight parameters are adjusted through the AdamW optimizer. The initial learning rate is set and dynamically adjusted using a cosine annealing strategy.

[0028] In this model, the global features extracted by the DINOv2 model are dimensionality-reduced through linear projection and then fused with the features extracted by the SwinT-Hybrid model using a weighted self-attention mechanism in the feature fusion module. The weights are learnable parameters α, and the fusion formula is as follows:

[0029] Z fused =α·Z Swin +(1-α)·Z DINO

[0030] Among them, Z fused For fusion features, Z Swin Features extracted for the SwinT-Hybrid model, Z DINO Global features extracted for the DINOv2 model.

[0031] The process involves processing real-time acquired MRI images of the hip labrum and inputting them into a trained hip labrum tear prediction model. The model's output determines whether a hip labrum tear exists, including:

[0032] The real-time acquired hip labrum MRI images are divided into three views: axial, coronal, and sagittal. The MRI sequence of each view is input into the trained hip labrum tear prediction model for prediction.

[0033] The predicted probabilities of all slices in each view are averaged to obtain the sequence-level probability. The sequence-level probabilities of the three views are averaged to obtain the final predicted probability of the overall MRI data.

[0034] The final predicted probability is compared with a preset prediction threshold to determine whether there is a labral tear in the hip joint MRI image.

[0035] The second objective of this invention is to provide a device for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models, comprising:

[0036] The data acquisition module is used to collect standard MRI images labeled with hip labral tears based on arthroscopic examination results within a preset time interval, as a hip labral tear prediction dataset, and to perform data preprocessing on the hip labral tear prediction dataset.

[0037] The model building module is used to build a hip labral tear prediction model based on the SwinT-Hybrid and DINOv2 fusion model, and to train the model using a preprocessed hip labral tear prediction dataset.

[0038] The prediction module processes real-time acquired hip labral MRI images and inputs them into a trained hip labral tear prediction model. The model outputs the results to determine whether a hip labral tear exists.

[0039] A third objective of the present invention is to provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method described above.

[0040] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the steps of the method according to the foregoing technical solution.

[0041] The advantages of this invention, which differ from existing technologies, include:

[0042] 1. This invention employs a fusion of SwinT-Hybrid and DINOv2, combining the advantages of both. SwinT-Hybrid excels at extracting local detailed features, while DINOv2 captures global contextual information. This fusion approach allows the model to better integrate local and global information when processing complex hip labral tear images, further enhancing the model's ability to identify different pathological conditions and improving its generalization effect in practical clinical applications.

[0043] 2. Traditional prediction methods heavily rely on MRI scans, requiring invasive procedures involving the injection of contrast agents, increasing patient discomfort and medical costs. This invention can accurately predict labial tears using standard MRI images, avoiding the need for invasive procedures and providing patients with a safer and more convenient treatment option.

[0044] 3. This invention fully utilizes pre-training techniques in model training, transferring features from the model on the MRNet knee MRI dataset to the prediction task of hip labral tears, significantly reducing the need for large-scale labeled data. Simultaneously, data preprocessing combined with data augmentation techniques effectively alleviates the class imbalance problem, significantly improving the model's generalization ability and making it suitable for multi-center deployment applications.

[0045] 4. The method of this invention fully utilizes the multi-view features of MRI images, including axial, coronal, and sagittal views. By extracting and fusing key features from each view using a deep learning model, it can effectively capture the full picture of hip labral tears. The fused features can comprehensively reflect the complex anatomical structure and pathological changes of the lesion area from different perspectives, thereby avoiding important lesion information that may be missed in single-view prediction.

[0046] 5. This invention innovatively utilizes a Hybrid Shifting Window Multi-Head Self-Attention Module (HSW-MSA) and a Residual-Based Multilayer Perceptron (ResMLP) to significantly improve the predictive performance of hip labral tears on MRI images. The HSW-MSA module effectively enhances cross-window information modeling capabilities through a unique window partitioning and fusion strategy. Its shifting window translation operation improves cross-window interaction, with horizontal and vertical strip windows capturing long-distance dependencies in different directions. While maintaining computational efficiency, it ensures data processing consistency through cyclic shifting and masking mechanisms, enabling the model to accurately capture local and global features and greatly improving the ability to identify complex pathological features. The ResMLP module, through residual connections, successfully solves the gradient vanishing problem in deep network training, ensuring stable and efficient training even with increasing model depth. The synergistic effect of these two modules significantly improves the accuracy and reliability of predictions, enabling the model to run efficiently when processing large amounts of data. This provides a more efficient and accurate solution for clinical prediction and powerfully promotes the development of hip labral tear prediction technology. Attached Figure Description

[0047] The present invention and / or its additional aspects and advantages will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0048] Figure 1 This is a flowchart illustrating an MRI image prediction method for hip labral tears based on the SwinT-Hybrid and DINOv2 models provided by the present invention.

[0049] Figure 2 This is a schematic diagram of the architecture of the hip labral tear prediction model in the MRI image prediction method for hip labral tears based on the SwinT-Hybrid and DINOv2 models provided by the present invention.

[0050] Figure 3This is a schematic diagram of the SwinT-Hybrid Block in the MRI image prediction method for hip labral tears based on the SwinT-Hybrid and DINOv2 models provided by the present invention.

[0051] Figure 4 This is a schematic diagram of the shift window for calculating self-attention in the MRI image prediction method for hip labral tears based on the SwinT-Hybrid and DINOv2 models provided by this invention.

[0052] Figure 5 This is a schematic diagram of the self-attention calculation method in the displacement window partition of an MRI image prediction method for hip labral tears based on the SwinT-Hybrid and DINOv2 models provided by the present invention.

[0053] Figure 6 This is a schematic diagram comparing the traditional MLP module and the ResMLP module in an MRI image prediction method for hip labral tears based on the SwinT-Hybrid and DINOv2 models provided by this invention.

[0054] Figure 7 This is a schematic diagram illustrating the use of MRI sequences for model training in a method for predicting hip labral tears based on the SwinT-Hybrid and DINOv2 models provided by this invention.

[0055] Figure 8 This is a schematic diagram illustrating the prediction of hip labral tears using a trained hip labral tear prediction model, as provided in this invention, based on the SwinT-Hybrid and DINOv2 models.

[0056] Figure 9 This is a schematic diagram of the structure of an MRI image prediction device for hip labral tears based on the SwinT-Hybrid and DINOv2 models provided by the present invention.

[0057] Figure 10 This is a schematic diagram of the structure of a non-transitory computer-readable storage medium storing computer instructions provided by the present invention. Detailed Implementation

[0058] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0059] like Figure 1 As shown, an embodiment of the present invention provides a method for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models, comprising:

[0060] S110: Collect standard MRI images labeled with hip labral tears based on arthroscopic examination results within a preset time interval, as a hip labral tear prediction dataset, and perform data preprocessing on the hip labral tear prediction dataset.

[0061] Constructing a hip labral tear prediction dataset involves the following steps:

[0062] S111: Collect a dataset of patients aged 12 to 65 years who underwent standard MRI within one year prior to arthroscopy.

[0063] A dataset of patients who underwent hip arthroscopy was collected clinically, including those aged 12 to 65 years who had undergone standard MRI within one year prior to the arthroscopy. Younger patients were excluded to avoid age-related pathological differences, and older patients were excluded to focus on acute surgical cleft lip rather than degenerative cleft lip.

[0064] S112: Based on the results of intraoperative arthroscopy, obtain standard MRI images containing positive and negative labels for hip labral tears from the patient dataset of standard MRI, and use them as a hip labral tear prediction dataset, and perform consistency verification.

[0065] Positive labels in the dataset, indicating labral tears, were derived from intraoperative arthroscopic findings, which is the gold standard for diagnosis. The labels were compiled by two orthopedic surgeons with standardized sports medicine training and underwent consistency verification to ensure their accuracy.

[0066] Data preprocessing of the hip labral tear prediction dataset includes the following steps:

[0067] S113: Standard MRI images with positive and negative labels for hip labral tears were resized to n×224×224 to focus on the region of interest.

[0068] S114: Standard MRI image intensity values ​​are normalized to a range of 0 to 1 based on MRI sequence type and fat saturation status.

[0069] The intensity values ​​were standardized using the distribution statistics of the training set based on the MRI sequence type (e.g., T1, T2, PD) and fat saturation status, scaling them to the range of 0 to 1.

[0070] S115: Expands standard MRI images tenfold using random rotation, translation, scaling, flipping, and Gaussian noise.

[0071] S120: Based on the fusion model of SwinT-Hybrid and DINOv2, a hip labral tear prediction model was constructed, and the model was trained using a preprocessed hip labral tear prediction dataset.

[0072] A hip labral tear prediction model was constructed based on the fusion model of SwinT-Hybrid and DINOv2, including:

[0073] S121: The model parameters of the SwinT-Hybrid and DINOv2 fusion model were initialized using the ImageNet dataset to obtain the initial hip labral tear prediction model.

[0074] S122: The initial hip labral tear prediction model was pre-trained using axial, coronal, and sagittal views from the MRNet knee MRI dataset. It was then fine-tuned by transferring the model to a standard hip MRI dataset to obtain the hip labral tear prediction model.

[0075] The SwinT-Hybrid and DINOv2 fusion model was pre-trained on the MRNet knee MRI dataset in axial, coronal, and sagittal views. During pre-training, the model was trained with an "abnormality" label, enabling it to learn and extract feature representations related to knee joint abnormalities. These feature representations have transfer value for the subsequent hip labral tear prediction task. The model was then transferred to a standard hip MRI dataset for fine-tuning. Based on the specific characteristics and labeled data of hip labral tears, the model parameters were further optimized to more accurately identify and predict hip labral tears.

[0076] The model was trained using a preprocessed hip labral tear prediction dataset, including:

[0077] S123: Each frame slice in a single sequence of MRI images from the preprocessed hip labral tear prediction dataset is repeatedly expanded to the RGB channels and divided into non-overlapping 4×4 image patches. These patches are then converted into high-dimensional vectors through a fully connected layer and used as model input for the hip labral tear prediction model.

[0078] The model parameters are initialized using weights pre-trained on the ImageNet dataset. Then, each frame slice (n slices, each 224×224 pixels) from a single sequence of preprocessed MRI images is repeatedly expanded to the RGB channels and segmented into non-overlapping image patches. Each patch is a fixed 4×4 size, resulting in a feature dimension of 4×4×3=48 for each patch. These image patches are transformed into high-dimensional vectors through fully connected layers to capture local image features; these vectors serve as the model input.

[0079] S124: The input image patch vector is used for feature extraction through multiple SwinT-Hybrid Blocks; each SwinT-Hybrid Block includes a window multi-head self-attention module, a hybrid shift window multi-head self-attention module, a fully connected layer, a multilayer perceptron, and a residual connection; the feature extraction process downsamples layer by layer and increases the number of feature channels through patch merging layers.

[0080] The input patch vector undergoes feature extraction through multiple SwinT-Hybrid Blocks. Each block includes a Windows Multi-head Self-Attention (W-MSA) module, a Shifted Window Multi-head Self-Attention (HSW-MSA) module, a fully connected layer (Layer Normalization, LN), a Residual Multi-Layer Perceptron (ResMLP) module, and residual connections. The HSW-MSA feature extraction process is divided into multiple stages. Each stage downsamples the resolution layer by layer and increases the number of feature channels through a patch merging layer to obtain deeper features. The model operates independently on each frame slice to capture local pathological features.

[0081] S125: The DINOv2 model extracts global features and reduces their dimensionality before fusing them with the SwinT-Hybrid features; the fused features are then aggregated using max pooling, and the aggregated global feature vector is input into the classification head to complete the classification task through a multilayer perceptron.

[0082] The DINOv2 model was used to extract global features from MRI slices, and the extracted feature vectors were used as global feature representations. The features extracted by DINOv2 were dimensionality-reduced to fit the feature dimensions of the SwinT-Hybrid model for fusion. A feature fusion module was introduced after the SwinT-Hybrid and DINOv2 models to combine the global features from DINOv2 with the features extracted by the SwinT-Hybrid model using a weighted self-attention mechanism.

[0083] The extracted fusion features are aggregated along the slice dimension using a max pooling operation. The max pooling method selects the most salient features from a set of slices, generating a compact global feature vector that represents the global features of the entire MRI scan.

[0084] The aggregated global feature vector is input into the classification head, and the final classification task is completed through an activation function. In this specific implementation, the number of categories in the classification layer is set to 2, and the model output includes the prediction result (positive or negative) for hip labral tear and the corresponding prediction probability. The model is trained using the cross-entropy loss function, and the weight parameters are adjusted using the AdamW optimizer. During training, an initial learning rate is set and dynamically adjusted using a cosine annealing strategy to improve the model's convergence speed and final performance.

[0085] Figure 2 This paper demonstrates the model architecture of the hip labral tear prediction model of this invention. The model uses a Patch Partition module to split the input MRI slice of size 224×224×3 into N×(4×4×3) non-overlapping, equally sized patches. Therefore, the feature dimension of each patch after flattening is 4×4×3=48, resulting in a total of... Each 224×224×3 MRI slice was processed into 3136 patches, and each patch was flattened into a 48-dimensional token vector, resulting in a flattened overall structure. 2D patch sequence.

[0086] The fully connected layer (Linear Embedding) will then have a dimension of [missing information]. Projecting a tensor onto any dimension C yields a tensor of dimension C. Linear Embedding.

[0087] These patch tokens are fed into several SwinT-Hybrid blocks with improved self-attention. The first SwinT-Hybrid block keeps the number of input and output tokens constant. It remains unchanged and is designated as Stage 1 along with the fully connected layer.

[0088] To generate a hierarchical representation, the number of patch tokens gradually decreases in the patch merging layer as the network deepens. The first patch merging layer concatenates each group of 2×2 adjacent patches, thus reducing the number of patch tokens to [the original value]. Right now The dimension of the patch token is increased fourfold, to 4C, and then reduced to 2C through a linear layer before being input into the SwinT-Hybrid Block for feature extraction, while maintaining resolution. The process remains unchanged. The first patch merge layer and this block constitute Stage 2. Similar processing is repeated in Stage 2 and subsequent stages, ultimately resulting in Stage 4 outputting a resolution with a specific number of patch tokens. The dimension is 80. The final features extracted by SwinT-Hybrid are obtained.

[0089] Global features are extracted using DINOv2, with the final layer outputting a 768-dimensional feature vector as the classification label. MRI slices are input into the pre-trained DINOv2 model to extract global features. Then, its dimension was reduced to match the feature dimension of the SwinT-Hybrid through a linear transformation, resulting in the dimensionality-reduced global features. In the embodiments of the present invention, SwinT-Hybrid is the SwinT-Hybrid model, and DINOv2 is the DINOv2 model.

[0090] Z Swin and Z DINO Input feature fusion module. (Z) DINO Extend to Z through broadcasting replication Swin The same number of tokens, that is The two are fused using a weighted self-attention mechanism, with the weights being learnable parameters α, to obtain the fused features.

[0091] Z fused =α·Z Swin +(1-α)·Z DINO

[0092] This multi-stage processing method effectively achieves dynamic adjustment of resolution and feature dimensions, while integrating global contextual information and local detail information to form a backbone network for processing hip labral tear prediction tasks.

[0093] Figure 3 The structure of the SwinT-Hybrid Block is shown. A SwinT-Hybrid Block consists of a Windowed Multi-Head Self-Attention (W-MSA) module and a Hybrid Shifted Windowed Multi-Head Self-Attention (HSW-MSA) module, followed by a module composed of a residual-based multilayer perceptron (ResMLP) where the non-linear activation function uses GeLU. Layer normalization (LN) is added before each MSA module and ResMLP module, and features are preserved through residual connections after the modules.

[0094] To achieve efficient modeling, a local window self-attention mechanism is used instead of traditional global self-attention. The input feature map is uniformly divided into non-overlapping windows, each containing N = M × M patch tokens. The computational complexity of W-MSA is significantly reduced compared to global self-attention, specifically:

[0095] Ω(MSA)=4HWC 2 +2(HW) 2 C

[0096] Ω(W-MSA)=4HWC 2 +2M 2 HWC

[0097] Where H and W are the height and width of the window, respectively, C is the feature dimension, and M is fixed with a default value of 7. The computational complexity of W-MSA increases linearly, and it has good scalability compared to global self-attention (whose complexity increases with the square of HW).

[0098] MSA has quadratic complexity with respect to the number of patch tokens H×W (HW patch tokens in total, each patch token is computed HW times globally). W-MSA, on the other hand, has linear complexity when M is fixed (default is 7) (HW patch tokens in total, each patch token is computed M² times within its local window). The huge H×W is unbearable for global self-attention computation, while window-based self-attention (W-MSA) has good scalability.

[0099] While W-MSA significantly reduces computational complexity, its ability to model cross-window information is relatively limited. To introduce cross-window connections while maintaining the computational efficiency of non-overlapping windows, the SwinT-Hybrid Block alternates between regular window and hybrid shifted window partitioning. Regular window partitioning involves uniformly dividing an 8×8 feature map into four 4×4 windows (local window size M=4). Hybrid shifted window partitioning enhances the ability to model cross-window information by combining three different shifted window processing strategies: shifted windows, horizontal strip windows, and vertical strip windows.

[0100] like Figure 4 As shown, the shift window is created by panning the rule window to the upper left. This is achieved using individual pixels, and its recursive calculation process is as follows:

[0101]

[0102]

[0103]

[0104]

[0105] in, and z l These represent the output features of the MSA and ResMLP modules for the l-th block, respectively. The introduction of a shift window enhances cross-window interactivity and is effective for hip labral tear prediction tasks.

[0106] For the shifted window portion, after the window is shifted, some windows may have a size smaller than M×M. To maintain computational consistency, a cyclic shift method is used, effectively creating a new partitioning method that includes non-adjacent sub-windows by shifting the entire window to the upper left. After this shift, the number of windows processed in a batch remains consistent with the regular partitioning (e.g., if there are 4 windows in the regular partitioning, there are still 4 windows after the shift). The calculation and stitching method for a 9-window partition is as follows: Figure 5 As shown, since a window can contain features from different windows, a masked MSA mechanism is needed to limit the self-attention calculation to within each sub-window to avoid cross-region interference. Finally, an inverse shift method is used to reset the processing result to the original feature map.

[0107] Self-attention is the core of the Transformer architecture, enabling it to dynamically monitor the relationships between elements at different positions in the input sequence. To further enhance the model's ability to model the positional relationships between elements, relative positional bias is introduced. Used to enhance the representation of spatial information in sequence data. The calculation formula is as follows:

[0108]

[0109] in, These represent the query, key, and value matrices, respectively, where 'a' is the query / key dimension and 'M' is the value matrix. 2 This represents the number of patches within the local window.

[0110] By combining element content similarity (dot product) with relative positional bias, the model can more accurately capture the spatial relationships between patch elements in MRI slices, thereby improving the ability to identify complex pathological features such as hip labral tears.

[0111] For the i-th horizontal strip window, the self-attention calculation can be expressed as:

[0112]

[0113] For the j-th vertical strip window, the self-attention calculation can be expressed as:

[0114]

[0115] The HSW-MSA module enhances the model's ability to capture local features, global information, and orientation dependencies by weighted fusion of the self-attention results from three window partitioning strategies (shifted window, horizontal bar window, and vertical bar window).

[0116] In the Transformer architecture, the MLP is responsible for processing information individually for each tag. In the SwinT-Hybrid architecture, the ResMLP module, inspired by the ResNet architecture, is used instead of the traditional MLP module. Its structure is similar to... Figure 6 As shown, implementing ResMLP includes the following steps:

[0117] Let the input tensor be X, and the feature dimension be C. The first linear transformation expands the feature dimension from C to 4C. Then, the GELU activation function is used to perform a non-linear transformation on the features, increasing the model's expressive power. The activated result is then fed into a Dropout layer for regularization to prevent overfitting.

[0118] The data after the Dropout layer is normalized to make the data distribution more stable. The normalized data then undergoes a second linear transformation to restore the feature dimension from 4C to C, and the result is then put into the Dropout layer for regularization again.

[0119] The result after the above series of processing is added to the original input tensor X to obtain the final output tensor ResMLP(X).

[0120] The addition of residual connection mechanism effectively solves the gradient vanishing problem in deep network training, allowing the model to skip layers with low information content and focus on learning complex representations with important features. It can still train stably and efficiently as the depth increases, thereby enhancing the ability to handle complex tasks.

[0121] With the above improvements, SwinT-Hybrid demonstrates faster convergence speed and higher accuracy on hip MRI data, significantly improving the performance of hip labral tear prediction.

[0122] Figure 7 This is a schematic diagram illustrating the training of the SwinT-Hybrid and DINOv2 fusion model using MRI sequences. The steps include:

[0123] A single sequence of preprocessed MRI images is input into the training pipeline. Each sequence consists of n slices, each 224×224 pixels in size. To fit the model input, the grayscale values ​​of these slices are repeatedly expanded to the RGB channels to form a standard color image format. Each expanded slice is then fed into the FeatureExtractor Backbone for feature extraction, generating a feature representation for each slice.

[0124] For the feature representation of each slice, aggregation is performed using a Max Pooling operation along the slice dimension. Specifically, the features of the n slices are compared dimension by dimension, and the most significant (i.e., the maximum value) feature value is selected to generate a unified global feature vector. This operation effectively preserves the most critical pathological features, facilitating improved accuracy in downstream classification tasks.

[0125] The global feature vector generated in step two is input into the classification head to complete the classification task. The classification head contains fully connected layers and activation functions, which are used to map the feature vector to the predicted probability of hip labral tear. The prediction results include binary classification results of positive (tear exists) and negative (no tear).

[0126] Step 4: The model is trained using a binary cross-entropy loss function, which is scaled to account for class imbalance, ensuring that underrepresented classes contribute equally during optimization. During training, the weight parameters are adjusted using the AdamW optimizer, and the initial learning rate is dynamically adjusted. Cosine annealing is used to improve convergence speed and model performance. Furthermore, an early stopping strategy is employed based on validation set accuracy, with a patience value set to 10 epochs. Training terminates when validation set accuracy fails to improve within 10 consecutive epochs, and the model weights corresponding to the epoch with the highest validation accuracy are selected for the inference phase.

[0127] S130: After processing the real-time acquired hip labrum MRI images, input them into the trained hip labrum tear prediction model, and determine whether a hip labrum tear exists based on the model output results.

[0128] like Figure 8 As shown, the prediction process specifically includes the following steps:

[0129] S131: The real-time acquired hip labral MRI images are divided into three views: axial, coronal, and sagittal. The MRI sequence of each view is input into the trained hip labral tear prediction model for prediction.

[0130] Real-time MRI images of the hip joint were acquired, consisting of multiple sequences, each containing n slices, each slice measuring 224×224 pixels. These MRI slices were then divided into three views: axial, coronal, and sagittal.

[0131] For each view (axial, coronal, sagittal) of the MRI sequence, a hip labral tear prediction model was used to make predictions.

[0132] S132: The predicted probabilities of all slices in each view are averaged to obtain the sequence-level probability. The sequence-level probabilities of the three views are averaged to obtain the final predicted probability of the overall MRI data.

[0133] The model for each view extracts features from all slices in the sequence and generates a predicted probability for each slice. Then, the probability values ​​of all slices in that view are averaged to generate a sequence-level probability representing that view. The sequence-level probabilities from the three views (axial, coronal, and sagittal) are averaged to obtain the final predicted probability for the overall MRI data.

[0134] The final predicted probability is compared with a preset prediction threshold to determine whether there is a labral tear in the hip joint MRI image.

[0135] Based on a preset prediction threshold, the presence of a labral tear in the hip joint MRI image is determined. If the fusion probability value is higher than the threshold, a labral tear is predicted to be present; if it is lower than the threshold, a labral tear is predicted to be absent.

[0136] The model was evaluated using a multi-view ensemble on a reserved test set (20% of the dataset), and performance metrics including accuracy, sensitivity, specificity, and AUC-ROC were calculated to comprehensively assess predictive performance. These performance metrics were compared with results from the original radiology reports in the dataset, and intraoperative findings were used as the true baseline to further validate the model's actual predictive ability.

[0137] The prediction method of the present invention is applicable to the rapid prediction of hip labral tears, including but not limited to hip arthritis, fasciitis and related bone injuries.

[0138] This invention proposes a method for predicting hip labral tears based on MRI images using the SwinT-Hybrid and DINOv2 models. By learning the correlations of features in different views, layers, and between layers of MRI images, this method achieves more accurate classification and prediction of hip labral tears. The improved SwinT-Hybrid and DINOv2 fusion model can effectively extract multi-level features from local to global MRI images, demonstrating superior recognition and classification capabilities for lesions with complex structures and diverse features, such as hip labral tears. Furthermore, this invention extracts ROIs from the image data during the preprocessing stage, standardizes the intensity values, and then augments the training samples tenfold, thereby further improving the model's predictive performance and robustness.

[0139] Compared to existing methods for predicting hip labral tears, the system proposed in this invention can not only quickly and accurately classify whether a tear has occurred, but also comprehensively calculate the probability of the lesion based on the image features of different views, ultimately obtaining reliable prediction results. By combining local details and global features, this invention achieves high-precision prediction of hip labral tears, demonstrating significant practical value and clinical significance.

[0140] like Figure 9 As shown, the present invention provides a hip labral tear MRI image prediction device 800 based on the SwinT-Hybrid and DINOv2 models, comprising:

[0141] The data acquisition module 810 is used to collect standard MRI images labeled with hip labral tears based on arthroscopic examination results within a preset time interval, as a hip labral tear prediction dataset, and to perform data preprocessing on the hip labral tear prediction dataset.

[0142] Model building module 820 is used to build a hip labral tear prediction model based on the SwinT-Hybrid and DINOv2 fusion model, and to train the model using a preprocessed hip labral tear prediction dataset.

[0143] The prediction module 830 is used to process the real-time acquired hip labrum MRI images and input them into the trained hip labrum tear prediction model. The model output results are used to determine whether a hip labrum tear exists.

[0144] To implement the embodiments, the present invention also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method described above.

[0145] like Figure 10 As shown, the non-transitory computer-readable storage medium 900 includes a memory 910 for instructions and an interface 930, the instructions of which can be executed by a processor 920 to complete the method. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0146] To implement the embodiments, the present invention also proposes a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in the embodiments of the present invention.

[0147] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0148] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0149] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0150] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0151] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the described embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0152] Those skilled in the art will understand that all or part of the steps of the method described in the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0153] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0154] The storage medium mentioned may be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the embodiments within the scope of the present invention.

Claims

1. A method for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models, characterized in that, include: Standard MRI images labeled with hip labral tears as determined by arthroscopy within a preset time interval are collected as a hip labral tear prediction dataset, and the hip labral tear prediction dataset is preprocessed. A hip labral tear prediction model was constructed based on the SwinT-Hybrid and DINOv2 fusion model, and the model was trained using the preprocessed hip labral tear prediction dataset, including: Each frame slice in a single sequence of MRI images in the preprocessed hip labral tear prediction dataset is repeatedly expanded to the RGB channels and divided into non-overlapping 4×4 image patches. These patches are then converted into high-dimensional vectors through a fully connected layer and used as the model input for the hip labral tear prediction model. The input image patch vector is used for feature extraction through multiple SwinT-Hybrid Blocks; each SwinT-Hybrid Block includes a window multi-head self-attention module, a hybrid shift window multi-head self-attention module, a fully connected layer, a residual-based multilayer perceptron, and a residual connection; the feature extraction process downsamples layer by layer and increases the number of feature channels through patch merging layers; The DINOv2 model extracts global features and reduces their dimensionality before fusing them with the SwinT-Hybrid features. The fused features are then aggregated using max pooling, and the aggregated global feature vector is input into the classification head to complete the classification task through a multilayer perceptron. The model training uses the cross-entropy loss function, and the weight parameters are adjusted using the AdamW optimizer. The initial learning rate is set and dynamically adjusted using a cosine annealing strategy. After processing the real-time acquired hip labrum MRI images, the data is input into the trained hip labrum tear prediction model. The model output results are used to determine whether a hip labrum tear exists.

2. The method for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models according to claim 1, characterized in that, Standard MRI images showing positive and negative labels for hip labral tears as determined by arthroscopy within a preset time interval were collected to form a hip labral tear prediction dataset, including: Data sets were collected from patients aged 12 to 65 years who underwent standard MRI within one year prior to arthroscopy. Based on the results of intraoperative arthroscopy, standard MRI images containing positive and negative labels for hip labral tears were obtained from the patient dataset of the standard MRI, which were used as the hip labral tear prediction dataset and their consistency was verified.

3. The method for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models according to claim 2, characterized in that, Data preprocessing of the hip labral tear prediction dataset includes: The standard MRI image was adjusted to n×224×224 to focus on the region of interest; Based on the MRI sequence type and fat saturation status, the standard MRI image intensity values ​​are standardized to a range of 0 to 1. The standard MRI images are augmented using random rotation, translation, scaling, flipping, and Gaussian noise.

4. The method for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models according to claim 1, characterized in that, A hip labral tear prediction model was constructed based on the fusion model of SwinT-Hybrid and DINOv2, including: The model parameters of the SwinT-Hybrid and DINOv2 fusion model were initialized using the ImageNet dataset to obtain an initial hip labral tear prediction model. The initial hip labral tear prediction model was pre-trained using axial, coronal, and sagittal views from the MRNet knee MRI dataset, and then fine-tuned using a standard hip MRI dataset to obtain the final hip labral tear prediction model.

5. The method for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models according to claim 1, characterized in that, The global features extracted by the DINOv2 model are dimensionality-reduced through linear projection and then fused with the features extracted by the SwinT-Hybrid model in the feature fusion module using a weighted self-attention mechanism, with the weights being learnable parameters. The fusion formula is: in, As a feature of fusion, Features extracted for the SwinT-Hybrid model Global features extracted for the DINOv2 model.

6. The method for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models according to claim 1, characterized in that, After processing the real-time acquired hip labral MRI images, the data is input into the trained hip labral tear prediction model. The model output results are used to determine whether a hip labral tear exists, including: The real-time acquired hip labrum MRI images are divided into three views: axial, coronal, and sagittal. The MRI sequence of each view is input into the trained hip labrum tear prediction model for prediction. The predicted probabilities of all slices in each view are averaged to obtain the sequence-level probability. The sequence-level probabilities of the three views are averaged to obtain the final predicted probability of the overall MRI data. The final predicted probability is compared with a preset prediction threshold to determine whether there is a labral tear in the hip joint MRI image.

7. A device for predicting hip labral tears using MRI images based on the SwinT-Hybrid and DINOv2 models, characterized in that, include: The data acquisition module is used to collect standard MRI images labeled with hip labral tears as the result of arthroscopic examination within a preset time interval, as a hip labral tear prediction dataset, and to perform data preprocessing on the hip labral tear prediction dataset. The model building module is used to construct a hip labral tear prediction model based on the SwinT-Hybrid and DINOv2 fusion model, and to train the model using the preprocessed hip labral tear prediction dataset, including: Each frame slice in a single sequence of MRI images in the preprocessed hip labral tear prediction dataset is repeatedly expanded to the RGB channels and divided into non-overlapping 4×4 image patches. These patches are then converted into high-dimensional vectors through a fully connected layer and used as the model input for the hip labral tear prediction model. The input image patch vector is used for feature extraction through multiple SwinT-Hybrid Blocks; each SwinT-Hybrid Block includes a window multi-head self-attention module, a hybrid shift window multi-head self-attention module, a fully connected layer, a residual-based multilayer perceptron, and a residual connection; the feature extraction process downsamples layer by layer and increases the number of feature channels through patch merging layers; The DINOv2 model extracts global features and reduces their dimensionality before fusing them with the SwinT-Hybrid features. The fused features are then aggregated using max pooling, and the aggregated global feature vector is input into the classification head to complete the classification task through a multilayer perceptron. The model training uses the cross-entropy loss function, and the weight parameters are adjusted using the AdamW optimizer. The initial learning rate is set and dynamically adjusted using a cosine annealing strategy. The prediction module is used to process the real-time acquired hip labral MRI images and input them into the trained hip labral tear prediction model. The model output results are used to determine whether a hip labral tear exists.

8. An electronic device, comprising: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform each step of the method according to any one of claims 1-6.

Images