A Method and System for Pontine Infarction Segmentation and End-of-Stroke Prediction Based on Multimodal Joint Learning

By employing a multimodal joint learning method, a feature encoding network is constructed using wavelet transform, and multi-scale decomposition is performed using Haar wavelet filters. Combined with a dual-task guided fusion module and a Mamba global feature aggregation module, the problems of poor deconstructibility and information loss in lesion segmentation and END prediction tasks for pontine infarction lesions are solved, achieving high-precision lesion segmentation and risk assessment.

CN122337670APending Publication Date: 2026-07-03HANGZHOU DIANZI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610546366.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-04-23
Publication Date
2026-07-03

AI Technical Summary

Technical Problem

In existing technologies, the lesion segmentation and early neurological function deterioration (END) prediction task of pontine infarction has poor deconstructibility, lacks a structure-function evidence chain, and traditional CNNs result in the loss of spatial information, insufficient multimodal fusion, and difficulty in capturing non-local associations across slices.

Method used

A multimodal joint learning approach is adopted, which constructs a feature encoding network through wavelet transform, performs multi-scale decomposition using Haar wavelet filters, and combines a dual-task guided fusion module and a Mamba global feature aggregation module to achieve deep semantic alignment and long-range dependency modeling between images and clinical variables. A multimodal second-order fusion classifier is then constructed for prediction.

Benefits of technology

It achieves bidirectional structure-function constraints, preserves frequency domain information, realizes high-precision capture of pontine infarct lesions, improves the accuracy of END prediction and segmentation precision, dynamically balances gradient contributions between tasks, and significantly improves the preservation of details and risk assessment of pontine infarct lesions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122337670A_ABST
    Figure CN122337670A_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for pontine infarct segmentation and END prediction based on multimodal joint learning. The method includes acquiring and preprocessing multimodal data; constructing a wavelet transform-based feature encoding network to perform multi-scale decomposition and detail preservation of features; constructing a dual-task guided fusion module to align the deep semantics of clinical variables and imaging features and generate task-specific representations; constructing a Mamba-based global feature aggregation module to model sequence dependencies using a state-space model; constructing a multimodal second-order fusion classifier to enhance the clinical-image interaction modeling using second-order statistics; and employing a two-stage joint training strategy for training and prediction, and outputting the prediction results. This invention utilizes the DWT / IWT mechanism to significantly improve the accuracy of capturing small pontine infarct lesions; it achieves explicit interaction between segmentation evidence and prediction signals, significantly improving the segmentation accuracy of small lesions and the reliability of stroke risk assessment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of medical image processing and risk prediction technology, specifically to a method and system for pontine infarction segmentation and END prediction based on multimodal joint learning. Background Technology

[0002] Pontine infarction (PI) is a high-risk subtype of ischemic stroke, characterized by high rates of disability and mortality. Early neurological deterioration (END) is a key factor affecting patient prognosis, and accurate lesion segmentation and timely risk prediction are crucial for developing a treatment plan.

[0003] Existing research often treats lesion segmentation and prognostic prediction as independent tasks, resulting in poor task deconstruction. Segmentation tasks typically focus only on voxel-level shape fitting, lacking feedback from clinical prognostic signals. Classification tasks, on the other hand, fail to fully utilize pathological structural evidence such as lesion volume and spatial distribution, leading to a break in the structure-function evidence chain. Furthermore, in deep convolutional neural networks, traditional average pooling or max pooling operations can cause irreversible loss of spatial information, especially for small, poorly defined infarcts in the pons region, where downsampling easily smooths out crucial subtle features. Existing multimodal fusion methods often only fuse clinical variables and imaging features at a shallow level, resulting in insufficient fusion depth and a lack of customized feature representations for specific tasks (segmentation requires local details, prediction requires global semantics), easily leading to modality dominance imbalance. End-point prediction highly depends on the distribution of lesions at the whole-brain scale and the spatial continuity between slices, while traditional CNNs, limited by local receptive fields, struggle to capture complex cross-slice non-local associations. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for pontine infarction segmentation and END prediction based on multimodal joint learning, so as to solve the problems mentioned in the background art.

[0005] The present invention provides the following technical solution: a method for pontine infarction segmentation and END prediction based on multimodal joint learning, comprising the following operational steps: Step S1: Acquire multimodal data and perform preprocessing.

[0006] Preferably, the acquired multimodal data includes: three-dimensional magnetic resonance diffusion-weighted imaging data and corresponding structured clinical variable data.

[0007] Preferably, the pretreatment includes: Image preprocessing includes: resampling the images to a uniform voxel size, performing Z-score intensity normalization, and extracting continuous 2D slice sequences containing pontine lesion regions along the anatomical axis; Clinical data preprocessing includes: selecting multidimensional variables, standardizing them using mean-standard deviation, and using one-hot encoding for categorical variables.

[0008] Step S2: Construct a feature coding network based on wavelet transform to perform multi-scale decomposition and detail preservation of features.

[0009] Preferably, the wavelet transform-based feature coding network includes: in the encoder downsampling stage, performing four-channel decomposition of the feature map using a Haar wavelet filter with a set step size; generating low-frequency subbands and high-frequency subband sets. The low-frequency subband components contain the macroscopic structure and are passed to the next layer, while the high-frequency subbands in the high-frequency subband set are concatenated into residual terms for storage, used to reconstruct edge details via IWT in the decoding stage.

[0010] Step S3: Construct a dual-task guided fusion module to align the deep semantics of clinical variables and imaging features, and generate task-specific representations.

[0011] Preferably, the dual-task guided fusion module includes: mapping image features to query terms, mapping clinical variables to key terms and value terms, and calculating fusion features through a multi-head cross-modal attention mechanism; Preferably, generating a task-specific representation includes: Segmentation bottleneck representation: Directly projecting the fused features through convolution; The classification bottleneck is represented by channel concatenation of the fused features and the high-frequency local features extracted from the first two layers of the encoder.

[0012] Step S4: Construct a global feature aggregation module based on Mamba and use a state-space model to model sequence dependencies.

[0013] Preferably, the global feature aggregation module based on Mamba includes: modeling sequence dependencies using a state-space model; the serialization modeling includes: sorting the patches within the slice according to the Hilbert curve and splicing them in the anatomical order, introducing 3D position encoding to construct a long-range token sequence, and also introducing feature channel flipping to keep the token order unchanged, and constructing a supplementary representation by flipping the feature channel dimension.

[0014] Step S5: Construct a multimodal second-order fusion classifier and use second-order statistics to enhance the clinical-image interaction modeling.

[0015] Preferably, enhancing clinical-image interaction modeling using second-order statistics includes: Second-order feature capture: Calculate the covariance matrix between image tokens and clinical tokens using global covariance pooling, and then perform matrix power normalization. For prediction: The normalized second-order features are adaptively weighted and fused with the global classification tokens from the Mamba-based global feature aggregation module; The weighted fused features are processed by a multilayer perceptron to output the END risk probability.

[0016] Step S6: Use a two-stage joint training strategy for training and prediction, and output the prediction results.

[0017] Preferably, the training using a two-stage joint training strategy includes: The first stage jointly optimizes the segmentation and classification losses, and adopts an automatic loss weighting mechanism based on homoscedastic uncertainty to dynamically balance the gradient contributions of different tasks. The second stage introduces a weighted random sampling strategy to optimize the classification decision boundary.

[0018] On the other hand, the present invention also provides a pontine infarction segmentation and END prediction system based on multimodal joint learning, which executes the pontine infarction segmentation and END prediction method based on multimodal joint learning, and includes the following functional modules: Data acquisition and preprocessing module: used to acquire and resample DWI images, and at the same time normalize and encode structured clinical variables; Wavelet feature encoding module: used to reduce feature dimensionality and preserve lesion edge details using high-frequency residuals; Dual-task guided fusion module: Used to generate spatially sensitive features for segmentation tasks and morphological enhancement features for classification tasks by using clinical variables as guiding signals; Mamba global feature aggregation module: used to capture long-range dependencies between slice sequences; Multimodal second-order fusion classification module: used to enhance clinical-image correlation modeling through higher-order statistics to achieve END risk assessment; Joint learning execution module: used to execute the two-stage training logic and finally output the voxel-level segmentation mask and END prediction results.

[0019] Compared with the prior art, the beneficial effects achieved by the present invention are: (1) Achieved bidirectional structure-function constraints: Through the joint learning framework, the anatomical evidence extracted by the segmentation task directly serves the END prediction, while the risk discrimination signal of the prediction task guides the model to focus on the lesions in key functional areas.

[0020] (2) Preservation of frequency domain information: This invention utilizes the DWT / IWT mechanism to overcome the edge blurring problem caused by traditional CNN pooling, and significantly improves the capture accuracy of small pontine infarct foci.

[0021] (3) Task-specific deep fusion is achieved: The DGFM module constructed in this invention achieves customized alignment of images and clinical features, avoiding the modality-dominated problem.

[0022] (4) Efficient long-range modeling is achieved: The MGFA module constructed in this invention utilizes the Mamba architecture and FCF operator to achieve comprehensive spatial information integration at the patient level with low computational cost. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the end-to-end process from multimodal input to segmentation map and classification prediction provided in an embodiment of the present invention; Figure 2 This is a visual diagram of the segmentation results generated by PINet according to an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1 In this embodiment, combined with Figures 1-2 As shown, the present invention provides the following technical solution: a method for pontine infarction segmentation and END prediction based on multimodal joint learning, comprising the following operational steps: Step S1: Multimodal data acquisition and preprocessing.

[0026] In this embodiment, the present invention acquires three-dimensional magnetic resonance diffusion-weighted imaging (DWI) data of patients and corresponding structured clinical variable data.

[0027] Image preprocessing includes: resampling DWI images to a uniform voxel size (e.g., 1.0). 1.0 The sample size was 1.0 mm^3, and Z-score intensity normalization was performed. Then, continuous 2D slice sequences containing pontine lesion regions were extracted along the anatomical axis.

[0028] Clinical data preprocessing included: selecting multidimensional variables including age, NIHSS score, blood pressure, blood glucose and coagulation indicators, standardizing them by mean-standard deviation, and using one-hot coding for categorical variables.

[0029] For example, this invention selects a 65-year-old male patient as the analysis subject, specifically including: Clinical variables were extracted: 11 core variables were extracted from the electronic medical record system: age (65), admission NIHSS score (4 points), systolic blood pressure (160 mmHg), admission random blood glucose (7.2 mmol / L), neutrophil count, lymphocyte count, platelet count, vertebrobasilar artery stenosis (binary code: 1), international normalized ratio (INR), prothrombin time (PT), and D-dimer.

[0030] Standardization: Continuous variables are normalized to the [0,1] interval by mean-standard deviation, while categorical variables are encoded using one-hot encoding.

[0031] Image preprocessing: Acquire the patient's 3D DWI images (raw resolution) ).

[0032] Resampling: Resample the image to a uniform voxel size .

[0033] Intensity normalization: Z-score normalization is used.

[0034] Size adjustment: Extract 20 key slices along the Z-axis (depth direction), and crop / fill each slice to... Pixel.

[0035] Step S2: Construct a feature coding network based on wavelet transform.

[0036] In this embodiment, the present invention utilizes Discrete Wavelet Transform (DWT) and its inverse wavelet transform (IWT) to replace traditional max pooling / upsampling. The encoding process includes: using a Haar wavelet filter with a step size of 2 to decompose the feature map into four channels, generating a low-frequency subband LL and a high-frequency subband set (LH, HL, HH); wherein, the LL component contains the macroscopic structure and is passed to the next layer, and the high-frequency subbands are concatenated into a residual term H_l for storage, which is used to reconstruct edge details through IWT in the decoding stage to prevent the loss of small lesion signals.

[0037] For example, the feature encoding details based on wavelet transform in this invention include: in the encoder downsampling stage, this embodiment uses Haar wavelets to slice the input features. Decompose it.

[0038] DWT decomposition formula: The Haar wavelet filter bank consists of a low-pass filter... and Qualcomm Composition. For a two-dimensional image, its four sub-bands are calculated as follows: (Low frequency: Feature map size is halved, brightness distribution is approximate); (Horizontal high frequency: captures horizontal edges); (Vertical high frequency: Captures vertical edges); (Diagonal high frequency: captures corner points and fine textures).

[0039] Residual storage: After concatenating LH, HL, and HH, pass through a... Convolution compresses channels as high-frequency feature residuals The advantage of this operation over MaxPooling is that, when entering the decoder stage, it can be transformed using inverse wavelet transform (IWT). The details in the brain are restored without loss to the low-resolution LL component, thus accurately reconstructing tiny infarct foci with a diameter of less than 5 mm in the pons.

[0040] Step S3: Construct the Dual-Task Bootstrap Fusion Module (DGFM).

[0041] In this embodiment, deep semantic alignment of clinical variables and image features is achieved to generate task-specific representations. The cross-modal fusion process includes mapping image features to query terms. Mapping clinical variables to key terms Sum of values The fusion features are calculated through a multi-head cross-modal attention mechanism. Task-specific bottlenecks include: Bottleneck representation : directly through Convolution pairs To project; Classification bottleneck representation :Will High-frequency local features extracted from the first two layers of the encoder Channel splicing is performed to enhance the model's ability to identify the microscopic morphology of lesions.

[0042] For example, the interactive implementation of the Dual-Task Guided Fusion Module (DGFM) includes: Cross-modal attention computation: unfolding slice features into sequences Clinical features are mapped into vectors using MLP. .

[0043] Query / Key / Value calculations are represented as follows: in , , All are learnable weight matrices used to map input features to the query, key, and value spaces of the attention mechanism.

[0044] Attention Fusion The calculation is expressed as: in, The "value" vector in the attention mechanism, related to the query ( ),key( They jointly participate in the calculation of attention weights, and finally generate fused features through weighted summation. This is the scaling factor.

[0045] In this way, clinical variables (NIHSS scores) serve as guiding signals for weighting, enabling image features to focus more on specific anatomical regions (such as the location of the corticospinal tract) that are strongly correlated with functional deterioration.

[0046] For example, task branch generation includes: Split branches: directly on Perform convolutional decoding.

[0047] Prediction branch: Raw high-frequency components of layers 1 and 2 of the stitching encoder By utilizing its raw edge signals that have not undergone deep abstraction, the model's sensitivity to the risk of lesion expansion is enhanced.

[0048] Step S4: Construct the Mamba-based Global Feature Aggregation Module (MGFA).

[0049] In this embodiment, a state-space model (SSM) is used to model sequence dependencies at the whole-brain scale. The sequence modeling includes: sorting patches within the slice according to Hilbert curves and assembling them in anatomical order, introducing 3D positional encoding to construct a long-range token sequence. This invention also introduces Feature Channel Flipping (FCF) technology: while maintaining the token order, a supplementary representation is constructed by flipping the feature channel dimension, calculated as follows: in, These are the aggregated global features. The input sequence is used to achieve bidirectional context accumulation without increasing the number of scans.

[0050] For example, a long-range modeling process based on Mamba includes: Hilbert serialization: To preserve the local spatial correlation of lesions within a slice, this invention does not employ traditional row-first scanning, but instead traverses according to the Hilbert curve. The patch block. Then, the sequences of the 20 slices were spliced ​​end-to-end in anatomical depth order to form a patch block of length [length missing]. Long sequences.

[0051] FCF-Mamba computation: For each of the sequences The discretization evolution equation of SSM is: in, The hidden state vector at the current moment stores the context information for sequence modeling; The state transition matrix controls the hidden state from... Time's up The evolutionary process of time; The hidden state vector from the previous time step serves as the basis for updating the current state. The input weight matrix is ​​used to map the input token to the hidden state space; For the first in the sequence Tokens (such as feature units of medical image slices). The output vector at the current time step is obtained from the hidden state through a linear transformation. To output the weight matrix, the hidden states are mapped to the final output. In this invention, the FCF operator flips the channel dimension. This allows the model to model the dependencies between the forward anatomical sequence (from cephalic to pedipal) and the reverse anatomical sequence in a single forward propagation, without the need for dual-path scanning as in the traditional Bi-Mamba, reducing memory usage by approximately 40%.

[0052] Step S5: Construct a multimodal second-order fusion classifier (MSOFC).

[0053] In this embodiment, second-order statistics are used to enhance the clinical-image interaction modeling: Second-order feature capture includes: calculating the covariance matrix between image tokens and clinical tokens using global covariance pooling (GCP), and performing matrix power normalization (MPN) processing.

[0054] The prediction process includes: adaptively weighted fusion of normalized second-order features and global classification tokens from MGFA. The normalized second-order features (from Global Covariance Pooling (GCP)) and the global classification tokens from MGFA are used as input features, employing the following fusion formula: in, This represents the final feature vector after fusion, which serves as the input to a multilayer perceptron (MLP) to output the END risk probability. These are learnable weights; The global classification token comes from the Global Feature Aggregation Module (MGFA) and contains sequence dependency information at the whole-brain scale. The normalized second-order features are calculated by global covariance pooling (GCP) and reflect the statistical interaction between image tokens and clinical tokens. The fused features are processed by multilayer perceptron (MLP) to output the END risk probability.

[0055] For example, the details of the MSOFC second-order fusion classification include: GCP Feature Extraction: The feature sequence output by MGFA Calculate its second-order covariance matrix: Where Σ is the second-order covariance matrix with dimension ? ( (Number of feature channels), reflecting the statistical interaction between different channels in the feature sequence; The length of the feature sequence (number of samples) is the total number of tokens contained in the feature sequence output by MGFA. For the first in the feature sequence Token vectors Each vector has a dimension of , representing the multi-channel feature of a single feature unit; The mean vector of the feature sequence is obtained by applying it to all... The average value is used to calculate the overall distribution center that reflects the characteristics.

[0056] Matrix power normalization (MPN): To ensure numerical stability, for Perform eigenvalue decomposition to obtain This indicates that the covariance matrix Σ can be decomposed into the product of the eigenvector matrix V, the eigenvalue diagonal matrix Λ, and the transpose of V, thus achieving a diagonalized representation of Σ. The second-order eigenvalues ​​after matrix power normalization are obtained through... The calculation yielded, where This is a diagonal matrix formed by square roots of the eigenvalues ​​(diagonal elements are the square roots of the eigenvalues ​​in Λ), used to ensure numerical stability and preserve the nonlinear covariance between features. Compared to first-order mean pooling, this second-order feature can capture the complex nonlinear covariance between "image feature distribution" and "clinical status" more delicately, significantly improving the prediction accuracy of END.

[0057] Step S6: Two-stage joint training and prediction.

[0058] In this embodiment, the training strategy adopted by the present invention includes: in the first stage, joint optimization of segmentation and classification loss is carried out by adopting an automatic loss weighting mechanism based on homoscedasticity uncertainty to dynamically balance the gradient contributions between different tasks; in the second stage, a weighted random sampling strategy is introduced to optimize the classification decision boundary in order to address the class imbalance problem.

[0059] For example, the present invention was validated on a dataset of 386 patients with pontine infarction: Segmentation accuracy: The Dice coefficient reached 90.9%, reducing the rate of missed diagnosis of small lesions by 15%.

[0060] END prediction: AUC reaches 0.909, which is about 20% higher than methods that only use clinical indicators (such as traditional regression models).

[0061] Visualization: The segmentation edges generated by PINet in this invention closely match the clinical gold standard, and in cases predicted as "high risk of END", the model activation map accurately covers the affected area at the base of the pons.

[0062] Example 2 In this embodiment, the present invention provides a pontine infarction segmentation and END prediction system based on multimodal joint learning, comprising the following functional modules: Data acquisition and preprocessing module: used to acquire and resample DWI images, and at the same time normalize and encode structured clinical variables.

[0063] Wavelet feature encoding module: contains a DWT / IWT-based feature encoder and decoder, used to preserve lesion edge details by utilizing high-frequency residuals while reducing feature dimensionality.

[0064] Dual-task guided fusion module: Includes multi-head cross-modal attention unit, used to generate spatially sensitive features for segmentation tasks and morphological enhancement features for classification tasks by using clinical variables as guiding signals.

[0065] The Mamba global feature aggregation module contains a serialization unit and an FCF-Mamba processing unit, used to capture long-range dependencies between whole patient slice sequences.

[0066] Multimodal second-order fusion classification module: includes covariance pooling unit and adaptive fusion head, used to enhance clinical-image correlation modeling through higher-order statistics to achieve END risk assessment.

[0067] Joint learning execution module: contains a multi-loss balancing unit, which is used to execute the two-stage training logic and finally output the voxel-level segmentation mask and END prediction results.

[0068] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0069] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for pontine infarction segmentation and END prediction based on multimodal joint learning, characterized by: The following steps are included: Step S1: Acquire multimodal data and perform preprocessing; Step S2: Construct a feature coding network based on wavelet transform to perform multi-scale decomposition and detail preservation of features; Step S3: Construct a dual-task guided fusion module to align the deep semantics of clinical variables and imaging features, and generate task-specific representations; Step S4: Construct a global feature aggregation module based on Mamba and use a state-space model to model sequence dependencies; Step S5: Construct a multimodal second-order fusion classifier and use second-order statistics to enhance the clinical-image interaction modeling; Step S6: Use a two-stage joint training strategy for training and prediction, and output the prediction results.

2. The method for pontine infarction segmentation and END prediction based on multimodal joint learning according to claim 1, characterized in that: The multimodal data acquired in step S1 includes: three-dimensional magnetic resonance diffusion-weighted imaging data and corresponding structured clinical variable data.

3. The pontine infarction segmentation and END prediction method based on multimodal joint learning according to claim 2, characterized in that: The preprocessing in step S1 includes: Image preprocessing includes: resampling the images to a uniform voxel size, performing Z-score intensity normalization, and extracting continuous 2D slice sequences containing pontine lesion regions along the anatomical axis; Clinical data preprocessing includes: selecting multidimensional variables, standardizing them using mean-standard deviation, and using one-hot encoding for categorical variables.

4. The pontine infarction segmentation and END prediction method based on multimodal joint learning according to claim 3, characterized in that: The wavelet transform-based feature coding network in step S2 includes: during the encoder downsampling stage, the feature map is decomposed into four channels using a Haar wavelet filter with a set step size; the decomposition is divided into four sub-bands including low-frequency components, horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components.

5. The pontine infarction segmentation and END prediction method based on multimodal joint learning according to claim 4, characterized in that: The low-frequency subband component contains the macroscopic structure and is passed to the next layer. The horizontal high-frequency component, vertical high-frequency component and diagonal high-frequency component are spliced ​​together as a residual term for storage, which is used to reconstruct edge details through IWT during the decoding stage.

6. The pontine infarction segmentation and END prediction method based on multimodal joint learning according to claim 5, characterized in that: The dual-task guided fusion module in step S3 includes: mapping image features to query terms, mapping clinical variables to key and value terms, and calculating fusion features through a multi-head cross-modal attention mechanism; generating task-specific representations includes: Segmentation bottleneck representation: Directly projecting the fused features through convolution; The classification bottleneck is represented by channel concatenation of the fused features and the high-frequency local features extracted from the first two layers of the encoder.

7. The method for pontine infarction segmentation and END prediction based on multimodal joint learning according to claim 6, characterized in that: The Mamba-based global feature aggregation module in step S4 includes: Modeling sequence dependencies using a state-space model; The serialization modeling includes: sorting the patches within the slice according to the Hilbert curve and splicing them in the anatomical order; introducing 3D position encoding to construct a long-range token sequence; and introducing feature channel flipping to keep the token order unchanged and constructing a supplementary representation by flipping the feature channel dimension.

8. The method for pontine infarction segmentation and END prediction based on multimodal joint learning according to claim 7, characterized in that: Step S5, which utilizes second-order statistics to enhance clinical-image interaction modeling, includes: Second-order feature capture: Calculate the covariance matrix between image tokens and clinical tokens using global covariance pooling, and then perform matrix power normalization. For prediction: The normalized second-order features are adaptively weighted and fused with the global classification tokens from the Mamba-based global feature aggregation module; The weighted fused features are processed by a multilayer perceptron to output the END risk probability.

9. The method for pontine infarction segmentation and END prediction based on multimodal joint learning according to claim 8, characterized in that: Step S6 employs a two-stage joint training strategy, including: The first stage jointly optimizes the segmentation and classification losses, and adopts an automatic loss weighting mechanism based on homoscedastic uncertainty to dynamically balance the gradient contributions of different tasks. The second stage introduces a weighted random sampling strategy to optimize the classification decision boundary.

10. A pontine infarction segmentation and END prediction system based on multimodal joint learning, implementing the pontine infarction segmentation and END prediction method based on multimodal joint learning as described in any one of claims 1-9, characterized in that: Includes the following functional modules: Data acquisition and preprocessing module: used to acquire and resample DWI images, and at the same time normalize and encode structured clinical variables; Wavelet feature encoding module: used to reduce feature dimensionality and preserve lesion edge details using high-frequency residuals; Dual-task guided fusion module: Used to generate spatially sensitive features for segmentation tasks and morphological enhancement features for classification tasks by using clinical variables as guiding signals; Mamba global feature aggregation module: used to capture long-range dependencies between slice sequences; Multimodal second-order fusion classification module: used to enhance clinical-image correlation modeling through higher-order statistics to achieve END risk assessment; Joint learning execution module: used to execute the two-stage training logic and finally output the voxel-level segmentation mask and END prediction results.