Cervical LSIL progress risk prediction method and system based on multi-modal time sequence fusion

Abstract

The invention discloses a cervical LSIL progress risk prediction method and system based on multi-modal time sequence fusion, and the method comprises the steps: collecting multi-modal data, and carrying out the standardization processing; extracting dynamic change characteristics in continuous annual TCT liquid-based pictures through a convolutional neural network, and positioning a high-risk cell region; carrying out interval sensing position coding on HPV detection records, constructing an inter-modal causal attention mechanism, and establishing a time sequence causal relationship between HPV infection events and cell abnormal evolution; a discrete time competition risk model is adopted, the progression, regression and maintenance probabilities of different time periods in the future after primary diagnosis are synchronously output, a time-varying covariable LSTM is introduced, and the weight of patient features changing along with time is dynamically updated; and generating a cell evolution thermodynamic diagram, marking a high-risk area space-time evolution path, outputting a time influence curve, and marking a key risk accumulation time window. According to the invention, accurate quantitative evaluation of the cervical low-level lesion progress risk is realized.

Description

Technical Field

[0001] The present invention belongs to the field of medical artificial intelligence technology, and in particular to a method and system for predicting the risk of cervical LSIL progression based on multimodal time series fusion. Background Art

[0002] Low-grade squamous intraepithelial lesions (LSIL) of the cervix have heterogeneous characteristics of spontaneous regression, persistence, or progression to high-grade lesions (HSIL) and cervical squamous cell carcinoma (SCC). The core of the management of LSIL patients lies in balancing the monitoring of spontaneous regression with early intervention for high-risk patients, but existing technologies have significant shortcomings. Although current international guidelines (such as ASCCP and WHO) propose management strategies based on risk stratification, their evaluation model, which relies on static single data (such as HPV typing and single cytology results), has obvious limitations: (1) Insufficient integration of dynamic risk factors: HPV infection status (such as duration of infection, changes in viral load) and temporal characteristics of cytological images (such as evolution of nuclear atypia) have not been systematically analyzed. (2) Multimodal data fragmentation: Existing studies often analyze clinical data (age, smoking history) and image data (TCT images) in isolation, and have not achieved a deep correlation between HPV load, cell morphological characteristics, and patient medical history. (3) Low accuracy of clinical decision-making leads to waste of resources for low-risk patients, excessive colposcopy biopsies (increasing trauma and medical costs), and the risk of missed diagnosis for high-risk patients. (4) Poor adaptability of technical scenarios: Existing prediction systems are mostly based on retrospective single-center data, with limited generalization capabilities, and do not solve the problem of integrating clinical multi-source heterogeneous data (such as differences in TCT image quality standards across different institutions and inconsistent HPV detection methods). In addition, existing research methods attempt to address the above problems, but there are still deficiencies, mainly reflected in insufficient multimodal data fusion, limited dynamic risk assessment capabilities, and lack of clinical interpretability.

[0003] Patent CN109543719A proposes a method for diagnosing atypical cervical lesions based on a multimodal attention model, but it only focuses on static acetic acid and iodine image analysis, fails to integrate the dynamic characteristics of time-series TCT images and the dynamic changes of HPV infection, and lacks the ability to predict long-term risks. CN111180071A analyzed the relationship between HPV types and cervical precancerous lesions through clustering and statistical methods, but did not involve the deep correlation of multimodal data, especially ignoring the synergistic effect of the temporal evolution of cytological images and clinical characteristics. CN113920110A uses image analysis technology to detect cervical lesions, but its method is limited to image classification at a single time point and does not solve the problem of multimodal time-series data alignment and dynamic weight update. These existing technologies have failed to achieve the transition from "homogeneous follow-up" to "high-risk precision intervention." Therefore, there is an urgent need to build a prediction system that can deeply integrate multimodal features, dynamically update risk probabilities, and meet clinical interpretability requirements, so as to accurately identify high-risk subgroups that will progress to HSIL / SCC within 3 or 5 years and optimize the timing of intervention. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for predicting the risk of cervical LSIL progression based on multimodal time series fusion.

[0005] The technical solution to achieve the purpose of the present invention is as follows: In the first aspect, the present invention provides a method for predicting the risk of cervical LSIL progression based on multimodal time series fusion, comprising:

[0006] Collect time-axis aligned multimodal data and standardize the data;

[0007] A convolutional neural network was used to extract dynamic features from TCT liquid-based images over consecutive years, and a dynamic ViT encoder was used to locate high-risk cell areas.

[0008] HPV test records are encoded using interval-aware position coding. To address the irregularity of annual follow-up intervals, an inter-modal causal attention mechanism is constructed to establish a temporal causal relationship between HPV infection events and cell abnormality evolution.

[0009] A discrete-time competing risk model is used to simultaneously output the probabilities of progression, regression, and maintenance at different time periods after the initial diagnosis. A time-varying covariate LSTM is introduced to dynamically update the weights of patient characteristics over time.

[0010] Generate a cell evolution heat map, mark the spatiotemporal evolution path of high-risk areas, output the time influence curve, and identify the key risk accumulation time window.

[0011] In a second aspect, the present invention provides a cervical LSIL progression risk prediction system based on multimodal time series fusion, which is used to implement the above method. The system includes:

[0012] A data acquisition module for collecting time-axis aligned multimodal data of patients;

[0013] Data preprocessing module, used to standardize the collected data;

[0014] The dynamic feature extraction module uses a convolutional neural network to extract dynamic features from TCT liquid-based images collected annually and uses a dynamic ViT encoder to locate high-risk cell areas. It is used to implement interval-aware position encoding for HPV test records. To address the irregularity of annual follow-up intervals, it constructs an inter-modal causal attention mechanism to establish a temporal causal relationship between HPV infection events and the evolution of cellular abnormalities.

[0015] The multimodal time series fusion module uses a discrete-time competing risk model to simultaneously output the probabilities of progression, regression, and maintenance at different time periods after the initial diagnosis. It also introduces a time-varying covariate LSTM to dynamically update the weights of patient characteristics over time.

[0016] The visualization and risk identification module is used to generate cell evolution heat maps, mark the spatiotemporal evolution paths of high-risk areas, output time influence curves, and identify key risk accumulation time windows.

[0017] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above method when loaded into the processor.

[0018] Compared with the existing technology, the present invention has the following significant advantages: by integrating time-series TCT cytological feature analysis, time-attenuated position encoding and cross-modal attention mechanism, based on the discrete time competitive risk model, the present invention synchronously outputs the cumulative probability of progression, regression and maintenance in different future time periods after the initial diagnosis, and through cell evolution heat map visualization and clinical static feature embedding module, constructs a multi-modal time-series fusion cervical LSIL progression risk prediction system and method, breaking through the limitations of traditional static analysis and realizing the transformation from "homogeneous follow-up" to "high-risk precise intervention". BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Flowchart of the method for predicting the risk of cervical LSIL progression based on multimodal time series fusion. DETAILED DESCRIPTION

[0020] This paper addresses the core issues of existing cervical lesion risk prediction technologies, such as the fragmentation and processing of multimodal temporal information and the insufficient dynamic risk assessment capabilities. It proposes an intelligent prediction solution that integrates the temporal evolution of cytopathology with dynamic clinical characteristics, specifically addressing the following technical pain points:

[0021] 1) Multimodal time series data fusion defects: Breaking through the traditional method's reliance on single-modality static data, solving the cross-modal correlation modeling challenges between the dynamic temporal characteristics of cervical TCT images, discrete HPV infection records, and clinical characteristics;

[0022] 2) Long-term dynamic risk prediction gap: This approach fills the gap in existing technologies that can only assess short-term fixed-time point risks, and establishes a survival analysis framework that supports dynamic probability output over multiple time windows (1 / 3 / 5 / 10 years);

[0023] 3) Insufficient Clinical Interpretability: To address the poor interpretability of traditional deep learning models, a high-risk cell evolution visualization module driven by pathological features was designed, and clinical guideline constraints were embedded to ensure that the model's reasoning process adheres to medical logic. Through these technological innovations, this system achieves precise quantitative assessment of the progression risk of low-grade cervical lesions, providing clinicians with an intelligent decision-making support tool that combines temporal sensitivity, multimodal collaboration, and medical interpretability.

[0024] Combine Figure 1 The present invention constructs a dynamic risk assessment system that collaborates with a multimodal time series fusion network (MTF-Net) and an interpretable disease progression analysis framework. By integrating the temporal evolution characteristics of cervical cytology images, the dynamic changes of HPV infection, and clinical static characteristics, a long-term quantitative prediction method for the progression of cervical LSIL to HSIL / SCC based on multimodal time series fusion is established. The steps are as follows:

[0025] We collected time-aligned multimodal data, including patient time-series cervical TCT images, HPV test records (subtype, load), static clinical characteristics (age, marital history, smoking history, etc.), and follow-up data. These data were then standardized, including image staining normalization and interpolation of non-uniformly spaced time series (to handle missing follow-up records).

[0026] A convolutional neural network was used to extract dynamic features such as abnormal cell density and nuclear-cytoplasmic ratio from consecutive annual TCT liquid-based images. A dynamic ViT encoder was used to locate high-risk cell areas (such as nuclear-cytoplasmic ratio, chromatin openness, nuclear membrane regularity, and cytoplasmic hollowing status).

[0027] Interval-aware position encoding (IAPE) is used for HPV test records. To address the irregularity of annual follow-up intervals, an intermodal causal attention (MCA) mechanism is constructed to establish a temporal causal relationship between HPV infection events and cell abnormality evolution.

[0028] A discrete-time competing risk model is used to simultaneously output the probabilities of progression, regression, and maintenance over the next τ years. Time-varying covariates (LSTMs) are introduced to dynamically update the weights of patient characteristics over time.

[0029] Generate a cell evolution heat map, mark the spatiotemporal evolution path of high-risk areas, output the time influence curve, and identify the key risk accumulation time window.

[0030] Each sub-step of the present invention is described in detail below.

[0031] Step 1: Data Collection

[0032] Methods: Cervical TCT imaging data of patients with LSIL were retrospectively collected. A digital pathology scanner was used to collect cervical TCT slides from consecutive years. High-resolution (40x objective lens) full-field digital images (WSI format) were obtained. Each patient had data from at least three annual examinations, and slides with substandard quality (uneven Papanicolaou staining, substandard cell count, or artifacts) were excluded.

[0033] HPV testing data of LSIL patients were collected retrospectively. HPV typing test results (including 14 high-risk types such as 16 / 18 / 31 / 33), viral load (RLU / CO value) and test time were extracted from the hospital LIS system, and the test methods (such as Cobas4800 and HC2) were recorded.

[0034] Static characteristic data such as age, gravidity, and smoking history were retrospectively collected from the electronic medical records of LSIL patients.

[0035] Follow-up data of LSIL patients on spontaneous regression, persistence, or progression to HSIL / SCC at each follow-up visit after initial diagnosis were retrospectively collected.

[0036] Step 2: Data Preprocessing

[0037] Image normalization: The Macenko staining normalization algorithm was used to eliminate staining differences, and the HSV color space threshold was set to locate the effective cell area. The WSI images were segmented into 512 × 512 pixel blocks, and blank areas (blocks with cell density <5%) were filtered out. Non-uniformly spaced time series were constructed, and virtual time nodes were generated using cubic spline interpolation for missing annual inspection data.

[0038] HPV dynamic sequence construction: The follow-up period was divided into monthly time windows, and the HPV load data at undetected time points were supplemented by linear interpolation to construct a three-dimensional feature vector, including: subtype toxicity score (0-5 points), load log value, and persistence marker.

[0039] Clinical feature codes:

[0040] Discrete variables were one-hot coded: smoking history (0 / 1), gravidity (0 / 1 / 2 / ≥3), and continuous variables were standardized: age (Z-score normalization), age at first sexual intercourse (Min-Max normalization).

[0041] The above data were time-aligned to establish a unified time coordinate system with the first LSIL diagnosis as the time origin. A sliding window alignment was used for the asynchronous detection data with a window width of ±90 days.

[0042] Step 3: Dynamic Feature Extraction

[0043] The method uses ResNet-34 and temporal self-attention to extract dynamic features of TCT images and locate high-risk cell regions. It also utilizes interval-aware position encoding to address irregular HPV detection intervals. An inter-modality causal attention mechanism is constructed to establish the temporal causal relationship between HPV infection and abnormal cell evolution. Finally, causal significance verification ensures that the unidirectional causal relationship between HPV infection events and cell feature changes conforms to biological laws. Dynamic feature extraction involves two steps: image feature extraction and HPV dynamic encoding.

[0044] Image feature extraction:

[0045] A ResNet-34 network was used to extract single-time point features, removing the fully connected layers and outputting a 1024-dimensional feature vector. A temporal self-attention module was used to calculate the dynamic trends of cell populations across three consecutive TCT examinations. Finally, spatial attention was applied in the Transformer decoding layer to localize high-risk areas across time points.

[0046] Establishing temporal causal relationships in the evolution of HPV-cell abnormalities:

[0047] 1) Interval-aware position encoding (IAPE) generation:

[0048] For the time t of the i-th HPV test i , calculate the interval with the previous detection point: Δt i =t i -t i-1 (i≥1,Δt0=0);

[0049] Design an exponential decay encoding function to increase the influence weight of recent events:

[0050]

[0051] Where D is the encoding dimension, d is the dimension index currently being calculated, γ is the decay rate, and λ is the balance factor;

[0052] 2) Constructing Inter-modal Causal Attention (MCA):

[0053] Construct the lower triangle mask matrix Where T is the total time step of patient follow-up, ensuring that HPV infection events can only affect cell characteristics at the time of occurrence and thereafter:

[0054]

[0055] M ij For the elements of the triangular mask matrix M, in the attention calculation, -∞ will make the attention weight of the corresponding position approach 0, completely shielding future information to ensure that the HPV infection event can only affect the time point of its occurrence and subsequent changes in cell characteristics.

[0056] Define the cross attention between HPV feature H and cell feature C, and calculate the correlation strength between HPV load and cell atypia by cross attention:

[0057]

[0058] Where W Q 、W K 、W V is a learnable parameter, d k is the scaling factor;

[0059] 3) Causal significance verification was performed to ensure that the unidirectional causal relationship between HPV infection events and changes in cellular characteristics was consistent with biological laws; the temporal order of HPV features was randomly disrupted and the inter-modal attention weights were recalculated, and a null distribution was generated through 1000 Monte Carlo simulations; if the percentile of the original attention score was greater than 95% (p < 0.05), the null hypothesis was rejected and the causal directionality was confirmed.

[0060] Step 4: Multimodal time series fusion

[0061] Multimodal time series input construction:

[0062] 1) Taking the time of first LSIL diagnosis as the time origin t0, align the time axes of all modal data, construct the time step t, and fill in the missing time points by cubic spline interpolation.

[0063] 2) Construct the feature vector of each time step t as:

[0064]

[0065] Among them: F image (t)∈R 1024 is the image feature output by the dynamic ViT encoder, F HPV (t)∈R 64 is the HPV dynamic feature output by the inter-modal causal attention (MCA), F clinical ∈R10 is the standardized static clinical feature; this step realizes the temporal alignment and feature fusion of multimodal data, providing standardized input for subsequent model building.

[0066] Time-varying covariate LSTM establishment:

[0067] 1) Build a two-layer stacked LSTM structure to output dynamic states:

[0068] h t ,c t =LSTM(X(t),h t-1 ,c t-1 )

[0069] Among them, h t is the short-term memory of LSTM at time step t, c t It is the long-term memory carrier of LSTM.

[0070] 2) Based on the above structure, generate dynamic weights:

[0071] w(t)=Softmax(MLP(h t ))·e -βt

[0072] Among them, MLP is a single-layer fully connected network, e -βt is the time decay term, and β is the time decay coefficient.

[0073] 3) The original features are then dynamically weighted to characterize how the importance of features changes over time:

[0074] X′(t)=w(t)⊙X(t)

[0075] The obtained time-varying pattern of feature importance can be used as input for calculating the risk probability of cervical LSIL progression;

[0076] Discrete-time competing risk calculation:

[0077] 1) For each time step t, calculate the risk probability of event k∈{progress, regression, maintenance}:

[0078]

[0079] Among them, β k is the feature weight vector of event k, γ k is the time trend coefficient of event k; the loss function L for model training is established based on the risk probability of event k;

[0080] 2) For the prediction time window τ years, in this embodiment, τ∈{1,3,5,10}, the cumulative incidence rate of event k is:

[0081]

[0082] Output after model training;

[0083] Model training:

[0084] 1) Combine partial likelihood loss and L2 regularization term:

[0085]

[0086] Where N is the sample size, T i is the effective follow-up time step of patient i, y ik (t)∈{0,1} indicates whether event k occurs to patient i at time t, and α is the regularization coefficient.

[0087] 2) Add dynamic weight constraints:

[0088]

[0089] Among them, G is the subset of patients with persistent HPV16 infection, w HPV16 is the dynamic feature weight of HPV16 viral load at time step t, with an average weight ≥ 0.6.

[0090] Result output:

[0091] For any patient, output the cumulative risk probability of each time window:

[0092] Output=[CIF1(1),CIF1(2),CIF1(…),CIF1(τ)]

[0093] Where CIF1(τ) represents the probability of progression to HSIL / SCC within τ years. The 95% confidence interval of CIF1(τ) was then estimated using the bootstrap method.

[0094] For example, in this embodiment, Output = [CIF1(1), CIF1(3), CIF1(5), CIF1(10)]

[0095] Step 5: Cell evolution heat map and risk time window identification

[0096] 1) Cell evolution heat map generation:

[0097] Extracting spatial attention weights:

[0098]

[0099] Among them, x, y are the two-dimensional space coordinates of the image, t is the time step, F k is the feature of the kth attention head, α k is the gradient weight of the corresponding channel. Accumulating activation values along the time axis generates a heat distribution across the time window:

[0100]

[0101] Among them, β is the time decay coefficient, and |t-τ| is the time interval between the historical time t and the current time τ.

[0102] 2) Spatiotemporal evolution of high-risk areas:

[0103] Perform DBSCAN clustering on the activation area of each time slice to identify stable lesions:

[0104]

[0105] Among them, x i 、y i is the spatial coordinate of the i-th pixel, Cluster n t is the clustering result;

[0106] Match the lesion areas of adjacent time slices and construct the evolution trajectory:

[0107]

[0108] An intersection over union (IoU) > 0.5 is considered to be the continuation of the same lesion.

[0109] 3) Key risk accumulation time window identification

[0110] Detect the mutation point of the time influence curve:

[0111]

[0112] Where I(k) is the Shapley value contribution at time point k, T is the total time step, μ0 is the average value of the Shapley values at all time points; S t is the cumulative deviation value in the time series, when the cumulative sum | S t ∣ Exceeds the threshold and determines time t as the key mutation point.

[0113] The present invention solves the problem of missing non-equally spaced time series data by generating virtual time nodes based on cubic spline interpolation, and realizes multimodal time axis synchronization of TCT images, HPV dynamic load and clinical characteristics by combining sliding window alignment; designs interval-aware position encoding, dynamically enhances the influence weight of recent HPV events on cell atypia through exponential decay function, and constructs an inter-modal causal attention mechanism, using the lower triangular mask matrix to force HPV infection events to only transmit associations to the cell features of subsequent time slices, and performs causal verification to ensure biological rationality.

[0114] This paper adopts a two-layer stacked LSTM structure and introduces a time decay factor to generate time-varying feature weights, achieving dynamic nonlinear mapping of multimodal feature importance. It also proposes a discrete-time competing risk cumulative probability calculation framework, quantifies the risk probability of patient progression, regression, or maintenance based on a partial likelihood loss function, and embeds a time trend coefficient to characterize the risk acceleration effect. It also constrains the dynamic weights of patients with persistent HPV16 infection to enhance clinical interpretability.

[0115] The present invention generates a heat map across time windows through spatial attention weight gradient accumulation, combines DBSCAN clustering with intersection-over-union threshold matching of lesion areas in adjacent time slices, and constructs the spatiotemporal evolution trajectory of cell atypia. It designs a mutation point detection algorithm for the time influence curve based on Shapley value contribution, dynamically identifies high-risk time windows through accumulation and thresholding, and realizes the quantitative positioning of key nodes of lesion progression.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include these modifications and variations.

Claims

1. A method for predicting the progression risk of cervical LSIL based on multimodal time series fusion, characterized by: include: Collect time-axis aligned multimodal data and standardize the data; A convolutional neural network was used to extract dynamic features from TCT liquid-based images over consecutive years, and a dynamic ViT encoder was used to locate high-risk cell areas. HPV test records are encoded using interval-aware position coding. To address the irregularity of annual follow-up intervals, an inter-modal causal attention mechanism is constructed to establish a temporal causal relationship between HPV infection events and cell abnormality evolution. A discrete-time competing risk model is used to simultaneously output the probabilities of progression, regression, and maintenance at different time periods after the initial diagnosis. A time-varying covariate LSTM is introduced to dynamically update the weights of patient characteristics over time. Generate a cell evolution heat map, mark the spatiotemporal evolution path of high-risk areas, output the time influence curve, and identify the key risk accumulation time window.

2. The method for predicting cervical LSIL progression risk based on multimodal time series fusion according to claim 1, characterized in that: The time-axis aligned multimodal data includes the patient's time-series cervical TCT images, HPV test records, clinical static characteristics and follow-up result data.

3. The method for predicting cervical LSIL progression risk based on multimodal time series fusion according to claim 2, characterized in that: The acquisition time axis alignment multimodal data is specifically as follows: Cervical TCT image data of patients with LSIL were retrospectively collected. A digital pathology scanner was used to collect consecutive annual cervical TCT slides of patients, obtaining high-resolution, full-field digital images. Each patient had data from at least three annual examinations, and substandard slides were excluded. HPV testing data of LSIL patients were retrospectively collected, and HPV typing test results, viral load, and test time were extracted from the hospital LIS system, and the test methods were recorded; Static characteristic data from the electronic medical records of LSIL patients were retrospectively collected, including age, gravidity, and smoking history; Follow-up data of LSIL patients on spontaneous regression, persistence, or progression to HSIL / SCC at each follow-up visit after initial diagnosis were retrospectively collected.

4. The method for predicting cervical LSIL progression risk based on multimodal time series fusion according to claim 1, characterized in that: The data standardization process is specifically as follows: Image normalization: The Macenko staining normalization algorithm was used to eliminate staining differences, and the HSV color space threshold was set to locate the effective cell area. The WSI image was segmented into 512 × 512 pixel blocks, and blank areas were filtered. Non-uniformly spaced time series were constructed, and virtual time nodes were generated using cubic spline interpolation for missing annual inspection data. Constructing an HPV dynamic sequence: Divide the follow-up period into monthly time windows, use linear interpolation to supplement the HPV load data at undetected time points, and construct a three-dimensional feature vector, including: subtype toxicity score, load log value, and persistence marker; Clinical characteristic coding included one-hot coding of discrete variables and standardization of continuous variables: one-hot coding was performed on discrete variables, including smoking history and gravidity, and standardization was performed on continuous variables, including age and age at first sexual intercourse; The above data were time-aligned to establish a unified time coordinate system with the first LSIL diagnosis as the time origin. A sliding window alignment was used for the asynchronous detection data with a window width of ±90 days.

5. The method for predicting cervical LSIL progression risk based on multimodal time series fusion according to claim 1, characterized in that: The method extracts dynamic change features from consecutive annual TCT liquid-based images through a convolutional neural network and locates high-risk cell areas using a dynamic ViT encoder, specifically: ResNet-34 is used to extract single-time point features, remove the fully connected layer, and output a 1024-dimensional feature vector. The temporal self-attention module is used to calculate the dynamic change trend of the cell population of three adjacent TCT examination results, and finally spatial attention is applied in the Transformer decoding layer to achieve cross-time point positioning of high-risk areas.

6. The method for predicting cervical LSIL progression risk based on multimodal time series fusion according to claim 1, characterized in that: The HPV test records are encoded using interval-aware position coding. To address the irregularity of annual follow-up intervals, an inter-modal causal attention mechanism is constructed to establish a temporal causal relationship between HPV infection events and abnormal cell evolution. Specifically: (1) Generate interval-aware position encoding IAPE: For the time t of the i-th HPV test i , calculate the interval with the previous detection point: Δt i =t i -t i-1 , i≥1,Δt0=0; Design an exponential decay encoding function to increase the influence weight of recent events: Where D is the encoding dimension, d is the dimension index currently being calculated, γ is the decay rate, and λ is the balance factor; (2) Constructing inter-modal causal attention MCA; Construct the lower triangle mask matrix Where T is the total time step of patient follow-up, ensuring that HPV infection events can only affect cell characteristics at the time of occurrence and thereafter: Define the cross attention between HPV feature H and cell feature C, and calculate the correlation strength between HPV load and cell atypia by cross attention: Where W Q 、W K 、W V is a learnable parameter, d k is the scaling factor; (3) Causal significance verification was used to ensure that the unidirectional causal relationship between HPV infection events and changes in cell characteristics was consistent with biological laws; the temporal order of HPV features was randomly disrupted and the inter-modal attention weights were recalculated, and a null distribution was generated through 1000 Monte Carlo simulations; if the percentile of the original attention score was greater than 95%, the null hypothesis was rejected and the causal directionality was confirmed.

7. The method for predicting cervical LSIL progression risk based on multimodal time series fusion according to claim 1, characterized in that: A discrete-time competing risk model is used to simultaneously output the probabilities of progression, regression, and maintenance in the next τ years. A time-varying covariate LSTM is introduced to dynamically update the weights of patient characteristics over time. Specifically, (1) Constructing multimodal time series input: Taking the time of first LSIL diagnosis as the time origin t0, the time axes of all modal data were aligned, the time step t was constructed, and missing time points were filled by cubic spline interpolation; Construct the feature vector for each time step t as: Among them: F image (t)∈R 1024 is the image feature output by the dynamic ViT encoder, F HPV (t)∈R 64 is the dynamic feature of HPV output of inter-modal causal attention, F clinical ∈R10 is the standardized static clinical feature; this step realizes the temporal alignment and feature fusion of multimodal data, providing standardized input for subsequent model building; (2) Establishing time-varying covariate LSTM: Build a two-layer stacked LSTM structure to output dynamic state: h t ,c t =LSTM(X(t),h t-1 ,c t-1 ) Among them, h t is the short-term memory of LSTM at time step t, c t It is the long-term memory carrier of LSTM; Based on the above structure, dynamic weights are generated: w(t)=Softmax(MLP(h t ))·e -βt Among them, MLP is a single-layer fully connected network, e -βt is the time decay term, β is the time decay coefficient; The original features are then dynamically weighted to represent how the importance of features changes over time: X′(t)=w(t)⊙X(t) The obtained time-varying pattern of feature importance is used as input for calculating the risk probability of cervical LSIL progression; (3) Calculation of discrete-time competing risk: For each time step t, calculate the risk probability of event k∈{progress, regression, maintenance}: Among them, β k is the feature weight vector of event k, γ k is the time trend coefficient of event k; the loss function L for model training is established based on the risk probability of event k; For the prediction time window τ years, the cumulative incidence of event k is: Output after model training; (4) Model training: Combine partial likelihood loss with L2 regularization term: Where N is the sample size, T i is the effective follow-up time step of patient i, y ik (t)∈{0,1} indicates whether patient i experiences event k at time t, and α is the regularization coefficient; Adding dynamic weight constraints Among them, G is the subset of patients with persistent HPV16 infection, w HPV16 is the dynamic feature weight of HPV16 viral load at time step t, with an average weight ≥ 0.6; (5) Result output: For any patient, output the cumulative risk probability of each time window: Output=[CIF1(1),CIF1(2),...,CIF1(τ)] Where CIF1(τ) represents the probability of progression to HSIL / SCC within τ years; the Bootstrap method was then used to estimate the 95% confidence interval of CIF1(τ).

8. The method for predicting cervical LSIL progression risk based on multimodal time series fusion according to claim 1, characterized in that: Generate a cell evolution heat map, mark the spatiotemporal evolution path of high-risk areas, output the time influence curve, and identify the key risk accumulation time window, specifically: (1) Generate cell evolution heat map: Extracting spatial attention weights: Among them, x, y are the two-dimensional space coordinates of the image, t is the time step, F k is the feature of the kth attention head, α k is the gradient weight of the corresponding channel; the activation value is accumulated along the time axis to generate the thermal distribution H(x,y) across the time window: Among them, β is the time decay coefficient, |t-τ| is the time interval between the historical time t and the current time τ; (2) Marking the spatiotemporal evolution paths of high-risk areas: Perform DBSCAN clustering on the activation area of each time slice to identify stable lesions: Among them, x i 、y i is the spatial coordinate of the i-th pixel, Cluster n t is the clustering result; Match the lesion areas of adjacent time slices and construct the evolution trajectory Cost ij : Intersection over Union (IoU) > 0.5 is considered as the continuation of the same lesion; (3) Identify the key risk accumulation time window Detect the mutation point of the time influence curve: Where I(k) is the Shapley value contribution at time point k, T is the total time step, μ0 is the average value of the Shapley values at all time points; S t is the cumulative deviation value in the time series, when the cumulative sum | S t ∣ Exceeds the threshold and determines time t as the key mutation point.

9. A cervical LSIL progression risk prediction system based on multimodal time series fusion, characterized by: For implementing the method described in any one of claims 1 to 8, the system comprises: A data acquisition module for collecting time-axis aligned multimodal data of patients; Data preprocessing module, used to standardize the collected data; The dynamic feature extraction module uses a convolutional neural network to extract dynamic features from TCT liquid-based images collected annually and uses a dynamic ViT encoder to locate high-risk cell areas. It is used to implement interval-aware position encoding for HPV test records. To address the irregularity of annual follow-up intervals, it constructs an inter-modal causal attention mechanism to establish a temporal causal relationship between HPV infection events and the evolution of cellular abnormalities. The multimodal time series fusion module uses a discrete-time competing risk model to simultaneously output the probabilities of progression, regression, and maintenance at different time periods after the initial diagnosis. It also introduces a time-varying covariate LSTM to dynamically update the weights of patient characteristics over time. The visualization and risk identification module is used to generate cell evolution heat maps, mark the spatiotemporal evolution paths of high-risk areas, output time influence curves, and identify key risk accumulation time windows.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is loaded into a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images