A tumor image tracking method and system based on lesion identity field and counterfactual spatiotemporal calibration

Abstract

This invention relates to the fields of medical image processing, artificial intelligence, and tumor follow-up assessment, and particularly to a tumor image tracking method and system based on lesion identity fields and counterfactual spatiotemporal calibration. The method includes: acquiring medical image sequences and clinical context data from multiple time points for the same subject; obtaining spatiotemporally aligned image data through cross-time point standardization and anatomical region constraint registration; obtaining the identity embedding, spatial occupancy, and morphological representation of each candidate lesion using a lesion identity field generation network; constructing a longitudinal lesion memory map and updating the inter-period correspondence based on reversible deformation consistency constraints; and separating the apparent changes caused by scanning protocol changes, registration errors, treatment-related alterations, and actual tumor evolution through a counterfactual spatiotemporal calibration module to obtain calibrated tracking results. This method can maintain lesion identity consistency and reduce the probability of mismatches and false progression judgments.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing, artificial intelligence, and tumor follow-up assessment, and in particular to a tumor image tracking method and system based on lesion identity field and counterfactual spatiotemporal calibration. Background Technology

[0002] In the diagnosis, treatment, and follow-up of tumors, medical imaging is crucial for observing the size, shape, location, enhancement pattern, metabolic activity, and dynamic changes of lesions. Commonly used follow-up imaging methods include CT, MRI, PET / CT, PET / MRI, ultrasound, and digital pathology imaging. For the same subject, tumor imaging tracking typically requires comparing examination results at multiple time points to identify past lesions, new lesions, disappeared lesions, and lesions that may have progressed or remitted, thereby creating a verifiable longitudinal record of changes. However, existing technologies still face several shortcomings in this scenario.

[0003] First, existing medical imaging AI algorithms mostly focus on lesion detection, segmentation, or classification at a single time point. While these methods can output the location, outline, or category of a lesion at a given examination moment, they have significant limitations in longitudinal follow-up scenarios. Medical images from different time points may originate from different scanning devices, protocols, reconstruction parameters, or enhancement phases, leading to non-tumor biological differences in image grayscale, boundary clarity, and local texture of the same lesion. Furthermore, the patient's position, respiratory status, organ filling level, and soft tissue deformation may all change during different examinations. Simply relying on spatial distance or image registration results for lesion matching can easily result in the loss of identification of the same lesion, confusion of adjacent lesions, or mismatches across different time periods. In addition, during post-treatment follow-up, imaging changes related to inflammation, edema, necrosis, fibrosis, radiotherapy reactions, or immunotherapy may cause short-term lesion enlargement, density changes, or enhancement changes. Tracking solely based on changes in volume or long axis may misjudge non-tumor progression as lesion progression.

[0004] Secondly, while some solutions combine lesion segmentation, image registration, and time-series modeling, these solutions typically splice different modules sequentially, lacking a mechanism to maintain lesion identity across multiple time points. Existing solutions often treat "discovering lesions at each time point" and "matching lesions at different time points" as two independent tasks, failing to establish a consistent identity representation of the same lesion across time points during the feature learning stage. Therefore, when lesions split, merge, have blurred boundaries, undergo morphological changes after treatment, or are densely distributed with multiple lesions, the system is prone to problems such as lesion number jumps, lesion trajectory breaks, and duplicate counting of the same lesion.

[0005] Furthermore, existing longitudinal tracking methods have limited explanatory power for the causes of image changes. Apparent changes in lesions may be caused by actual tumor growth, or by variations in scan slice thickness, differences in enhancement phase, differences in reconstructed nuclear structures, respiratory motion, registration errors, or treatment-related responses. If AI algorithms cannot calibrate for these factors, they will struggle to provide stable and verifiable tracking results. For clinical auxiliary applications, the system needs not only to provide numerical values ​​for whether a lesion has "grown" or "shrank," but also to explain the extent to which this change is influenced by acquisition conditions, registration reliability, and the treatment context.

[0006] In summary, existing technologies urgently need a tumor image tracking solution that can maintain consistent lesion identity across multiple time points in medical images, jointly handle spatial deformation and image protocol differences, and perform counterfactual calibration on apparent changes. This solution should not be a simple combination of detection, segmentation, and registration networks, but rather should form a unified technical mechanism within the algorithm structure that matches the longitudinal lesion tracking task, enabling collaborative modeling of lesion identity, spatial deformation, morphological changes, and contextual factors within the same framework. Summary of the Invention

[0007] In order to overcome the above-mentioned defects of the prior art, the present invention provides a tumor image tracking method and system based on lesion identity field and counterfactual spatiotemporal calibration, so as to solve the problems existing in the background art.

[0008] This invention provides the following technical solution: a tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration, comprising the following steps: Acquire medical image sequences and associated clinical context data of the same subject collected at multiple time points; perform cross-time point standardization, anatomical region constraint registration and modality consistency processing on the medical image sequences to obtain spatiotemporally aligned image data; The spatiotemporally aligned image data is input into the lesion identity field generation network to obtain the identity embedding, spatial occupancy distribution and morphological characterization of each candidate tumor lesion at different time points. A longitudinal lesion memory map is constructed based on the identity embedding, the spatial occupancy distribution, and the morphological representation, and the inter-period correspondence between each lesion node is updated through reversible deformation consistency constraints. The longitudinal lesion memory map is input into the counterfactual spatiotemporal calibration module to separate the apparent changes caused by changes in scanning protocol, registration error, treatment-related image changes and actual tumor evolution, and obtain the calibrated tumor tracking results.

[0009] Furthermore, the medical imaging sequence includes one or more of CT images, MRI images, PET images, ultrasound images, and digital images of pathological sections; the associated clinical context data includes one or more of the following: acquisition time, scan slice thickness, reconstructed nucleus, enhancement phase, patient position, treatment time, treatment category, marked lesion area, and organ segmentation results.

[0010] Furthermore, the process of performing cross-time point standardization, anatomical region constraint registration, and modality consistency on the medical image sequence to obtain spatiotemporally aligned image data includes: Voxel spacing resampling, grayscale or signal intensity normalization, and noise suppression processing were performed on medical images at different time points. Generate an anatomical constraint mask based on the segmentation results of the target organ or anatomical region; Rigid registration, affine registration, and non-rigid registration are performed within the anatomical constraint mask to obtain the deformation field across time points; Based on the cross-time point deformation field, medical images from multiple time points are mapped to a unified reference coordinate system to obtain the spatiotemporally aligned image data.

[0011] Furthermore, the lesion identity field generation network includes a shared image encoder, a local lesion decoder, and an identity embedding head; the output of the lesion identity field generation network satisfies: in, Indicates the first Spatiotemporally aligned image data at each time point Indicates the first Anatomical constraint mask corresponding to each time point Indicates the first Data collection and clinical context features corresponding to each time point The parameter is The lesion identity field generation network, Indicates the first The first time point Embedding of the identity of each candidate lesion Indicates the first The first time point Spatial distribution of candidate lesions Indicates the first The first time point Morphological characterization of candidate lesions.

[0012] Furthermore, the identity embedding is used to characterize the consistency of the identity of the same lesion at different time points; the spatial occupancy distribution is used to characterize the probabilistic occupancy area of ​​the candidate lesion in three-dimensional space; the morphological characterization includes one or more of the following: lesion volume, major axis, minor axis, boundary irregularity, image texture features, enhancement features, and metabolic uptake features.

[0013] Furthermore, the longitudinal lesion memory map includes lesion nodes and inter-period candidate edges. Each lesion node corresponds to a candidate lesion at a given time point, and each inter-period candidate edge represents a candidate relationship where two candidate lesions at different time points belong to the same tumor lesion. The matching score of the inter-period candidate edge satisfies: in, Indicates the first The first time point The candidate lesion and the first The first time point Interphase matching scores between candidate lesions , , , This represents the preset or learnable weight coefficients. This represents an identity embedding similarity function. and These represent the identity embeddings of the two candidate lesions, respectively. Indicates by the first The time point is mapped to the first Deformation field at each time point and These represent the spatial centers of the two candidate lesions. Represents the spatial distance function. Indicates spatial distance and temperature parameters. and These represent the morphological characteristics of the two candidate lesions. Represents the norm distance. Temperature parameters representing morphological differences This indicates uncertainty in intertemporal matching.

[0014] Furthermore, the reversible deformation consistency constraint satisfies: in, This represents the consistency loss of reversible deformation. This represents the spatial coordinate domain of the image to be registered. Represents the position of any voxel in the coordinate domain. This indicates the number of voxels within the coordinate domain. Indicates by the first The time point is mapped to the first Deformation field at each time point Indicates by the first The time point is mapped to the first The reverse deformation field at each time point Represents the square of the L2 norm. Indicates the deformation field at position Spatial gradient at that location This represents the smoothing constraint coefficient.

[0015] Furthermore, the counterfactual spatiotemporal calibration module is used to generate a counterfactual morphological representation of the same lesion under the assumptions of consistent scanning protocols, limited registration errors, and suppressed treatment-related image changes, and to obtain a calibration progress score based on the difference between the observed morphological representation and the counterfactual morphological representation; the calibration progress score satisfies: in, Indicates the first The identity of the longitudinal lesion in the first Calibration progress score at each time point This represents the Sigmoid function. Represents a learnable weight vector. Indicates the bias term. Indicates the first The lesion was in the first Changes in the observed morphology at each time point relative to a reference time point This indicates the expected change in form under counterfactual conditions. Indicating uncertainty characteristics, This indicates the embedding of treatment and acquisition contexts. This indicates a vector concatenation operation.

[0016] A tumor image tracking system based on lesion identity field and counterfactual spatiotemporal calibration includes: The data acquisition module is used to acquire medical image sequences and associated clinical context data of the same subject at multiple time points; The spatiotemporal alignment module is used to perform cross-time point standardization, anatomical region constraint registration, and modality consistency processing on the medical image sequence to obtain spatiotemporally aligned image data. The lesion identity field generation module is used to input the spatiotemporally aligned image data into the lesion identity field generation network to obtain the identity embedding, spatial occupancy distribution and morphological characterization of each candidate tumor lesion at different time points. The longitudinal memory map update module is used to construct a longitudinal lesion memory map based on the identity embedding, the spatial occupancy distribution and the morphological representation, and update the inter-period correspondence between each lesion node through reversible deformation consistency constraints. The counterfact calibration module is used to input the longitudinal lesion memory map into the counterfact spatiotemporal calibration module to obtain the calibrated tumor tracking results.

[0017] An electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the tumor image tracking method as described in any of the preceding claims.

[0018] A computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the tumor image tracking method as described in any of the preceding claims.

[0019] This application, through the synergistic effect of lesion identity field, longitudinal lesion memory map, and counterfactual spatiotemporal calibration mechanism, has the following beneficial effects: First, by using the lesion identity field mechanism, we can learn the lesion identity embedding that can be maintained across time points during the candidate lesion generation stage, instead of performing post-processing matching based on spatial distance after detection at a single time point. This can reduce the risk of identity mismatch in cases of dense distribution of multiple lesions, morphological changes after treatment, and blurred lesion boundaries.

[0020] Second, by using the longitudinal lesion memory map mechanism, identity embedding, spatial occupancy, morphological representation, deformation field and uncertainty are incorporated into a unified graph structure for updating. This allows each lesion node to contain not only the image information at the current time point, but also historical trajectory information and future candidate association information, which is conducive to improving the continuity of lesion tracking.

[0021] Third, by using a counterfactual spatiotemporal calibration mechanism, changes in scanning protocols, registration errors, treatment-related alterations, and actual tumor evolution can be separated from observed changes. The output of calibrated lesion changes and decomposition results of causes of change helps to enhance the interpretability and verifiability of tracking results.

[0022] Fourth, by incorporating registration reliability into the lesion matching score through reversible deformation consistency constraints, it helps to reduce the situation of blindly judging lesion inheritance in unreliable registration areas and reduces the risk of erroneous tracking caused by organ deformation, respiratory motion and image artifacts.

[0023] Fifth, this application calculates the inter-period matching score by jointly using identity embedding similarity, spatial distance after deformation, morphological differences and uncertainty, rather than relying on a single feature for matching judgment, which is conducive to improving the accuracy of inter-period lesion correspondence.

[0024] Sixth, the tumor tracking results output by this application include lesion identification number, cross-stage correspondence, lesion spatial occupancy, calibration volume change, calibration long diameter change, progression score, uncertainty and decomposition information of the causes of change, which can provide structured auxiliary reference information for the generation of medical imaging follow-up reports, the organization of clinical research data and multidisciplinary discussions. Attached Figure Description

[0025] Figure 1 A flowchart illustrating a tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration provided for embodiments of this application; Figure 2 A schematic diagram of the structure of the lesion identity field generation network provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of a longitudinal lesion memory map provided in an embodiment of this application; Figure 4 A schematic diagram of the processing flow of the counterfactual spatiotemporal calibration module provided in the embodiments of this application; Figure 5 This is a structural block diagram of the tumor image tracking system provided in the embodiments of this application; Figure 6 A structural block diagram of an electronic device provided in an embodiment of this application; Figure 7 This is a schematic diagram illustrating the intermediate processes of follow-up data, feature extraction and inter-period matching, and counterfactual spatiotemporal calibration for example cases in this application embodiment. Detailed Implementation

[0026] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. These embodiments are only used to explain the present invention and are not intended to limit the scope of protection of the present invention.

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of this application, and not all embodiments. Other implementation methods obtained by those skilled in the art based on the embodiments of this application without creative effort should all fall within the protection scope of this application.

[0028] In the description of this application, the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. The term "lesion identity" refers to the continuous correspondence of the same tumor lesion in images at multiple time points, and is not equivalent to pathological diagnostic identity. The term "counterfactual spatiotemporal calibration" refers to constructing a baseline of expected changes under specific conditions within the model and comparing observed changes with this baseline to reduce the influence of non-tumor biological factors on tracking results.

[0029] I. Overall Method and Process Please see Figure 1 This application provides a tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration, the method comprising the following steps: S100: Acquire medical image sequences and associated clinical context data of the same subject collected at multiple time points.

[0030] Among them, medical imaging sequences can be represented as: in, This represents a medical image sequence representing the same patient. Indicates the first Medical images collected at specific time points, This indicates the total number of follow-up time points. This indicates the time point number. If multimodal images exist at the same time point, then... It can be further expressed as ,in, Indicates the number of modes. Indicates the first The first time point Medical imaging modalities.

[0031] Associated clinical context data can be represented as: in, This represents the set of context data corresponding to a medical image sequence. Indicates the first The contextual features corresponding to each time point may include information such as scan slice thickness, reconstruction nucleus, enhancement phase, examination equipment, examination site, treatment time, treatment type, treatment cycle, and patient position.

[0032] S200: Performs cross-time point standardization, anatomical region constraint registration, and modality consistency processing on medical image sequences to obtain spatiotemporally aligned image data.

[0033] Specifically, the system first resamples images from different time points to ensure uniform voxel spacing across all images; then, it performs grayscale or signal intensity standardization based on the image modality. For example, for CT images, grayscale truncation can be performed according to preset window widths and levels, and the truncated HU values ​​can be normalized to a preset range; for PET images, SUV values ​​can be normalized to a uniform scale; and for MRI images, intensity standardization can be performed in conjunction with a reference tissue region.

[0034] After image standardization, the system generates an anatomical region constraint mask based on the target organ segmentation results. For lung lesions, the anatomical region constraint mask can be a lobe, segment, or the entire lung; for liver lesions, it can be the liver region; and for brain lesions, it can be the intracranial soft tissue region. This constraint mask is used to reduce the probability of the registration process being affected by non-target regions.

[0035] The deformation field across time points can be expressed as: in, Indicates by the first Image coordinates at time point 1 mapped to the 1st time point Deformation field of image coordinates at each time point The parameter is The registration model, Indicates the image at the reference time point. Indicates the image at the target time point. A constraint mask representing the anatomical region at a reference time point. The anatomical region constraint mask represents the target time point.

[0036] S300: Input spatiotemporally aligned image data into the lesion identity field generation network to obtain the identity embedding, spatial occupancy distribution and morphological characterization of each candidate tumor lesion at different time points.

[0037] Please see Figure 2 The lesion identity field generation network includes a shared image encoder, a local lesion decoder, an identity embedding head, a spatial occupancy head, and a morphological representation head. Its output can be represented as: in, Indicates the first Spatiotemporally aligned image data at each time point Indicates the first Anatomical region constraint mask at each time point, Indicates the first Contextual features at each time point The parameter is The lesion identity field generation network, Indicates the first The first time point Embedding of the identity of each candidate lesion Indicates the first The first time point Spatial distribution of candidate lesions Indicates the first The first time point Morphological characterization of candidate lesions.

[0038] Identity Embedding Used for cross-time point comparisons, its training objective is not to distinguish between benign and malignant lesions or pathological types, but rather to ensure that the same lesion has similar embeddings at different time points, and that different lesions have distinguishable embeddings. Spatial occupancy distribution. It can be a three-dimensional probability map, used to represent the probability of the existence of candidate lesions in voxel space. Morphological representation. Features can include volume, major axis, minor axis, surface area, sphericity, boundary irregularity, texture intensity, enhancement degree, and metabolic uptake.

[0039] S400: A longitudinal lesion memory map is constructed based on identity embedding, spatial occupancy distribution and morphological representation, and the inter-period correspondence between each lesion node is updated through reversible deformation consistency constraints.

[0040] Please see Figure 3 The longitudinal lesion memory map can be represented as: in, Represents a longitudinal lesion memory map. Represents the set of lesion nodes. This represents the set of candidate edges across different time periods. Each lesion node... Corresponding to the The first time point Each candidate lesion. Each interphase candidate edge Indicates the first The first time point The candidate lesion and the first The first time point The candidate lesions may belong to the same longitudinal lesion identity.

[0041] The intertemporal matching score can be expressed as: in, This represents the interphase matching score between two candidate lesions; This indicates the similarity weight of the identity embedding; Indicates spatial consistency weight; Indicates the weight of morphological consistency; Indicates the penalty weight for uncertainty; Cosine similarity can be used to represent the similarity between two identity embeddings. Indicates the first The first time point The spatial center of each candidate lesion is mapped to the first... Coordinates after each point in time; Indicates the first The first time point The spatial center of each candidate lesion; Represents a spatial distance function; The temperature parameter represents the rate at which spatial distance decays. The 1-norm difference in the morphological characteristics of two candidate lesions; Temperature parameter indicating the rate of decay of morphological differences; This indicates the uncertainty of cross-period matching between two candidate lesions.

[0042] The uncertainty It can be determined jointly by registration uncertainty, segmentation uncertainty, identity embedding uncertainty, and image quality uncertainty. For example: in, This indicates registration uncertainty. This indicates uncertainty in the segmentation or occupancy distribution. This indicates uncertainty in identity embedding. This indicates uncertainty in image quality. , , , These represent the weighting coefficients of the corresponding uncertainty components.

[0043] Reversible deformation consistency constraints are used to limit unexplainable folding or excessive deformation caused by non-rigid registration. Their loss function is: in, This represents the consistency loss of reversible deformation; Represents the image spatial coordinate domain; Represents the position of any voxel in the coordinate domain; Indicates the number of voxels in the coordinate domain; Indicates by the first The time point is mapped to the first Deformation field at each time point; Indicates by the first The time point is mapped to the first The reverse deformation field at each time point; Indicates first position By the The time point is mapped to the first At the first point in time, then from the second The time point is mapped back to the first Coordinates after each point in time; Represents the square of the L2 norm; Indicates the deformation field at position Spatial gradient at a given location; This represents the smoothing constraint coefficient.

[0044] S500: Input the longitudinal lesion memory map into the counterfactual spatiotemporal calibration module to separate the apparent changes caused by changes in scanning protocol, registration error, treatment-related image changes and real tumor evolution, and obtain calibrated tumor tracking results.

[0045] Please see Figure 4 The counterfactual spatiotemporal calibration module is used to decompose the causes of changes observed in lesions. For the first... The identity of the longitudinal lesion, in the first The observed morphological changes at each time point can be represented as: in, Indicates the first The identity of the longitudinal lesion in the first Each time point relative to the reference time point Observational morphological changes, Indicates the first The identity of the longitudinal lesion in the first Morphological representation at each point in time. Indicates the identity of the same longitudinal lesion at the reference time point Morphological representation.

[0046] Counterfactual changes can be represented as: in, It indicates the expected change in form under counterfactual conditions; The parameter is The counterfactual spatiotemporal calibration model; This indicates the morphological characteristics of the lesion at a reference time point; This indicates the reference time point for data collection and clinical context features; This indicates the data collected at the target time point and the clinical context features. This represents the contextual embedding related to treatment; This indicates the uncertainty in the process of lesion tracking.

[0047] The calibrated residual can be expressed as: in, Indicates the first The identity of the longitudinal lesion in the first The calibration change residuals at each time point. These residuals represent the remaining components of change after deducting variations in scanning protocol, registration errors, and treatment-related apparent changes. The calibration progress score can be expressed as: in, Indicates the first The identity of the longitudinal lesion in the first Calibration progress score at each time point; Represents the Sigmoid function; Represents a learnable weight vector; express Transpose of; Indicates the calibration variation residual; Indicates uncertainty; This indicates the embedding of treatment and acquisition contexts; Indicates the bias term; This indicates a vector concatenation operation.

[0048] It should be noted that the calibration progress score is used to output auxiliary structured results at the image tracking level, and does not directly replace the physician's diagnosis, nor is it used as the sole basis for treatment decisions.

[0049] II. Training Methods In some implementations, the lesion identity field generation network, the longitudinal lesion memory map update module, and the counterfactual spatiotemporal calibration module can be trained jointly or in stages.

[0050] In the phased training approach, the first phase trains the lesion identity field generation network, enabling it to stably output the spatial occupancy distribution and identity embedding of candidate lesions. This phase can use manually labeled lesion segmentation masks, lesion center points, lesion numbers, or follow-up correspondences as supervisory information.

[0051] Loss of identity retention can be expressed as: in, This indicates a loss of identity retention; and This represents two candidate lesion samples; A marker indicating whether two lesions belong to the same longitudinal lesion identity; when two lesions belong to the same longitudinal lesion identity... ,otherwise ; This indicates the probability that the model predicts that the two belong to the same longitudinal lesion identity.

[0052] The second stage trains the registration model and the longitudinal lesion memory map update module, enabling them to form stable lesion trajectories across different time points. The loss function in this stage may include matching loss, reversible deformation consistency loss, and graph structure continuity loss. in, This indicates the total training loss. This indicates intertemporal mismatch of lesions; This represents the consistency loss of reversible deformation; This indicates a loss of structural continuity in the longitudinal lesion memory map; , , These represent the weighting coefficients of the three loss terms.

[0053] The third stage trains the counterfactual spatiotemporal calibration module, enabling it to learn the effects of scan protocol variations, treatment context, and registration uncertainties on lesion appearance. This stage can be trained using samples with stable lesion annotations, treatment records, and multi-protocol images, or using protocol-enhanced samples generated from the same image under different reconstruction parameters, slice thicknesses, or intensity perturbations.

[0054] The comprehensive training objective can be expressed as: in, This represents the overall training loss of the model; This indicates the space occupied or segmented by the lesion; This indicates a loss of identity retention; This indicates the loss in lesion tracking; Indicates counterfactual calibration loss; Indicates the calibration loss due to uncertainty; to These represent the weighting coefficients of the corresponding loss terms.

[0055] III. Output Results The calibrated tumor tracking results output in this application embodiment may include: A unique identifier for each longitudinal lesion; Spatial distribution of lesions at each time point; The lesion's long diameter, short diameter, volume, boundary characteristics, enhancement characteristics, or metabolic uptake characteristics; Correspondence between lesions at different time points; Calibrated volume change rate and major diameter change rate; Calibration progress score; Match the results of uncertainty and cause-of-change decomposition; Visual overlays or structured report fields corresponding to the tracking results.

[0056] The calibrated volume change rate can be expressed as: in, Indicates the first The identity of the longitudinal lesion in the first Each time point relative to the reference time point The rate of change of the calibration volume; Indicates the first The lesion was in the first Calibration volume at each time point; This indicates the same lesion at the reference time point. The calibration volume; This represents a very small positive number used to prevent the denominator from being zero.

[0057] IV. System Implementation Examples Please see Figure 5 This application also provides a tumor image tracking system based on lesion identity field and counterfactual spatiotemporal calibration, including a data acquisition module 100, a spatiotemporal alignment module 200, a lesion identity field generation module 300, a longitudinal memory map update module 400, and a counterfactual calibration module 500.

[0058] The data acquisition module 100 is used to acquire medical image sequences and associated clinical context data of the same subject collected at multiple time points.

[0059] The spatiotemporal alignment module 200 is used to perform cross-time point standardization, anatomical region constraint registration and modality consistency processing on medical image sequences to obtain spatiotemporally aligned image data.

[0060] The lesion identity field generation module 300 is used to input spatiotemporally aligned image data into the lesion identity field generation network to obtain the identity embedding, spatial occupancy distribution and morphological characterization of each candidate tumor lesion at different time points.

[0061] The longitudinal memory map update module 400 is used to construct a longitudinal lesion memory map based on identity embedding, spatial occupancy distribution and morphological representation, and to update the inter-period correspondence between each lesion node through reversible deformation consistency constraints.

[0062] The counterfact calibration module 500 is used to input the longitudinal lesion memory map into the counterfact spatiotemporal calibration module to separate the apparent changes caused by changes in scanning protocol, registration error, treatment-related image changes and real tumor evolution, and obtain calibrated tumor tracking results.

[0063] It should be noted that the above system embodiments correspond to the method embodiments, and their specific implementation processes and technical effects can be found in the descriptions in the method embodiments. To avoid repetition, they will not be repeated here.

[0064] V. Examples of Electronic Equipment Please see Figure 6This application also provides an electronic device. The electronic device may include a processor, a memory, a communication interface, and a communication bus. The communication bus is used to realize data transmission between the processor, the memory, and the communication interface. The memory is used to store a computer program, and the processor is used to execute the computer program to implement the tumor image tracking method described in any embodiment of this application.

[0065] The processor can be a central processing unit, graphics processing unit, neural network processor, digital signal processor, application-specific integrated circuit, field-programmable gate array, or other programmable logic device. The memory can include random access memory, read-only memory, flash memory, disk storage, or optical disk storage. The communication interface can be used to exchange data with medical image storage and transmission systems, hospital information systems, radiology information systems, or external servers.

[0066] This application also provides a computer-readable storage medium storing a computer program or computer instructions, which, when executed by a processor, implements the tumor image tracking method described in any embodiment of this application.

[0067] VI. Application Examples Please see Figure 7 In one exemplary application, the subject underwent chest CT scans at three follow-up time points: time point (t1) was March 12, 2024; time point (t2) was May 18, 2024; and time point (t3) was July 20, 2024. The system acquires the CT images from these three time points, along with the corresponding acquisition protocols, treatment records, and image quality information, and inputs this data as example case follow-up data into the tumor image tracking system provided in this application embodiment.

[0068] At time point (t1), the system identified lesion L1, with center coordinates ((112,86,54)), a major diameter of 18.6 mm, a volume of 2.31 cm³, and an average CT value of 42 HU. At time point (t2), due to the presence of adjacent candidate lesions in the image, the system simultaneously obtained two candidate objects: L1 candidate A and L2 candidate B. Among them, L1 candidate A has center coordinates ((114,88,55)), a major diameter of 21.4 mm, a volume of 2.96 cm³, and an average CT value of 45 HU; L2 candidate B has a major diameter of 13.2 mm and a volume of 1.08 cm³. At time point (t3), the system again identified lesion L1, with center coordinates ((116,89,55)), a major diameter of 20.8 mm, a volume of 2.84 cm³, and an average CT value of 44 HU.

[0069] Based on the aforementioned follow-up data, the lesion identity field generation network extracted the identity embedding, spatial mapping distance, and morphological differences of each candidate lesion, and calculated the inter-period matching score. Specifically, the identity embedding similarity between L1 at time point (t1) and candidate A at time point (t2) was 0.91, the spatial mapping distance was 2.3 mm, the morphological difference was 0.18, and the matching score (S) was 0.93; the identity embedding similarity between L1 at time point (t1) and candidate B at time point (t2) was 0.54, the spatial mapping distance was 11.8 mm, the morphological difference was 0.61, and the matching score (S) was 0.41; the identity embedding similarity between candidate A at time point (t2) and L1 at time point (t3) was 0.94, the spatial mapping distance was 1.9 mm, the morphological difference was 0.12, and the matching score (S) was 0.95.

[0070] Based on the matching results, the longitudinal lesion memory map identifies L1 at time point (t1), candidate A at time point (t2), and L1 at time point (t3) as the same longitudinal lesion trajectory and assigns them the longitudinal lesion identity K01. Correspondingly, the inheritance relationship formed by the system is: (t1(L1)→t2(candidate A)→t3(L1)). Since the matching score between L1 at time point (t1) and candidate B at time point (t2) is significantly lower than that of candidate A, candidate B is not included in the longitudinal lesion identity K01, thus avoiding lesion identity exchange in the presence of adjacent lesions.

[0071] Furthermore, the counterfactual spatiotemporal calibration module calibrates the volume change between time point (t1) and time point (t2). The system first calculates the observed volume change ΔVOBS as +0.65 cm³; subsequently, it makes corrections based on the acquisition protocol context, registration reliability, and changes in treatment-related images. Specifically, the protocol difference correction was -0.08 cm³, the registration error correction was -0.03 cm³, and the treatment-related image change correction was -0.05 cm³. After the above counterfactual spatiotemporal calibration, the system obtained a calibrated volume change ΔVcal of +0.49 cm³.

[0072] Based on the calibrated volume change, major diameter change, identity matching results, and uncertainty features, the system outputs the calibration progress score GK01,t2 of the longitudinal lesion identity K01 at time point t2, which is 0.78. The major diameter change rate after calibration is +15.1%, and the matching uncertainty is 0.09. Therefore, the system generates the following tracking conclusion: the same lesion persists and shows an increasing trend. This example demonstrates that the embodiments of this application can maintain the continuity of lesion identity based on identity embedding, spatial mapping distance, morphological differences, and counterfactual calibration results, and output structured, verifiable tumor image tracking results, even in the presence of adjacent candidate lesions and differences in image acquisition.

[0073] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration, characterized in that, Includes the following steps: Acquire medical image sequences and associated clinical context data of the same subject collected at multiple time points; perform cross-time point standardization, anatomical region constraint registration and modality consistency processing on the medical image sequences to obtain spatiotemporally aligned image data; The spatiotemporally aligned image data is input into the lesion identity field generation network to obtain the identity embedding, spatial occupancy distribution and morphological characterization of each candidate tumor lesion at different time points. A longitudinal lesion memory map is constructed based on the identity embedding, the spatial occupancy distribution, and the morphological representation, and the inter-period correspondence between each lesion node is updated through reversible deformation consistency constraints. The longitudinal lesion memory map is input into the counterfactual spatiotemporal calibration module to separate the apparent changes caused by changes in scanning protocol, registration error, treatment-related image changes and actual tumor evolution, and obtain the calibrated tumor tracking results.

2. The tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration according to claim 1, characterized in that, The medical imaging sequence includes one or more of CT images, MRI images, PET images, ultrasound images, and digital images of pathological sections; the associated clinical context data includes one or more of the following: acquisition time, scan slice thickness, reconstructed nucleus, enhancement phase, patient position, treatment time, treatment category, marked lesion area, and organ segmentation results.

3. The tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration according to claim 1, characterized in that, The process of performing cross-time point standardization, anatomical region constraint registration, and modality consistency on medical image sequences to obtain spatiotemporally aligned image data includes: Voxel spacing resampling, grayscale or signal intensity normalization, and noise suppression processing were performed on medical images at different time points. Generate an anatomical constraint mask based on the segmentation results of the target organ or anatomical region; Rigid registration, affine registration, and non-rigid registration are performed within the anatomical constraint mask to obtain the deformation field across time points; Based on the cross-time point deformation field, medical images from multiple time points are mapped to a unified reference coordinate system to obtain the spatiotemporally aligned image data.

4. The tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration according to claim 1, characterized in that, The lesion identity field generation network includes a shared image encoder, a local lesion decoder, and an identity embedding head; the output of the lesion identity field generation network satisfies: in, Indicates the first Spatiotemporally aligned image data at each time point Indicates the first Anatomical constraint mask corresponding to each time point Indicates the first Data collection and clinical context features corresponding to each time point The parameter is The lesion identity field generation network, Indicates the first The first time point Embedding of the identity of each candidate lesion Indicates the first The first time point Spatial distribution of candidate lesions Indicates the first The first time point Morphological characterization of candidate lesions.

5. The tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration according to claim 4, characterized in that, The identity embedding is used to characterize the consistency of the identity of the same lesion at different time points; the spatial occupancy distribution is used to characterize the probabilistic occupancy area of ​​the candidate lesion in three-dimensional space; the morphological characterization includes one or more of the following: lesion volume, major axis, minor axis, boundary irregularity, image texture features, enhancement features, and metabolic uptake features.

6. The tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration according to claim 1, characterized in that, The longitudinal lesion memory map includes lesion nodes and inter-stage candidate edges. Each lesion node corresponds to a candidate lesion at a given time point, and each inter-stage candidate edge represents a candidate relationship where two candidate lesions at different time points belong to the same tumor lesion. The matching score of the inter-stage candidate edge satisfies: in, Indicates the first The first time point The candidate lesion and the first The first time point Interphase matching scores between candidate lesions , , , This represents the preset or learnable weight coefficients. This represents an identity embedding similarity function. and These represent the identity embeddings of the two candidate lesions, respectively. Indicates by the first The time point is mapped to the first Deformation field at each time point and These represent the spatial centers of the two candidate lesions. Represents the spatial distance function. Indicates spatial distance and temperature parameters. and These represent the morphological characteristics of the two candidate lesions. Represents the norm distance. Temperature parameters representing morphological differences This indicates uncertainty in intertemporal matching.

7. The tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration according to claim 1, characterized in that, The reversible deformation consistency constraint satisfies: in, This represents the consistency loss of reversible deformation. This represents the spatial coordinate domain of the image to be registered. Represents the position of any voxel in the coordinate domain. This indicates the number of voxels within the coordinate domain. Indicates by the first The time point is mapped to the first Deformation field at each time point Indicates by the first The time point is mapped to the first The reverse deformation field at each time point Represents the square of the L2 norm. Indicates the deformation field at position Spatial gradient at that location This represents the smoothing constraint coefficient.

8. The tumor image tracking method based on lesion identity field and counterfactual spatiotemporal calibration according to claim 1, characterized in that, The counterfactual spatiotemporal calibration module is used to generate a counterfactual morphological representation of the same lesion under the assumptions of consistent scanning protocols, limited registration errors, and suppressed treatment-related imaging changes, and to obtain a calibration progress score based on the difference between the observed morphological representation and the counterfactual morphological representation; the calibration progress score satisfies: in, Indicates the first The identity of the longitudinal lesion in the first Calibration progress score at each time point This represents the Sigmoid function. Represents a learnable weight vector. Indicates the bias term. Indicates the first The lesion was in the first Changes in the observed morphology at each time point relative to a reference time point This indicates the expected change in form under counterfactual conditions. Indicating uncertainty characteristics, This indicates the embedding of treatment and acquisition contexts. This indicates a vector concatenation operation.

9. A tumor image tracking system based on lesion identity field and counterfactual spatiotemporal calibration, characterized in that, include: The data acquisition module is used to acquire medical image sequences and associated clinical context data of the same subject at multiple time points; The spatiotemporal alignment module is used to perform cross-time point standardization, anatomical region constraint registration, and modality consistency processing on the medical image sequence to obtain spatiotemporally aligned image data. The lesion identity field generation module is used to input the spatiotemporally aligned image data into the lesion identity field generation network to obtain the identity embedding, spatial occupancy distribution and morphological characterization of each candidate tumor lesion at different time points. The longitudinal memory map update module is used to construct a longitudinal lesion memory map based on the identity embedding, the spatial occupancy distribution and the morphological representation, and update the inter-period correspondence between each lesion node through reversible deformation consistency constraints. The counterfact calibration module is used to input the longitudinal lesion memory map into the counterfact spatiotemporal calibration module to obtain the calibrated tumor tracking results.

10. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the tumor image tracking method as described in any one of claims 1 to 8.

Images