Longitudinal chest radiograph progression monitoring method and system based on anatomical anchor semantic difference

Abstract

The application discloses a longitudinal chest radiograph progress monitoring method and system based on anatomical anchor semantic difference, relates to the field of medical image processing, and generates a shared anatomical support mask by performing spatial union on candidate boxes of front and rear chest radiographs in a unified coordinate system; the mask and the original image are soft fused through a learnable gate to obtain an image enhanced in anatomical anchoring; longitudinal difference processing is performed on the front and rear enhanced images in an image-report semantic embedding space to obtain report-aware semantic difference, a visual encoder is used to extract multi-scale visual features to calculate difference features, the difference features are refined in a nonlinear manner, the refined difference information is asymmetrically injected into current visual features, and finally, multi-scale fusion, semantic recalibration and progress classification are completed. The application can stably, accurately and explainably monitor the disease performance changes in the front and rear chest images of the same patient without relying on the input of radiology reports in the reasoning stage.

Description

Technical Field

[0001] This application belongs to the field of medical image processing technology, and in particular relates to a method and system for monitoring the progression of longitudinal chest radiographs based on anatomical anchoring semantic difference. Background Technology

[0002] Currently, intelligent analysis methods for chest X-ray images have been widely applied in disease classification, abnormal region detection, anatomical structure localization, image report generation, and disease progression analysis. With the development of deep learning technology, convolutional neural networks, graph neural networks, Transformer models, and object detection models have been used to extract high-dimensional visual features from chest X-ray images. For longitudinal chest X-ray progression monitoring, existing methods have attempted to model the relationship between changes in two consecutive chest X-rays using anatomical perception map structures, hierarchical visual Transformers, Siamese network structures, feature difference modules, or object detection-based local representation learning.

[0003] Current methods for monitoring chest progression using longitudinal chest radiographs still have the following limitations. For example, many methods directly compare the entire image or global visual features. However, two consecutive chest X-ray images are often affected by differences in projection angle, patient position, inspiratory intensity, field of view coverage, and equipment parameters. In the absence of stable anatomical constraints, models are prone to misinterpreting superficial changes caused by non-pathological factors as pathological changes, thereby reducing the stability and accuracy of progression assessment.

[0004] For example, many methods use only simple feature subtraction, ordinary multilayer perceptron or linear difference modeling, which are insufficient to fully characterize the nonlinear, asymmetric and fine-grained changes in disease progression. This can easily lead to confusion between the three states of aggravation, improvement and no significant change, and is especially prone to increasing false alarms in cases with no significant change.

[0005] For example, existing visual models mostly rely on the images themselves to learn differential features, failing to fully utilize the semantic knowledge contained in the existing report texts during the training phase to constrain the representation of visual changes. If radiology reports are directly input, the system will be limited by factors such as reports not yet being generated, missing reports, delays in report generation during the inference phase, differences in writing habits among different institutions, and inconsistent text quality, which is not conducive to deployment in actual clinical workflows. Summary of the Invention

[0006] This application provides a method and system for monitoring the progression of disease on longitudinal chest X-rays based on anatomical anchoring semantic difference. It can stably, accurately and interpretably monitor the progression of disease in the same patient's chest X-ray images without relying on radiological report input during the inference stage, by combining anatomical region knowledge, report semantic knowledge during the training stage and nonlinear visual difference modeling capabilities.

[0007] Firstly, this application provides a longitudinal chest radiograph progression monitoring method based on anatomically anchored semantic difference, including: Anatomical candidate regions from historical chest X-ray images and current chest X-ray images are extracted and spatially joined in a unified image coordinate system to obtain a shared anatomical support mask; By using learnable gating, the shared anatomical support mask is softly fused with historical chest X-ray images and current chest X-ray images respectively to obtain historical enhanced chest X-ray images and current enhanced chest X-ray images. The historical enhanced chest X-ray image and the current enhanced chest X-ray image are subjected to longitudinal difference processing in the image-report semantic embedding space to obtain the report-aware semantic difference; The historical enhanced chest X-ray image and the current enhanced chest X-ray image are input into the visual encoder to obtain multi-scale historical visual features and multi-scale current visual features. The historical visual features and current visual features corresponding to each scale are then subjected to difference operations to obtain the original visual differences for each scale. The original visual differences at each scale are processed by a nonlinear differential projector to generate refined differential representations at each scale. Then, based on the historical visual features at each scale, gate parameters for the corresponding scale are generated. The refined differential representations at each scale are asymmetrically injected into the current visual features at the corresponding scale through the gate parameters to obtain the updated current visual features at each scale. Multi-scale differential fusion is performed on historical visual features at various scales, updated current visual features, and refined differential representations to obtain a comprehensive visual change map. After semantic recalibration of the comprehensive visual change map using report-aware semantic differential, the progression status of chest diseases is output through global pooling and classification head.

[0008] Secondly, this application provides a longitudinal chest radiograph progression monitoring system based on anatomical anchoring semantic difference, comprising: The mask construction module is used to extract anatomical candidate regions from historical chest X-ray images and current chest X-ray images, respectively, and perform spatial union in a unified image coordinate system to obtain a shared anatomical support mask; The mask enhancement module is used to softly fuse a shared anatomical support mask with historical chest X-ray images and current chest X-ray images respectively through learnable gating to obtain historical enhanced chest X-ray images and current enhanced chest X-ray images; The semantic difference module is used to perform longitudinal difference processing in the image-report semantic embedding space on historical enhanced chest X-ray images and current enhanced chest X-ray images to obtain report-aware semantic difference. The visual difference module is used to input historical enhanced chest X-ray images and current enhanced chest X-ray images into the visual encoder to obtain multi-scale historical visual features and multi-scale current visual features, and to perform difference operations on the historical visual features and current visual features corresponding to each scale to obtain the original visual difference of each scale. The visual feature update module is used to generate refined difference representations of each scale from the original visual differences at each scale through a nonlinear difference projector, and then generate corresponding scale gating parameters based on the historical visual features at each scale. The refined difference representations of each scale are asymmetrically injected into the current visual features of the corresponding scale through the gating parameters to obtain the updated current visual features at each scale. The results output module is used to perform multi-scale difference fusion of historical visual features, updated current visual features, and refined difference representations at various scales to obtain a comprehensive visual change map. After semantic recalibration of the comprehensive visual change map using report-aware semantic difference, the progress status of chest diseases is output through global pooling and classification head.

[0009] The longitudinal chest radiograph progression monitoring method and system based on anatomical anchoring semantic difference provided in this application have at least the following beneficial effects: Anatomical candidate boxes are extracted from anterior and posterior chest X-rays using an anatomical region detector with the same set of parameters. These candidate boxes are then spatially united in a unified coordinate system to generate a shared anatomical support mask. A learnable gating method is then used to softly fuse the mask with the original image, resulting in an anatomically anchored enhanced image. This ensures that all subsequent differential modeling is performed within the shared anatomical support. Next, the historical enhanced chest X-ray image and the current enhanced chest X-ray image undergo longitudinal differential processing in the image-report semantic embedding space to obtain a report-aware semantic difference. This allows for the extraction of features with clinical semantic priors without requiring a report input, avoiding the deployment obstacles of report dependence. The semantic difference directly represents the differences between the anterior and posterior images in the clinical semantic space. A visual encoder is used to extract multi-scale visual features. After calculating the original visual difference, a nonlinear... The differential projector performs nonlinear refinement on the differential features; then, based on historical visual features, it generates gating parameters and asymmetrically injects the refined differential information into the current visual features (historical features remain unchanged, only the current features are updated). The nonlinear differential projector can learn independent nonlinear response curves for different feature dimensions, accurately depicting fine-grained and asymmetric lesion changes. The asymmetric injection method aligns with the clinical logic of "using history as a benchmark to judge current changes." The gating mechanism can adaptively modulate the differential weights of different regions. Finally, after multi-scale fusion and semantic recalibration, the progression classification is completed. The recalibration can guide visual features to focus on changes related to clinical progression, suppress irrelevant grayscale and background differences, improve the semantic consistency of classification, and ultimately reduce the probability of confusion between the three progression states.

[0010] In summary, this application can stably, accurately, and interpretably monitor the progression of disease manifestations in the anterior and posterior chest X-ray images of the same patient without relying on radiological report input during the inference phase, by combining anatomical region knowledge, report semantic knowledge during the training phase, and nonlinear visual difference modeling capabilities. Attached Figure Description

[0011] Figure 1 This is the overall architecture diagram of the longitudinal chest radiograph progression monitoring method and system based on anatomical anchoring semantic difference in this application; Figure 2 This is a flowchart illustrating the longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference provided in this application embodiment; Figure 3 This is a schematic diagram of the anatomical anchoring shared support mechanism provided in the embodiments of this application; Figure 4 This is a schematic diagram of the report-aware semantic differential extraction mechanism provided in the embodiments of this application; Figure 5 This is a schematic diagram of the nonlinear visual difference modeling and asymmetric difference injection mechanism provided in the embodiments of this application; Figure 6 This is a schematic diagram of the multi-scale differential fusion, semantic recalibration, and progress classification output provided in the embodiments of this application. Detailed Implementation

[0012] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0013] Chest X-ray progression monitoring is a task focused on identifying changes in longitudinal image pairs. The core of this task is to identify clinically significant pathological changes from before and after examination images, while minimizing the appearance changes caused by non-pathological factors such as shooting angle, patient position, degree of inspiration, field of view coverage, and differences in equipment parameters.

[0014] Patent document CN111401398B discloses a system and method for determining disease progression based on artificial intelligence detection output. It processes patient image data from first and second time points to obtain corresponding severity classification results and determines disease progression by comparing these classification results. While this method can use artificial intelligence detection output to judge changes in the condition, it primarily relies on comparing two detection results or severity classification results. This method does not construct a unified anatomical comparison support region for pre- and post-chest X-ray images, nor does it utilize report semantic knowledge from the training phase to form a semantic differential representation that does not require report input during the inference phase. Therefore, when there are differences in patient position, projection angle, or visual field coverage, this method struggles to exclude apparent changes caused by non-pathological factors, easily affecting the stability of progression status judgment. Furthermore, due to the lack of report semantic constraints, its ability to fine-grainedly distinguish between the three states of aggravation, improvement, and no significant change remains limited.

[0015] Patent document EP4319640B1 discloses a personalized intensive care imaging scheme that focuses on the consistency of image acquisition using mobile X-ray equipment in the intensive care environment, pointing out that consistent and accurate image acquisition and processing are crucial for subsequent analysis. This type of scheme can reduce acquisition differences between chest X-ray images from the perspective of imaging equipment and acquisition procedures, but its emphasis is on improving or standardizing image acquisition conditions. This scheme does not use longitudinal chest X-ray image pairs as core input, nor does it output the aggravation, improvement, or no significant change in disease manifestations through shared anatomical support, report-aware semantic difference, and nonlinear visual difference modeling. Therefore, this type of scheme cannot directly solve the problem of longitudinal progression monitoring under existing chest X-ray data conditions, nor can it perform model-level anatomical constraints and semantic recalibration for existing before-and-after image differences.

[0016] Patent document WO2021248187A1 discloses a system and method for automatically analyzing medical images. This system inputs anatomical images into a neural network, which outputs feature vectors and calculates the presence of multiple visual findings. This approach is primarily used for detecting multiple visual findings in a single medical image, assisting in identifying specific abnormalities. However, this approach does not use pre- and post-operative chest X-ray images of the same patient as input, nor does it model the relationship between changes before and after examinations. If two independent detection results are directly compared, the system is susceptible to spatial inconsistencies between pre- and post-operative images, differences in acquisition conditions, and localized changes due to the lack of shared anatomical support and nonlinear differential modeling. This makes it difficult to reliably identify minor changes in lesion extent, density, or no significant changes.

[0017] Therefore, existing methods for monitoring chest progression via longitudinal chest radiographs still have the following shortcomings: Regarding spatial consistency, many methods directly compare the entire image or global visual features. However, two consecutive chest X-ray images are often affected by differences in projection angle, patient position, inspiratory intensity, visual field coverage, and equipment parameters. Without stable anatomical region constraints, models can easily misinterpret superficial changes caused by non-pathological factors as pathological changes, thus reducing the stability and accuracy of progression assessment. While some methods introduce local region detection or anatomical region feature extraction, they typically only extract local features from the current and previous images separately before subsequent comparison. These methods fail to establish a shared anatomical support region between consecutive images. Therefore, even with slight anatomical shifts, lesion enlargement or shrinkage, or differences in image coverage, the model may still be affected by spatial inconsistencies, thus impacting the differentiation between worsening, improving, and unchanged states.

[0018] In differential modeling, disease progression on chest X-rays does not always manifest as simple pixel differences or linear feature changes. Some lesion changes are subtle, some abnormalities are visually similar but have different clinical significance, and some cases without obvious changes may also show strong apparent differences due to changes in imaging conditions. Therefore, using only simple feature subtraction, ordinary multilayer perceptron, or linear differential modeling is insufficient to fully characterize the nonlinear, asymmetric, and fine-grained changes in disease progression, easily leading to confusion between the three states of worsening, improvement, and no obvious change, and especially increasing false alarms in cases with no obvious changes.

[0019] Regarding the utilization of semantic knowledge in reports, radiology reports typically include descriptions of lesion location, abnormalities, severity, and changes compared to previous examinations. This textual information reflects the physician's professional judgment on disease progression and has significant clinical semantic value. However, existing visual models largely rely on the images themselves to learn differential features, failing to fully utilize the semantic knowledge inherent in the existing report text during the training phase to constrain the representation of visual changes. If radiology reports are directly input during the inference phase, the system will be limited by factors such as reports not yet being generated, missing reports, delays in report generation, differences in writing habits among different institutions, and inconsistent text quality, making it unsuitable for deployment in actual clinical workflows.

[0020] To address at least one of the above problems, this application provides a method for longitudinal chest radiograph progression monitoring based on anatomical anchoring semantic difference. Anatomical candidate boxes for both front and rear chest radiographs are extracted using an anatomical region detector with the same set of parameters. These candidate boxes are then spatially united in a unified coordinate system to generate a shared anatomical support mask. A learnable gating mechanism is then used to softly fuse the mask with the original image, resulting in an anatomically anchored enhanced image. This ensures that all subsequent differential modeling is performed within the shared anatomical support. Next, the historical enhanced chest radiograph image and the current enhanced chest radiograph image undergo longitudinal differential processing in the image-report semantic embedding space (e.g., using a pre-trained and frozen image encoder with embedded report semantic knowledge; during the inference stage, the corresponding chest radiograph image can be directly input to extract features with clinical semantic priors). This yields report-aware semantic difference, which can extract features with clinical semantic priors without requiring a report input, thus avoiding the deployment obstacles of report dependence. The semantic difference directly... The differences between the images before and after representation in the clinical semantic space are analyzed, and multi-scale visual features are extracted using a visual encoder. After calculating the original visual differences, the differences are refined nonlinearly using a nonlinear difference projector. Then, gating parameters are generated based on historical visual features, and the refined difference information is asymmetrically injected into the current visual features (historical features remain unchanged, only the current features are updated). The nonlinear difference projector can learn independent nonlinear response curves for different feature dimensions, accurately depicting fine-grained and asymmetric lesion changes. The asymmetric injection method conforms to the clinical logic of "using history as a benchmark to judge current changes". The gating mechanism can adaptively modulate the difference weights of different regions. Finally, after multi-scale fusion and semantic recalibration, the progression classification is completed. The recalibration can guide the visual features to focus on changes related to clinical progression, suppress irrelevant grayscale and background differences, improve the semantic consistency of classification, and ultimately reduce the confusion probability of the three progression states.

[0021] The method for monitoring longitudinal chest radiograph progression based on anatomical anchoring semantic difference provided in this application will be described in detail below with reference to the accompanying drawings. In the embodiments of this application, the execution subject of the process can be a terminal device. The terminal device includes, but is not limited to, servers, computers, smartphones, and tablets, etc., that are capable of executing the method for monitoring longitudinal chest radiograph progression based on anatomical anchoring semantic difference provided in this application.

[0022] like Figure 1 As shown, Figure 1 This paper illustrates the overall architecture of the longitudinal chest radiograph progression monitoring method and system based on anatomical anchoring semantic difference provided in this application. The system consists of a main processing link and parallel semantic difference branches, which ultimately converge at the semantic recalibration stage to complete the progression determination. The main processing chain consists of nine stages: longitudinal image pair acquisition, anatomical region detection, shared anatomical support construction, anatomical anchoring enhancement, shared weight visual encoding, nonlinear visual difference modeling, multi-scale multi-segment fusion, semantic recalibration, and progress classification output. A report-aware semantic difference extraction branch is set in parallel below, and its output is fed into the semantic recalibration stage.

[0023] The specific content and flow of each step are as follows: Longitudinal image pair acquisition: Input historical chest X-ray images and current chest X-ray images, and match them to form longitudinal image pairs; Anatomical region detection: Using shared detector parameters, candidate regions are extracted from historical chest X-ray images and current chest X-ray images respectively, resulting in historical candidate regions and current candidate regions. Shared anatomical support construction: Based on the spatial extent of historical candidate regions and current candidate regions, a shared anatomical support (i.e., a "shared anatomical support mask") is generated. Anatomical anchoring enhancement: Enhancement processing is performed on historical chest X-ray images and current chest X-ray images based on shared anatomical support, and the corresponding output is historical enhanced chest X-ray images and current enhanced chest X-ray images. Figure 1 (These are referred to as "historical enhanced image" and "current enhanced image" respectively). Shared weight visual encoding: Taking historical enhanced images and current enhanced images as input, the images are processed by a shared weight encoder (or "shared weight visual encoder"), and the corresponding outputs are historical visual features and current visual features. Nonlinear visual difference modeling: The refined difference representation is output through visual difference calculation and nonlinear modeling in sequence. The refined difference representation is then asymmetrically injected into the current visual feature through gating to update the current visual feature. Multi-scale multi-segment fusion: fusing historical visual features, updated current visual features, and refined difference representations to output fused features (also referred to as "comprehensive visual change map" in this application); Report-aware semantic difference extraction (parallel branch): Taking historical and current enhanced images as input, the images are processed by a shared-weight semantic encoder (in this application, referring to an image encoder with embedded report semantic knowledge, pre-trained and frozen) to obtain historical and current semantic features. These features are then further processed by difference to obtain the report-aware semantic difference (in...). Figure 1 (referred to as "semantic difference" in Chinese) Semantic recalibration: Receive the fusion features output by multi-scale multi-segment fusion, and combine them with semantic difference to complete the recalibration process, outputting semantically enhanced features (also referred to as "change representation after semantic recalibration" in this application); Progress classification output: Based on semantic enhancement features, three progress status results are output: aggravated, improved, and no significant change.

[0024] based on Figure 1 See the provided overall architecture diagram. Figure 2 , Figure 2 This illustration shows a flowchart of a longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference, according to an embodiment of this application. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference provided in this embodiment includes: S10: Extract anatomical candidate regions from historical chest X-ray images and current chest X-ray images respectively, and perform spatial union in a unified image coordinate system to obtain a shared anatomical support mask.

[0025] S20, through learnable gating, the shared anatomical support mask is softly fused with the historical chest X-ray image and the current chest X-ray image respectively to obtain the historical enhanced chest X-ray image and the current enhanced chest X-ray image.

[0026] S30, perform longitudinal difference processing in the image-report semantic embedding space on the historical enhanced chest X-ray image and the current enhanced chest X-ray image to obtain the report-aware semantic difference.

[0027] S40: Input the historical enhanced chest X-ray image and the current enhanced chest X-ray image into the visual encoder to obtain multi-scale historical visual features and multi-scale current visual features. Then, perform difference operations on the historical visual features and current visual features corresponding to each scale to obtain the original visual difference of each scale.

[0028] S50: The original visual differences at each scale are processed by a nonlinear differential projector to generate refined differential representations at each scale. Then, based on the historical visual features at each scale, gate parameters for the corresponding scale are generated. The refined differential representations at each scale are asymmetrically injected into the current visual features at the corresponding scale through the gate parameters to obtain the updated current visual features at each scale.

[0029] S60 performs multi-scale differential fusion of historical visual features at each scale, updated current visual features, and refined differential representation to obtain a comprehensive visual change map. After semantic recalibration of the comprehensive visual change map using report-aware semantic differential, the progression status of chest diseases is output through global pooling and classification head.

[0030] Optionally, the above S10 may include: Acquire historical chest X-ray images and current chest X-ray images, and preprocess the historical chest X-ray images and current chest X-ray images. The preprocessing includes at least one of the following: image size normalization, grayscale normalization, invalid edge removal, and quality control. By inputting historical chest X-ray images and current chest X-ray images into an anatomical region detector with the same set of parameters, a set of historical candidate boxes and a set of current candidate boxes are obtained. Map the historical candidate box set and the current candidate box set to a unified image coordinate system, and construct a shared anatomical support mask according to the following rules: For any pixel location in the unified image coordinate system, if the pixel location is within the coverage area of ​​the historical candidate box set or the current candidate box set, the corresponding pixel location of the shared anatomical support mask is set to 1; otherwise, it is set to 0.

[0031] In practice, chest X-ray images with anatomical region annotations are used in the training set to train or fine-tune the target detection network, resulting in an anatomical region detector. Anatomical region detector Faster R-CNN, region proposal networks, DETR-type detectors, or other detection networks capable of outputting anatomical candidate boxes can be used.

[0032] Specifically, such as Figure 3 As shown, historical chest X-ray images and the current chest X-ray image are used to extract candidate regions using the same anatomical detector with shared parameters, resulting in a set of historical candidate boxes. and the current candidate box set Mapped to a unified image coordinate system and based on the historical candidate box set. With the current set of candidate boxes Constructing a shared anatomical support mask using a spatial union in a unified image coordinate system In this embodiment, a shared anatomical support mask is used. Construct it as follows: .

[0033] in, Indicates the image pixel location. Shared anatomical support mask. Used to indicate the location of the detected anatomical region at any given time point, allowing comparison of historical chest X-ray images and the current chest X-ray image within a common anatomical support area.

[0034] Shared anatomical support mask Instead of requiring rigid or non-rigid registration between historical and current chest X-ray images, the system utilizes the spatial union of anterior and posterior anatomical candidate regions to form a common anatomical range for longitudinal comparison. This approach allows for comparisons within a common anatomically relevant range even if the lesion area expands, shrinks, or undergoes slight positional changes during examinations.

[0035] Optionally, such as Figure 3 As shown, in S20 above, the original image is soft-gated fused using the following formula to obtain the corresponding anatomically enhanced chest X-ray: Anatomical Enhancement Images = ×(Original Image) Shared anatomical support mask) + (1- ) × original image. Where, For learnable gating parameters, This indicates element-wise multiplication. The original images are either historical chest X-rays or the current chest X-ray, and the anatomical enhanced chest X-rays correspond to either historical enhanced chest X-rays or the current enhanced chest X-ray, respectively.

[0036] This step reduces interference from irrelevant background responses and differences in acquisition conditions, while preserving the global context of the chest radiograph and avoiding the loss of lesion information caused by hard cropping.

[0037] Please see Figure 4 The diagram illustrates the two-stage implementation architecture of report-aware semantic differential in S30, which is divided into two parts: the optional semantic adaptation stage and the semantic differential extraction stage. The specific process is as follows: Optional Semantic Adaptation Stage: This stage takes the training set images and reports as input, feeds them into the visual language semantic adaptation module for training, and outputs the adapted image encoder. There are two implementation paths for this stage: either domain adaptation can be completed using the above-mentioned training set image-text pair, or a publicly available pre-trained image encoder can be directly used; regardless of the path used, the image encoder is frozen after adaptation and made available for use by downstream stages.

[0038] The semantic difference extraction stage takes anatomically anchored enhanced historical images (i.e., "historical enhanced chest X-ray images") and anatomically anchored enhanced current images (i.e., "current enhanced chest X-ray images") as inputs. These images are fed into the frozen image encoder via shared parameters, outputting historical semantic embeddings and current semantic embeddings, respectively. Subsequently, a difference operation (current-historical) is performed on the two sets of embeddings, ultimately outputting a report-aware semantic difference representation. This stage corresponds to the operation of the main progress monitoring model. During the main model training and inference stages, no radiological reports are input; the semantic priors of the reports have already been solidified into the image embeddings through the frozen image encoder.

[0039] More specifically, the optional semantic adaptation stage (or "pre-semantic adaptation stage") is based on the image encoder and text encoder, and uses chest X-ray images and corresponding radiology reports from the training set as supervision to complete image-text alignment training. A combination of image-text contrast loss and image-text matching loss is used for domain adaptation, aligning the image encoder and text encoder in the same semantic space, allowing the image encoder to acquire radiological semantic priors. After adaptation, the image encoder parameters are frozen. In this stage, the radiology reports are used as training supervision signals, enabling the frozen image encoder to acquire the semantic information of the radiology reports. After freezing the image encoder parameters, chest X-ray images from two different time points of the same patient are used as model inputs for training the main model; the main model is trained without radiology reports input throughout the training process.

[0040] The main model's network components include an anatomical region detector, a soft fusion network component based on learnable gating, a shared-weight visual encoder, a shared-weight semantic encoder, nonlinear differential projectors for each scale, a current visual feature update module with asymmetric differential injection, a multi-scale differential fusion network component, a semantic recalibration network component, and a classification head. The main model's network components also include an anatomical region detector (corresponding to...). Figure 1 "anatomical region detection") and report-aware semantic differential extraction network components (corresponding to Figure 1 (The parallel branches are shown). The frozen image encoder serves as the shared-weight semantic encoder of the main model. That is, the network architecture of the main model includes... Figure 1 The parameters of the anatomical region detector and the shared weight semantic encoder (or "image encoder") of each module / network component shown are frozen before the main model is trained and do not participate in the training process. After the main model is trained, the historical chest X-ray image and the current chest X-ray image are input into the main model to execute the process of S10 to S60.

[0041] In one implementation, the graph-text semantic adaptation loss Represented as: .

[0042] in, This represents chest X-ray images from the training set. Indicates a chest X-ray image The corresponding radiological report, This indicates the loss in image-text contrast. Indicates a text encoder. This represents the image-text matching loss. This represents the adjustable weighting coefficient that balances the two losses. This represents an image encoder. In one implementation, The value can be selected within the range of [0.1, 1.0] and can be adjusted according to the size of the training set, the quality of image-text alignment, and the performance of the validation set.

[0043] If training set image-report pairs are used for image-text semantic adaptation, the adaptation process uses only images and corresponding reports from the training set, without using reports, tags, or patient information from the validation or test sets.

[0044] After the visual language model is selected or semantic adaptation is completed, freeze the image encoder. The parameters remain unchanged during subsequent master model training and inference phases. No radiological reports are input during master model training and inference phases. The system only uses anatomically anchored historical enhanced chest radiographs. and current enhanced chest X-ray images Input frozen image encoder To obtain historical semantic embedding and current semantic embedding : ;

[0045] In obtaining historical semantic embeddings and current semantic embedding Then, embed the current semantics. Subtracting historical semantic embedding The report obtained is a semantic difference perception.

[0046] Through the above scheme, the embodiments of this application can still introduce the radiological semantic priors learned in the image-text alignment model without relying on the radiological report input during the inference stage, so as to provide semantic constraints for subsequent visual change image semantic recalibration.

[0047] In S40 above, the visual encoder Convolutional neural networks, Transformer, SwinTransformer, ConvNeXt, or combinations thereof can be used. L The first in the scale l Historical visual features of scale , No. l Current visual features at scale They are represented as follows:

[0048] .

[0049] in, l Indicates scale level. L Indicates the total number of scales.

[0050] like Figure 5 As shown, Figure 5 A schematic diagram of nonlinear visual differential modeling and asymmetric differential injection mechanism is shown. The following steps are performed in the intermediate layers or feature layers at various scales of the visual encoder: For the l Current visual features at scale and historical visual features Perform a difference operation to obtain the first... l Original visual difference at scale :

[0051] The original visual difference As input to step S50, it is used for subsequent nonlinear differential refining and asymmetric differential injection.

[0052] Then, the original visual difference Input nonlinear differential projector A refined difference representation is obtained using a nonlinear difference projector. KAN layers are preferred, but learnable edge activation networks, spline basis function networks, multilayer perceptrons, or variations thereof can also be used. KAN layers learn parameterizable nonlinear activation functions or spline basis functions on the edges, thus learning different change response curves for different feature dimensions, thereby enhancing the ability to express fine-grained, nonlinear, and asymmetric changes in the progression of disease on chest X-rays.

[0053] Subsequently, the refined difference representations at each scale are asymmetrically injected into the current visual features at the corresponding scale using gating parameters, including: By processing historical visual features at various scales through activation functions, spatial gating or channel gating at various scales is generated. The refined differential representation is modulated using spatial gating or channel gating at each scale, and the modulated differential information is injected into the current visual features at the corresponding scale to obtain the updated current visual features at each scale.

[0054] More specifically, according to the first l Historical visual features of scale Conduct the first l Scale-based spatial gating or channel gating Generation: .

[0055] in, This represents the Sigmoid function. and Indicates the first l Learnable parameters corresponding to the scale.

[0056] Subsequently, using the first l Scale-based spatial gating or channel gating For the l Refined difference representation of scale Modulation is performed, and the modulated differential information is injected into the first... l Current visual features at scale , obtained the l Current visual features after scale update This is used to enhance the ability to model fine-grained, nonlinear, and non-linear aspects: .

[0057] in This indicates element-wise multiplication. A learnable scaling factor. Historical visual features. The feature is retained as a reference feature and no symmetrical update is performed. This asymmetric update method is used to emphasize the changes in the current check relative to the historical check, rather than a simple symmetrical fusion of historical and current features.

[0058] like Figure 6 As shown, Figure 6 This diagram illustrates the multi-scale differential fusion, semantic recalibration, and progress classification output involved in an embodiment of this application. In S60 above, the historical visual features at each scale in the multi-scale historical visual features are processed. , ... The updated current visual features at each scale in the multi-scale current visual features , ... and the refined difference representations of each scale in the multi-scale refined difference representation. , ... (Refined difference representation in) Figure 6 (Not shown in the image) The corresponding scale difference is constructed to obtain the scale-level difference map. .

[0059] Among them, multi-scale difference is obtained through differential convolution modules, differential attention modules, or a combination of both. l Scale difference plot :

[0060] in, Indicates the first l The differential feature extraction module corresponding to the scale. This indicates a feature splicing operation.

[0061] Subsequently, the scale-level difference maps at multiple scales are input into the multi-scale difference fusion module. The multi-scale difference fusion module adopts a top-down feature pyramid fusion strategy to fuse high-level visual semantic difference information and low-level spatial difference information to obtain fused difference features (or comprehensive visual change map). ).

[0062] In one implementation, the multi-scale fusion process is represented as:

[0063] .

[0064] in, Indicates an upsampling operation. This represents a multi-scale difference fusion function. , , , These represent the difference map at the first scale after upsampling (incorporating higher-level information), the difference map at the second scale after upsampling and recursive optimization, and the difference map at the third scale after upsampling and recursive optimization, respectively. l The difference plot of the scale, the first after upsampling recursive optimization l+ A difference map at scale 1, and so on, with layer L being the deepest layer, therefore... Without integrating high-level information, there is = .

[0065] Then, the comprehensive visual change diagram will be used. and report-aware semantic difference The input semantic recalibration module uses report-aware semantic differential representation. Comprehensive visual change diagram Perform semantic recalibration to obtain the semantically recalibrated change representation. Semantic recalibration can be implemented using methods such as cross-attention, scaling-offset modulation, gated fusion, or combinations thereof.

[0066] In one implementation, the integrated visual change map As a query feature, report-aware semantic differential Mapped to keys and values, the semantically recalibrated changes are obtained through cross-attention. : .

[0067] in, This indicates a cross-attention operation. and This represents the learnable mapping parameters used to generate keys and values, where Q represents the query vector, K represents the key vector, and V represents the value vector.

[0068] In another implementation, report-aware semantic differential is used. Input a lightweight mapping network to generate channel-level scaling parameters. and offset parameters :

[0069]

[0070] Subsequently, the comprehensive visual change diagram was analyzed. Scale-offset modulation is performed to obtain the semantically recalibrated change representation. :

[0071] in, 、 、 and Indicates learnable parameters, This indicates element-wise multiplication. This step is used to enhance the visual channel or region response that is semantically consistent with radiological changes and to suppress background differences, grayscale differences, or acquisition condition differences that are not related to disease progression.

[0072] Subsequently, the changes after semantic recalibration are represented. Global average pooling is performed to obtain the anatomical anchoring semantic difference representation. .

[0073] Finally, the anatomy anchored semantic differential representation will be analyzed. Input a classification header, output the probability of progression state corresponding to the disease label, anatomical region, or disease-region combination: .

[0074] in, This represents the probability distribution of the progression state given historical chest X-ray images and the current chest X-ray image. and The classification header parameter indicates that the progress status includes at least three categories: aggravation, improvement, and no significant change.

[0075] The classification head can have different output formats. In one implementation, the classification head outputs a three-class classification result for a single disease label.

[0076] In other implementations, the classification head outputs multi-label progression status results for multiple disease labels, multiple anatomical regions, or disease-region combinations.

[0077] Optionally, the longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference can also output a semantically recalibrated visual change map, anatomical region-level response, or disease-related region response to assist doctors in reviewing the basis for progress judgment.

[0078] The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference provided in this application has the following advantages compared with the prior art: 1. This application can improve the consistency of longitudinal chest radiograph spatial comparison. By constructing a shared anatomical support mask and using learnable gating to perform anatomical anchoring enhancement on historical and current chest radiograph images, the images can be compared within a common anatomical support range. This reduces the interference of patient position, projection angle, inspiratory degree, field of view coverage, and equipment parameter differences on progression assessment, while avoiding the loss of effective lesion information caused by hard cropping.

[0079] 2. This application enables the utilization of report semantic knowledge during the inference phase without relying on radiological report input. It achieves text-image semantic adaptation using a publicly pre-trained CLIP-like visual language model, or solely through training set image-report pairs, and freezes the image encoder after model selection or adaptation. During the main model training and inference phases, the system only requires input of anatomically anchored enhanced historical chest X-ray images and the current chest X-ray image to obtain report-aware semantic differences. Thus, it is possible to introduce radiological semantic priors learned in the text-image alignment model into the longitudinal variation representation without introducing report dependence during the inference phase.

[0080] 3. This application enhances the ability to model fine-grained, nonlinear, and asymmetric disease changes. It calculates the original visual difference between historical and current visual features and obtains a refined difference representation using a nonlinear difference projector. Furthermore, it generates spatial or channel gating based on historical visual features and asymmetrically injects the refined difference representation into the current visual features. This allows for a more comprehensive characterization of the complex visual relationships in disease manifestation changes, reducing confusion between the three states of aggravation, improvement, and no significant change.

[0081] 4. This application can improve the consistency between visual change features and radiological semantics. It performs semantic recalibration on the comprehensive visual change map through report-aware semantic difference, obtaining a semantically recalibrated change representation. Semantic recalibration can be achieved using cross-attention, channel-level scale-offset modulation, gated fusion, or a combination thereof. Cross-attention can establish a global correspondence between report-aware semantic difference and the visual change map, making it suitable for capturing semantically relevant changes across regions and scales. Channel-level scale-offset modulation adjusts the visual response by generating channel-level scaling and offset parameters, resulting in lower computational overhead and suitability for deployment scenarios requiring high inference efficiency. Therefore, it can enhance the visual channel or region response relevant to clinical change descriptions and suppress irrelevant responses caused by background, grayscale changes, or differences in acquisition conditions.

[0082] 5. This application can provide structured progression monitoring results. It can output the probabilities of aggravation, improvement, and no significant change according to disease labels, anatomical regions, or disease-region combinations; optionally, it can also output semantically recalibrated visual change maps, anatomical region-level responses, or disease-related region responses. These output formats facilitate physician review of the model's judgment basis and are easily integrated with hospital image archiving, interpretation, and quality control processes.

[0083] Accordingly, embodiments of this application also provide a longitudinal chest radiograph progression monitoring system based on anatomical anchoring semantic difference, comprising: The mask construction module is used to extract anatomical candidate regions from historical chest X-ray images and current chest X-ray images, respectively, and perform spatial union in a unified image coordinate system to obtain a shared anatomical support mask; The mask enhancement module is used to softly fuse a shared anatomical support mask with historical chest X-ray images and current chest X-ray images respectively through learnable gating to obtain historical enhanced chest X-ray images and current enhanced chest X-ray images; The semantic difference module is used to perform longitudinal difference processing in the image-report semantic embedding space on historical enhanced chest X-ray images and current enhanced chest X-ray images to obtain report-aware semantic difference. The visual difference module is used to input historical enhanced chest X-ray images and current enhanced chest X-ray images into the visual encoder to obtain multi-scale historical visual features and multi-scale current visual features, and to perform difference operations on the historical visual features and current visual features corresponding to each scale to obtain the original visual difference of each scale. The visual feature update module is used to generate refined difference representations of each scale from the original visual differences at each scale through a nonlinear difference projector, and then generate corresponding scale gating parameters based on the historical visual features at each scale. The refined difference representations of each scale are asymmetrically injected into the current visual features of the corresponding scale through the gating parameters to obtain the updated current visual features at each scale. The results output module is used to perform multi-scale difference fusion of historical visual features, updated current visual features, and refined difference representations at various scales to obtain a comprehensive visual change map. After semantic recalibration of the comprehensive visual change map using report-aware semantic difference, the progress status of chest diseases is output through global pooling and classification head.

[0084] It should be noted that the information interaction and execution process between the above systems are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0085] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0086] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for monitoring the progression of chest radiographs based on anatomical anchoring semantic difference, characterized in that, include: Anatomical candidate regions from historical chest X-ray images and current chest X-ray images are extracted and spatially joined in a unified image coordinate system to obtain a shared anatomical support mask; By using learnable gating, the shared anatomical support mask is softly fused with historical chest X-ray images and current chest X-ray images respectively to obtain historical enhanced chest X-ray images and current enhanced chest X-ray images. The historical enhanced chest X-ray image and the current enhanced chest X-ray image are subjected to longitudinal difference processing in the image-report semantic embedding space to obtain the report-aware semantic difference; The historical enhanced chest X-ray image and the current enhanced chest X-ray image are input into the visual encoder to obtain multi-scale historical visual features and multi-scale current visual features. The historical visual features and current visual features corresponding to each scale are then subjected to difference operations to obtain the original visual differences for each scale. The original visual differences at each scale are processed by a nonlinear differential projector to generate refined differential representations at each scale. Then, based on the historical visual features at each scale, gate parameters for the corresponding scale are generated. The refined differential representations at each scale are asymmetrically injected into the current visual features at the corresponding scale through the gate parameters to obtain the updated current visual features at each scale. Multi-scale differential fusion is performed on historical visual features at various scales, updated current visual features, and refined differential representations to obtain a comprehensive visual change map. After semantic recalibration of the comprehensive visual change map using report-aware semantic differential, the progression status of chest diseases is output through global pooling and classification head.

2. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference as described in claim 1, characterized in that, The step of performing longitudinal difference processing in the image-report semantic embedding space on historical enhanced chest X-ray images and current enhanced chest X-ray images to obtain report-aware semantic difference includes: Input the historical enhanced chest X-ray image and the current enhanced chest X-ray image into a pre-trained and frozen image-text aligned image encoder to obtain the historical semantic embedding and the current semantic embedding; The difference between historical semantic embeddings and current semantic embeddings is calculated to obtain the report-aware semantic difference.

3. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference as described in claim 2, characterized in that, The image encoder with image-text alignment is obtained through training in the preceding image-text semantic adaptation stage. The pre-image-text semantic adaptation stage is based on the image encoder and the text encoder, and the image-text alignment training is completed with the chest X-ray images and corresponding radiological reports in the training set as supervision. The combined objective of image-text contrast loss and image-text matching loss is used for domain adaptation, so that the image encoder and the text encoder are aligned in the same semantic space, and the image encoder obtains radiological semantic prior. The method also includes freezing the image encoder parameters after the adaptation is completed.

4. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference as described in claim 3, characterized in that, The method further includes: After freezing the image encoder parameters, chest X-ray images of the same patient at two different time points were used as input to train the main model. No radiological reports were input during the training of the main model. The main model's network components include an anatomical region detector, a soft fusion network component based on learnable gating, a visual encoder, a shared weight semantic encoder, nonlinear differential projectors for each scale, a current visual feature update module with asymmetric differential injection, a multi-scale differential fusion network component, a semantic recalibration network component, and a classification head; among which, the frozen image encoder serves as the shared weight semantic encoder of the main model.

5. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference as described in claim 1, characterized in that, The step of extracting anatomical candidate regions from historical chest X-ray images and the current chest X-ray image, respectively, and performing spatial union in a unified image coordinate system to obtain a shared anatomical support mask includes: Acquire historical chest X-ray images and current chest X-ray images, and preprocess the historical chest X-ray images and current chest X-ray images. The preprocessing includes at least one of the following: image size normalization, grayscale normalization, invalid edge removal, and quality control. By inputting historical chest X-ray images and current chest X-ray images into an anatomical region detector with the same set of parameters, a set of historical candidate boxes and a set of current candidate boxes are obtained. Map the historical candidate box set and the current candidate box set to a unified image coordinate system, and construct a shared anatomical support mask according to the following rules: For any pixel location in the unified image coordinate system, if the pixel location is within the coverage area of ​​the historical candidate box set or the current candidate box set, the corresponding pixel location of the shared anatomical support mask is set to 1; otherwise, it is set to 0.

6. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference as described in claim 1, characterized in that, The process of softly fusing a shared anatomical support mask with historical and current chest X-ray images using learnable gating to obtain historical and current enhanced chest X-ray images includes: The following formula is used for soft fusion to obtain the corresponding historical enhanced chest X-ray image and the current enhanced chest X-ray image: Historical enhanced chest X-ray image = ×(Historical chest X-ray image) Shared anatomical support mask) + (1- ) × Historical chest X-ray images; Current enhanced chest X-ray image = ×(Current chest X-ray image) Shared anatomical support mask) + (1- × Current chest X-ray image; in, For learnable gating parameters, This indicates element-wise multiplication.

7. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference as described in claim 1, characterized in that, The process involves generating corresponding scale-specific gating parameters based on historical visual features at each scale. These gating parameters are then used to asymmetrically inject refined difference representations of each scale into the current visual features at the corresponding scale, resulting in updated current visual features at each scale. This includes: By processing historical visual features at various scales through activation functions, spatial gating or channel gating at various scales is generated. The refined differential representation is modulated using spatial gating or channel gating at each scale, and the modulated differential information is injected into the current visual features at the corresponding scale to obtain the updated current visual features at each scale.

8. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference as described in claim 1, characterized in that, The multi-scale differential fusion adopts a top-down feature pyramid fusion strategy; And / or, the semantic recalibration is achieved using one or a combination of at least two of the following: cross-attention, channel-level scale-offset modulation, and gated fusion.

9. The longitudinal chest radiograph progression monitoring method based on anatomical anchoring semantic difference as described in claim 1, characterized in that, The output formats of the classification head include: three-class classification results for a single disease label, multi-label progression results for multiple disease labels, regional-level progression results for multiple anatomical regions, or fine-grained progression results for disease-region combinations.

10. A longitudinal chest radiograph progression monitoring system based on anatomical anchoring semantic difference, characterized in that, include: The mask construction module is used to extract anatomical candidate regions from historical chest X-ray images and current chest X-ray images, respectively, and perform spatial union in a unified image coordinate system to obtain a shared anatomical support mask; The mask enhancement module is used to softly fuse a shared anatomical support mask with historical chest X-ray images and current chest X-ray images respectively through learnable gating to obtain historical enhanced chest X-ray images and current enhanced chest X-ray images; The semantic difference module is used to perform longitudinal difference processing in the image-report semantic embedding space on historical enhanced chest X-ray images and current enhanced chest X-ray images to obtain report-aware semantic difference. The visual difference module is used to input historical enhanced chest X-ray images and current enhanced chest X-ray images into the visual encoder to obtain multi-scale historical visual features and multi-scale current visual features, and to perform difference operations on the historical visual features and current visual features corresponding to each scale to obtain the original visual difference of each scale. The visual feature update module is used to generate refined difference representations of each scale from the original visual differences at each scale through a nonlinear difference projector, and then generate corresponding scale gating parameters based on the historical visual features at each scale. The refined difference representations of each scale are asymmetrically injected into the current visual features of the corresponding scale through the gating parameters to obtain the updated current visual features at each scale. The results output module is used to perform multi-scale difference fusion of historical visual features, updated current visual features, and refined difference representations at various scales to obtain a comprehensive visual change map. After semantic recalibration of the comprehensive visual change map using report-aware semantic difference, the progress status of chest diseases is output through global pooling and classification head.

Images