A dynamic interactive sacroiliac joint image assisted analysis system simulating a clinical reading path and an image processing method

Abstract

The application belongs to the technical field of image processing, and discloses a dynamic interactive sacroiliac joint image auxiliary analysis system simulating a clinical reading path and an image processing method, which comprises: an image data acquisition and preprocessing module, which is used for acquiring and preprocessing X-ray image data and MRI image data that have been shot; an X-ray preliminary judgment positioning module, which is used for performing feature extraction and partition positioning processing on the X-ray image data and the MRI image data based on a first visual language model; an MRI directional iterative auxiliary analysis module, which is used for performing feature extraction on the MRI image data based on a second visual language model; a decision module combining a large language model and a knowledge graph is used to generate structured auxiliary analysis information; and an auxiliary analysis convergence and output module, which is used for integrating the auxiliary analysis information, generating and outputting a structured radiology auxiliary analysis report and a visual result. The application can significantly reduce the slice processing amount, improve the auxiliary analysis efficiency, accuracy and interpretability.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a dynamic interactive sacroiliac joint image-assisted analysis system and image processing method that simulates a clinical image reading path. Background Technology

[0002] Clinical auxiliary analysis of ankylosing spondylitis (AS) relies heavily on X-ray and MRI images of the sacroiliac joint. The diagnosis is mainly based on imaging signs such as bone marrow edema, fatty infiltration, bone erosion, and joint space narrowing. These auxiliary analysis methods have drawbacks such as low efficiency, poor auxiliary analysis capabilities, and low accuracy.

[0003] Currently, AI (artificial intelligence)-assisted analysis of medical images has been widely used, and related AI solutions have also emerged in the field of sacroiliac joint / ankylosing spondylitis: 1. A deep learning-based detection / segmentation model for feature recognition and grading of MRI slices; 2. Image report generation based on Visual Language Model (VLM), directly outputting auxiliary analysis text; 3. End-to-end classification model, directly outputting classification results.

[0004] However, existing AI-assisted medical image analysis technologies still suffer from problems such as inefficient input methods, inconsistent analysis paths, lack of dynamic decision-making, and poor interpretability.

[0005] Therefore, existing technologies still need improvement. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a dynamic interactive sacroiliac joint image-assisted analysis system and image processing method that simulates the clinical image reading path, in order to solve the problems of inefficient input methods, inconsistent analysis paths, lack of dynamic decision-making and poor interpretability in existing medical image AI-assisted analysis technology.

[0007] The technical solution adopted by this invention to solve the technical problem is as follows: In a first aspect, the present invention provides a dynamic interactive sacroiliac joint image-assisted analysis system that simulates a clinical image reading path, comprising: The image data acquisition and preprocessing module is used to acquire the captured X-ray image data and MRI image data; to annotate the X-ray image data and MRI image data using a medical image annotation tool; and to filter the annotated image data to obtain the filtered X-ray image data and MRI image data. The X-ray initial positioning module is used to perform feature extraction and zonal positioning processing on the X-ray image data and the MRI image data based on the first visual language model to obtain a sacroiliac joint zonal positioning map. The MRI-oriented iterative auxiliary analysis module is used to extract features from the MRI image data based on a second visual language model to obtain sacroiliac joint features and corresponding attribute information. The attribute information includes location, range, degree, and confidence level. The module is also used to generate structured auxiliary analysis information based on the sacroiliac joint zonal localization map, the sacroiliac joint features, and the corresponding attribute information, using a decision module that combines a large language model and a knowledge graph. The auxiliary analysis convergence and output module is used to integrate all structured auxiliary analysis information, generate and output structured radiological auxiliary analysis reports and visualization results.

[0008] In one implementation, the X-ray initial positioning module includes: The first input unit is used to input the X-ray image data and the MRI image data into the first visual language model; wherein, the first visual language model is a fine-tuned model; A radiological lesion feature recognition unit is used to identify radiological lesion features and their corresponding location, type, severity, confidence level, and anatomical region; wherein, the radiological lesion features include: bone sclerosis features, joint space narrowing features, and bone erosion features; The anatomical partition unit is used to locate the entire area of ​​the sacroiliac joint through target detection and automatically divide it into multiple anatomical partitions according to the anatomical structure, obtaining the detection box and pixel-level semantic segmentation mask for each anatomical partition. The abnormal partitioning unit is used to mark the anatomical partitions with abnormalities based on the correspondence between the identified radiological lesion features and the located anatomical partitions, and to score them according to the priority scoring rules, sorting them from high to low according to the comprehensive score to generate a priority list of suspicious abnormal partitions. The first output unit is used to output the sacroiliac joint zoning map and the priority list of the suspected abnormal zoning.

[0009] In one implementation, the MRI-guided iterative analysis module includes: The second input unit is used to input a single MRI slice from the MRI image data into the second visual language model; wherein the second visual language model is a model that is derived from the first visual language model but is independently fine-tuned; The MRI pathological feature extraction unit is used to extract MRI pathological features based on MRI fine-tuning knowledge, and output the corresponding lesion signals, extent, grade, and affected areas layer by layer to obtain the sacroiliac joint features and corresponding attribute information; wherein, the MRI pathological features include: bone marrow edema features, fat infiltration features, bone sclerosis features, bone erosion features, synovitis features, and tendonitis features. The dual memory bank status generation unit is used to automatically classify the corresponding partitions and features into the effective memory bank or classify the negative normal areas into the exclusion memory bank according to the positive high confidence rule, based on the sacroiliac joint partition location map, the priority list of suspicious abnormal partitions, the sacroiliac joint features and corresponding attribute information, and update the partitions, lesion types and sign confidence information in the memory bank in real time to generate the dual memory bank status. The candidate examination action generation unit is used to combine the current iteration round, the information of the double memory that has been investigated / diagnosed, and the priority of the lesions in the partition, match the sequence priority rules of the knowledge graph of the ankylosing spondylitis auxiliary analysis, and output candidate examination actions that include sequence type, target anatomical partition, and slice level range. The candidate examination action optimization unit is used to constrain, score, and correct the candidate examination actions based on the dual memory bank state and the candidate examination actions, and to eliminate actions that do not conform to clinical norms. The second output unit is used to output the next check action of the current dual memory state to obtain the structured auxiliary analysis information.

[0010] In one implementation, the auxiliary analysis convergence and output module includes: The auxiliary analysis report generation unit is used to integrate and generate the structured radiological auxiliary analysis report based on the ankylosing spondylitis imaging detection standards, using all the structured auxiliary analysis information in the dual memory bank. The dual memory bank includes: an effective memory bank for storing positive signs, high-confidence lesion areas, and key evidence slices, and an exclusion memory bank for storing areas confirmed to be without abnormalities. The output and visualization unit outputs the structured radiological analysis report and the corresponding visualization results.

[0011] In a second aspect, the present invention provides an image processing method based on the dynamic interactive sacroiliac joint image-assisted analysis system that simulates a clinical image reading path as described in the first aspect, comprising: Acquire the captured X-ray and MRI image data; Based on the first visual language model, feature extraction and zoning localization processing are performed on the X-ray image data and the MRI image data to obtain a sacroiliac joint zoning localization map. Feature extraction is performed on the MRI image data based on a second visual language model to obtain sacroiliac joint features and corresponding attribute information; wherein, the attribute information includes: location, range, degree, and confidence level; Based on the sacroiliac joint zoning map, the sacroiliac joint features, and the corresponding attribute information, a structured auxiliary analysis information is generated using a decision module that combines a large language model and a knowledge graph. All structured auxiliary analysis information is integrated to generate and output structured radiological auxiliary analysis reports and visualization results.

[0012] In one implementation, acquiring the captured X-ray image data and MRI image data includes: Acquire X-ray images of the pelvis in the anteroposterior position and MRI images for anatomical spatial alignment and zonal localization; The X-ray image data and the MRI image data are annotated using a medical image annotation tool, and the annotated image data are then filtered to obtain the filtered X-ray image data and MRI image data.

[0013] In one implementation, the step of performing feature extraction and zoning localization processing on the X-ray image data and the MRI image data based on a first visual language model to obtain a sacroiliac joint zoning localization map includes: The X-ray image data and the MRI image data are input into the first visual language model; wherein, the first visual language model is a fine-tuned model; Identify radiographic lesion features and their corresponding location, type, severity, confidence level, and anatomical region; wherein, the radiographic lesion features include: bone sclerosis features, joint space narrowing features, and bone erosion features; The entire area of ​​the sacroiliac joint is located by target detection and automatically segmented into multiple anatomical regions according to the anatomical structure, resulting in a detection box and pixel-level semantic segmentation mask for each anatomical region. Based on the correspondence between the identified radiological lesion features and the located anatomical regions, the anatomical regions with abnormalities are marked, and scores are assigned according to priority scoring rules. The regions are then sorted from high to low based on their comprehensive scores to generate a priority list of suspected abnormal regions. Output the sacroiliac joint zonal location map and the priority list of the suspected abnormal zonal regions.

[0014] In one implementation, the step of extracting features from the MRI image data based on a second visual language model to obtain sacroiliac joint features and corresponding attribute information includes: A single MRI slice from the MRI image data is input into the second visual language model; wherein, the second visual language model is a model that is of the same origin as the first visual language model but is independently fine-tuned; Based on MRI fine-tuning knowledge, MRI pathological features are extracted, and the corresponding lesion signals, extent, grade, and involved regions are output layer by layer to obtain the sacroiliac joint features and corresponding attribute information. The MRI pathological features include: bone marrow edema, fatty infiltration, bone sclerosis, bone erosion, synovitis, and tendonitis.

[0015] In one implementation, the step of generating structured auxiliary analysis information based on the sacroiliac joint localization map, the sacroiliac joint features, and corresponding attribute information, using a decision module combining a large language model and a knowledge graph, includes: Based on the sacroiliac joint zoning map, the priority list of suspected abnormal zoning areas, the sacroiliac joint features and corresponding attribute information, the corresponding zoning areas and features are automatically classified into the effective memory bank according to the positive high confidence rule, or the negative normal areas are classified into the exclusion memory bank, and the zoning areas, lesion types, and sign confidence information in the bank are updated in real time to generate a dual memory bank status. Combining the current iteration round, the information of the double memory database that has been investigated / diagnosed, and the priority of the lesions in the partition, the sequence priority rules of the knowledge graph of the ankylosing spondylitis auxiliary analysis are matched, and candidate examination actions containing sequence type, target anatomical partition, and slice level range are output. Based on the dual memory bank state and the candidate examination actions, the ankylosing spondylitis auxiliary analysis knowledge graph is used to constrain, score, and correct the candidate examination actions, eliminating actions that do not conform to clinical norms. Output the next check action for the current dual memory state to obtain the structured auxiliary analysis information.

[0016] In one implementation, the integration of all structured auxiliary analysis information to generate and output a structured radiological auxiliary analysis report and visualization results includes: For all the structured auxiliary analysis information in the dual memory bank, the structured radiological auxiliary analysis report is integrated and generated according to the imaging detection standards for ankylosing spondylitis; wherein, the dual memory bank includes: an effective memory bank for storing positive signs, high-confidence lesion areas and key evidence slices, and an exclusion memory bank for storing areas confirmed to be without abnormalities; Output the structured radiological analysis report and the corresponding visualization results.

[0017] The present invention, by employing the above technical solution, has the following effects: 1) The system of the present invention reconstructs the image analysis process into three stages: X-ray initial positioning → MRI directional iterative screening → convergence and output of analysis information. It achieves efficient storage and reuse of auxiliary analysis information through a structured dual memory bank, thereby solving the problem of long sequence context exceeding the limit. 2) The present invention introduces a knowledge graph of ankylosing spondylitis sacroiliac joint auxiliary analysis to constrain and score the agent's decision-making, ensuring that the decision-making conforms to the norm; 3) The system of the present invention can dynamically select the MRI sequence type, anatomical region and slice range based on the results of the preceding auxiliary analysis, realizing lightweight, directional and interpretable iterative auxiliary analysis; 4) The system of the present invention can significantly reduce the amount of slice processing, improve the efficiency and accuracy of auxiliary analysis, and make the decision-making process traceable and visualized. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0019] Figure 1 This is a functional principle diagram of the dynamic interactive sacroiliac joint image-assisted analysis system based on a simulated clinical image reading path in this invention.

[0020] Figure 2 This is a flowchart of the image processing method of the dynamic interactive sacroiliac joint image-assisted analysis system based on a simulated clinical image reading path in this invention.

[0021] Figure 3 This is an overall flowchart of the image processing method in this invention.

[0022] Figure 4 This is a detailed flowchart of the MRI-guided iterative auxiliary analysis in this invention.

[0023] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0025] Exemplary System Currently, AI (artificial intelligence)-assisted analysis of medical images has been widely used, and related AI solutions have also emerged in the field of sacroiliac joint / ankylosing spondylitis: 1. A deep learning-based detection / segmentation model for feature recognition and grading of MRI slices; 2. Image report generation based on Visual Language Model (VLM), directly outputting auxiliary analysis text; 3. End-to-end classification model, directly outputting classification results.

[0026] However, existing AI-assisted medical image analysis technologies still have the following drawbacks: 1) Inefficient input method: Static full slice input is used, which sends all sequences and all layers into the model at once, processing a large amount of irrelevant information, resulting in serious waste of computing resources and slow inference speed.

[0027] 2) Disconnected from clinical pathways: The model does not follow the real clinical analysis process of "X-ray initial screening and localization → MRI sequence orientation verification → key area review". The model's decision-making logic is inconsistent with doctors' habits and is difficult to be accepted clinically.

[0028] 3) Lack of dynamic decision-making ability: The results are output in one go and cannot be dynamically adjusted according to the previous findings to adjust the subsequent examination sequence, anatomical partitions and slice ranges. It cannot achieve iterative auxiliary analysis of "making decisions while looking".

[0029] 4) Limited context length: Large visual models have a token length limit, which makes it impossible to fully process long MRI sequences and easily loses key information.

[0030] 5) Insufficient interpretability: Most of the output is black box, and it is impossible to know which slices, regions, and signs the model draws conclusions on, lacking evidence chains and traceability.

[0031] 6) Unstructured memory mechanism: It cannot save and reuse the auxiliary analysis information at each step, cannot manage positive and excluded areas, and is prone to duplicate and contradictory judgments.

[0032] To address the above-mentioned technical problems, this invention provides a dynamic interactive sacroiliac joint image-assisted analysis system that simulates a clinical image reading path. This system can significantly reduce the amount of image slides processed and improve the efficiency, accuracy, and interpretability of the assisted analysis.

[0033] like Figure 1 As shown, this embodiment provides a dynamic interactive sacroiliac joint image-assisted analysis system that simulates a clinical image reading path, including: The image data acquisition and preprocessing module 10 is used to acquire the captured X-ray image data and MRI image data; to annotate the X-ray image data and MRI image data using a medical image annotation tool; and to filter the annotated image data to obtain the filtered X-ray image data and MRI image data. X-ray initial positioning module 20 is used to perform feature extraction and zonal positioning processing on the X-ray image data and the MRI image data based on the first visual language model to obtain a sacroiliac joint zonal positioning map. The MRI-oriented iterative auxiliary analysis module 30 is used to extract features from the MRI image data based on a second visual language model to obtain sacroiliac joint features and corresponding attribute information; wherein, the attribute information includes: location, range, degree, and confidence level; and is used to generate structured auxiliary analysis information based on the sacroiliac joint zonal localization map, the sacroiliac joint features, and the corresponding attribute information, using a decision module that combines a large language model and a knowledge graph. The auxiliary analysis convergence and output module 40 is used to integrate all the structured auxiliary analysis information, generate and output structured radiological auxiliary analysis reports and visualization results.

[0034] In this embodiment, the X-ray initial positioning module 20 includes: The first input unit is used to input the X-ray image data and the MRI image data into the first visual language model; wherein, the first visual language model is a fine-tuned model; A radiological lesion feature recognition unit is used to identify radiological lesion features and their corresponding location, type, severity, confidence level, and anatomical region; wherein, the radiological lesion features include: bone sclerosis features, joint space narrowing features, and bone erosion features; The anatomical partition unit is used to locate the entire area of ​​the sacroiliac joint through target detection and automatically divide it into multiple anatomical partitions according to the anatomical structure, obtaining the detection box and pixel-level semantic segmentation mask for each anatomical partition. The abnormal partitioning unit is used to mark the anatomical partitions with abnormalities based on the correspondence between the identified radiological lesion features and the located anatomical partitions, and to score them according to the priority scoring rules, sorting them from high to low according to the comprehensive score to generate a priority list of suspicious abnormal partitions. The first output unit is used to output the sacroiliac joint zoning map and the priority list of the suspected abnormal zoning.

[0035] In this embodiment, the MRI-guided iterative auxiliary analysis module 30 includes: The second input unit is used to input a single MRI slice from the MRI image data into the second visual language model; wherein the second visual language model is a model that is derived from the first visual language model but is independently fine-tuned; The MRI pathological feature extraction unit is used to extract MRI pathological features based on MRI fine-tuning knowledge, and output the corresponding lesion signals, extent, grade, and affected areas layer by layer to obtain the sacroiliac joint features and corresponding attribute information; wherein, the MRI pathological features include: bone marrow edema features, fat infiltration features, bone sclerosis features, bone erosion features, synovitis features, and tendonitis features. The dual memory bank status generation unit is used to automatically classify the corresponding partitions and features into the effective memory bank or classify the negative normal areas into the exclusion memory bank according to the positive high confidence rule, based on the sacroiliac joint partition location map, the priority list of suspicious abnormal partitions, the sacroiliac joint features and corresponding attribute information, and update the partitions, lesion types and sign confidence information in the memory bank in real time to generate the dual memory bank status. The candidate examination action generation unit is used to combine the current iteration round, the information of the double memory that has been investigated / diagnosed, and the priority of the lesions in the partition, match the sequence priority rules of the knowledge graph of the ankylosing spondylitis auxiliary analysis, and output candidate examination actions that include sequence type, target anatomical partition, and slice level range. The candidate examination action optimization unit is used to constrain, score, and correct the candidate examination actions based on the dual memory bank state and the candidate examination actions, and to eliminate actions that do not conform to clinical norms. The second output unit is used to output the next check action of the current dual memory state to obtain the structured auxiliary analysis information.

[0036] In this embodiment, the auxiliary analysis convergence and output module 40 includes: The auxiliary analysis report generation unit is used to integrate and generate the structured radiological auxiliary analysis report based on the ankylosing spondylitis imaging detection standards, using all the structured auxiliary analysis information in the dual memory bank. The dual memory bank includes: an effective memory bank for storing positive signs, high-confidence lesion areas, and key evidence slices, and an exclusion memory bank for storing areas confirmed to be without abnormalities. The output and visualization unit outputs the structured radiological analysis report and the corresponding visualization results.

[0037] This embodiment achieves the following technical effects through the above technical solution: 1) In this embodiment, the system reconstructs the image analysis process into three stages: X-ray initial positioning → MRI directional iterative screening → convergence and output of analysis information. The system achieves efficient storage and reuse of auxiliary analysis information through a structured dual memory library, thereby solving the problem of long sequence context exceeding the limit. 2) In this embodiment, the system introduces a knowledge graph of ankylosing spondylitis sacroiliac joint auxiliary analysis to constrain and score the agent's decision-making, ensuring that the decision-making conforms to the specifications; 3) The system in this embodiment can dynamically select the MRI sequence type, anatomical region and slice range based on the results of the preceding auxiliary analysis, realizing lightweight, directional and interpretable iterative auxiliary analysis; 4) The system in this embodiment can significantly reduce the amount of slice processing, improve the efficiency and accuracy of auxiliary analysis, and make the decision-making process traceable and visualized.

[0038] Exemplary methods Based on the above embodiments, the present invention also provides an image processing method for a dynamic interactive sacroiliac joint image-assisted analysis system based on a simulated clinical image reading path.

[0039] like Figure 2 As shown, the image processing method includes the following steps: Step S100: Acquire the captured X-ray image data and MRI image data.

[0040] In this embodiment, the image analysis process is reconstructed into three stages: initial X-ray localization → MRI-guided iterative screening → convergence and output of analysis information. A structured dual-memory library is used to achieve efficient storage and reuse of auxiliary analysis information, solving the problem of long sequence context exceeding limits. An ankylosing spondylitis (AS) sacroiliac joint auxiliary analysis knowledge graph is introduced to constrain and score the agent's decision-making, ensuring that the decision conforms to clinical standards. The system dynamically selects the MRI sequence type, anatomical partition, and slice range based on the results of the preceding auxiliary analysis, realizing lightweight, directional, and interpretable iterative auxiliary analysis.

[0041] Specifically, in one implementation of this embodiment, step S100 includes the following steps: Step S101: Acquire X-ray image data of the pelvis in the anteroposterior position and MRI image data for anatomical spatial alignment and zonal localization. Step S102: Use a medical image annotation tool to annotate the X-ray image data and the MRI image data, and then filter the annotated image data to obtain the filtered X-ray image data and MRI image data.

[0042] like Figure 3 As shown in this embodiment, the overall process of the dynamic interactive sacroiliac joint image-assisted analysis method simulating a clinical image reading path includes: Phase 1: Initial X-ray positioning.

[0043] This stage of input includes two types of image data: 1) X-ray imaging data (i.e., sacroiliac joint X-ray): routinely taken anteroposterior X-ray images of the pelvis, used to observe macroscopic lesions such as the overall bone structure and interspace changes of the sacroiliac joint; 2) MRI image data (i.e. MRI localization images): Low-resolution, large-area panoramic images acquired before the formal MRI scan. They are only used for anatomical spatial alignment and zonal localization. They are not used for lesion auxiliary analysis and feature extraction, and no lesion identification or auxiliary analysis output is performed. They provide scanning orientation, range and slice angle reference for subsequent fine sequences.

[0044] Specifically, the dataset construction and annotation process is as follows: First, X-ray and MRI images of patients with ankylosing spondylitis (AS), suspected AS patients, and healthy controls were collected. MRI images included STIR, T1WI, and T2WI sequences, all specific sequences used in MRI. Corresponding clinical auxiliary analysis reports were also collected. For suspected AS patients, a standardized 6-zone anatomical labeling system, identical to that used for confirmed AS patients, was applied to label multiple zones of the sacroiliac joint. For the collected X-ray and MRI images, only the case auxiliary analysis label type was differentiated; the anatomical zone labeling logic and format remained unchanged. Suspicious abnormalities in the images were simultaneously labeled to ensure the model consistently learns the anatomical spatial structure of the sacroiliac joint. Then, after excluding cases with substandard image quality (such as severe artifacts or incomplete scanning range) and unclear clinical auxiliary analysis, a labeled dataset containing ≥1000 samples was finally constructed and divided into training set, validation set and test set in an 8:1:1 ratio.

[0045] In the above dataset construction and annotation process, the specific annotation specifications and categories for the collected X-ray and MRI image data are as follows: 1) Anatomical region annotation: The sacroiliac joint is annotated at the pixel level in 6 anatomical regions, including the upper left, middle left, lower left, upper right, middle right, lower right, and subchondral region (with supplementary annotation of the subchondral bone boundary). The annotation content includes the semantic segmentation mask, target detection box (bbox), and region ID for each region. 2) Lesion feature annotation: For X-ray and MRI image data, annotate the radiological features related to AS, including: Radiographic features include: osteosclerosis, joint space narrowing, bone erosion, joint stiffness, and bone hyperplasia. MRI pathological features include bone marrow edema, fatty infiltration, bone erosion, bone sclerosis, synovitis, and tendonitis; the annotation includes pixel-level mask of the lesion area, detection box, lesion type, severity (0-3 grade), and confidence level; 3) Semantic association annotation: Annotate the correspondence between lesions and anatomical regions, the continuity of lesions in different sequences / levels, and clinical imaging assessment conclusions (AS staging, grading).

[0046] Specifically, in the semantic association annotation method mentioned above, the correspondence between lesions and anatomical regions is as follows: it is clear that features such as osteosclerosis and joint space narrowing are concentrated in the lower and middle parts of both sides and the subchondral region, bone marrow edema preferentially affects the lower and middle parts, and bone erosion and joint ankylosis are mostly distributed in the entire articular surface and the lower and middle joint region. Each lesion is marked with its precise anatomical region, realizing one-to-one binding of spatial location.

[0047] In the semantic association annotation method described above, the continuity of lesions in different sequences / levels includes: cross-sequence continuity (correspondence of signal features of lesions in the same partition in STIR / T2WI / T1WI sequences), cross-level continuity (the spatial location of the same lesion in adjacent slices is consistent with the partition classification), and disease course temporal continuity (the evolution of lesions from mild to severe, from active to chronic in follow-up images of the same patient). When annotating, the same lesion ID is associated to ensure one-to-one correspondence between lesions across levels and sequences.

[0048] As an example, the multi-sequence linkage annotation method is as follows: the lower right bone marrow edema lesion and segmentation mask are annotated in the STIR sequence, and the T2WI and T1WI sequences at the same level are simultaneously linked and aligned to match the spatial location of the same lesion, supplement the signal characteristics and severity of each sequence, and unify the lesion numbering across adjacent levels to avoid duplicate annotation and omission.

[0049] The labeling method for suspected AS patients is as follows: a standardized 6-zone labeling system that is completely consistent with that for confirmed cases is adopted, only abnormal signs in imaging are labeled, and the imaging assessment label is separately marked as a suspected case, so as not to be confused with confirmed positive or healthy negative samples, ensuring that the model learns the anatomical zone structure rather than simply the classification label.

[0050] In the actual semantic annotation process of this embodiment, medical image annotation tools such as LabelMe and ITK-SNAP can be used for annotation. These tools support DICOM format (international standard format for medical images) image reading, multi-sequence linkage annotation, and JSON format (a lightweight data exchange format) annotation result export.

[0051] The specific method of multi-sequence linkage annotation is as follows: First, complete the localization and segmentation mask annotation of single lesions in STIR sequences, automatically spatially register and align with T1WI and T2WI sequences at the same level, synchronously match the anatomical region to which the same lesion belongs, and supplement the signal characteristics and lesion severity of each sequence; at the same time, bind the unique ID of the same lesion across adjacent continuous levels to unify the classification of lesions across layers and sequences, and avoid duplicate annotation, omission annotation and spatial misalignment; after the first round of annotation is completed, perform annotation consistency test (Cohen's kappa, for example, test coefficient ≥ 0.85), review and correct inconsistent annotations, and ensure that the annotation quality meets the requirements of model training.

[0052] This embodiment obtains X-ray and MRI image data for fine-tuning the visual language model and subsequent analysis through the above-described dataset construction and annotation process. Based on these image data, the model outputs JSON structured text containing zoning and lesion information, and also outputs JSON structured lesion information corresponding to the MRI slices.

[0053] like Figure 2 As shown, the image processing method further includes the following steps: Step S200: Based on the first visual language model, feature extraction and zoning localization processing are performed on the X-ray image data and the MRI image data to obtain a sacroiliac joint zoning localization map.

[0054] In this embodiment, the Qwen3-VL-8B visual language model (VLM model) is selected to perform feature extraction and zonal localization processing on the X-ray image data and the MRI image data, thereby obtaining a sacroiliac joint zonal localization map.

[0055] The reason for choosing the Qwen3-VL-8B visual language model in this embodiment is as follows: 1) Unified multi-task capability: Natively supports parallel multi-tasks such as object detection, semantic segmentation, and image and text understanding, eliminating the need to build multiple independent models (such as U-Net segmentation model + YOLO detection model), simplifying system architecture and reducing deployment complexity; 2) Open vocabulary generalization ability: Based on large-scale general image and text data pre-training, it can flexibly identify diverse radiological signs related to AS, without the need to train a separate model for each lesion category, and adapt to the complexity of clinical lesion manifestations. 3) High domain adaptation efficiency: Only a small amount of AS-specific labeled data is needed to complete domain fine-tuning, which greatly reduces data labeling costs and training cycle compared to conventional segmentation models trained from scratch; 4) Structured output capability: It can directly generate structured results in JSON format that conform to clinical standards, without the need for additional post-processing modules, and can directly connect to downstream iterative auxiliary analysis processes.

[0056] Table 1. Comparison of the selected model with conventional image segmentation models (such as U-Net and Mask R-CNN):

[0057] In this embodiment, for the selected Qwen3-VL-8B visual language model, a parameter efficient fine-tuning (PEFT) + multi-task joint training strategy is adopted. The specific process is as follows: 1) Model initialization: Load the complete pre-trained weights of the Qwen3-VL-8B visual language model, including the original parameters of the three core modules: visual encoder, image-text projection alignment layer, and text decoder; freeze the global network of the model backbone, do not change the original general pre-trained parameters, and only fine-tune a few parameters of the visual encoder ViT attention layer (inserting LoRA low-rank adapter), image-text projection alignment layer, and top-level text decoder to achieve medical field adaptation without destroying the model's general capabilities, and significantly reduce the training computational requirements.

[0058] 2) Fine-tuning the dataset construction: Convert the labeled data into image-text pairing samples, in the following format: Input: Image + prompt (prompt word): "Please identify the 6 anatomical regions of this sacroiliac joint image, mark the detection box and segmentation mask of each region, and identify all AS-related lesions, marking the lesion type, location, and severity"; Output: JSON structured text containing partition information and lesion information.

[0059] 3) Training strategy: Training is performed using a deep learning server, with parameters adjusted according to the actual hardware configuration. 4) Model Validation: Evaluate model performance on the test set, using metrics including: Segmentation metrics: mIoU (mean intersection-union ratio), Dice coefficient (Dice similarity coefficient); Detection metrics: mAP (mean precision), recall; Structured output accuracy: lesion type identification accuracy, zonal location accuracy.

[0060] In this embodiment, the selected Qwen3-VL-8B visual language model is fine-tuned in the manner described above to obtain a first visual language model after specific fine-tuning, thereby using the first visual language model to perform the first stage of X-ray initial positioning process.

[0061] Specifically, in one implementation of this embodiment, step S200 includes the following steps: Step S201: Input the X-ray image data and the MRI image data into the first visual language model; wherein, the first visual language model is a fine-tuned model; Step S202: Identify the radiological lesion features and their corresponding location, type, severity, confidence level, and anatomical region; wherein, the radiological lesion features include: bone sclerosis features, joint space narrowing features, and bone erosion features; Step S203: Locate the entire area of ​​the sacroiliac joint through target detection, and automatically segment it into multiple anatomical regions according to the anatomical structure to obtain the detection box and pixel-level semantic segmentation mask for each anatomical region. Step S204: Based on the correspondence between the identified radiological lesion features and the located anatomical regions, mark the anatomical regions with abnormalities, score them according to the priority scoring rules, sort them from high to low according to the comprehensive score, and generate a priority list of suspicious abnormal regions. Step S205: Output the sacroiliac joint zoning map and the priority list of the suspected abnormal zoning areas.

[0062] In this embodiment, the processing flow of the first visual language model is as follows: 1) Feature extraction and lesion identification: After specialized fine-tuning, the model can automatically identify radiological lesions in X-ray image data. For example, osteosclerosis is manifested as high-density thickening and whitening of the articular surface, joint space narrowing is manifested as uneven reduction of the distance between the two articular surfaces, and bone erosion is manifested as worm-eaten defects and discontinuity of bone on the articular surface. The model outputs the location, type, severity, confidence level and anatomical region to which each lesion belongs.

[0063] 2) Anatomical region localization and division: Based on the labeled 6-region data, the model first locates the entire area of ​​the sacroiliac joint through target detection, and then automatically divides it into 6 anatomical regions according to the anatomical structure: left / right upper / right middle / right lower part and subchondral region. The model outputs the detection box (bbox) of each region and the pixel-level semantic segmentation mask to achieve accurate spatial boundary division.

[0064] 3) Abnormal area marking and priority ranking: Based on the correspondence between lesions and zonal areas, anatomical zonal areas with abnormalities are marked; the priority scoring rule is: comprehensive score = lesion severity weight × sequence signal specificity × confidence level, where edema has the highest weight in STIR sequences and the lower zonal area has a higher weight than the middle and upper zonal areas; sort by comprehensive score from high to low to generate a priority list of suspicious abnormal zonal areas, providing a reference range for subsequent MRI iterative examinations.

[0065] After the above model processing, the initial X-ray positioning stage outputs three types of results, all stored in a standardized format for use in downstream processes: Sacroiliac joint zonal localization map: Visualizes the detection boxes and segmentation masks of 6 anatomical regions, and overlays suspected abnormal area markers for clinicians to review; List of suspected abnormal partitions: includes the ID, name, lesion type, severity, confidence level, and corresponding imaging slice for each abnormal partition; JSON structured text: Completely stores all location and lesion information, as shown in the example below: json { "report_id": "AS_20260407_001", "patient_id": "PAT_20260407_001", "imaging_type": ["X-ray", "MRI_localizer"], "anatomy_regions": [ { "region_id": "R_UPPER", "region_name": "Top Right Section", "bbox": [120, 200, 300, 450], "mask_path": ". / mask / R_UPPER_AS_20260407_001.png", "is_abnormal": true, "abnormal_score": 0.89 }, { "region_id": "R_MIDDLE", "region_name": "Right Middle Section", "bbox": [120, 450, 300, 700], "mask_path": ". / mask / R_MIDDLE_AS_20260407_001.png", "is_abnormal": true, "abnormal_score": 0.75 } { "region_id": "R_LOWER", "region_name": "Lower Right Section", "bbox": [120, 700, 300, 950], "mask_path": ". / mask / R_LOWER_AS_20260407_001.png", "is_abnormal": false, "abnormal_score": 0.12 } { "region_id": "L_UPPER", "region_name": "Left Upper Part", "bbox": [500, 200, 680, 450], "mask_path": ". / mask / L_UPPER_AS_20260407_001.png", "is_abnormal": true, "abnormal_score": 0.72 } { "region_id": "L_MIDDLE", "region_name": "Left Middle Part", "bbox": [500, 450, 680, 700], "mask_path": ". / mask / L_MIDDLE_AS_20260407_001.png", "is_abnormal": false, "abnormal_score": 0.23 } { "region_id": "L_LOWER", "region_name": "Left Lower Part", "bbox": [500, 700, 680, 950], "mask_path": ". / mask / L_LOWER_AS_20260407_001.png", "is_abnormal": false, "abnormal_score": 0.18} ], "lesions": [ { "lesion_id": "L_001", "lesion_type": "sclerosis", "lesion_name": "Osteosclerosis", "region_id": "R_UPPER", "severity": 2, "confidence": 0.92, "bbox": [150, 220, 280, 430], "mask_path": ". / mask / L_001_AS_20260407_001.png" } ], "abnormal_regions_list": ["R_UPPER", "R_MIDDLE", "L_UPPER"], "stage1_result": "completed"} In this embodiment, the initial X-ray localization process of stage one is implemented in the manner described above, and the sacroiliac joint zonal localization map and the list of suspected abnormal zonal regions are stored in JSON structured text for subsequent MRI-guided iterative auxiliary analysis in stage two.

[0066] like Figure 2 As shown, the image processing method further includes the following steps: Step S300: Based on the second visual language model, feature extraction is performed on the MRI image data to obtain sacroiliac joint features and corresponding attribute information; wherein, the attribute information includes: location, range, degree and confidence level.

[0067] like Figure 4 As shown, in this embodiment, Phase Two (MRI-guided iterative assisted analysis) is achieved through the collaborative work of three core modules: the perception module, the decision module, and the memory module, realizing a closed-loop process of "directional inspection - feature extraction - decision iteration - memory update". It is worth mentioning that Phase Two is a dynamic iterative process, which continues until the termination condition is met before entering Phase Three to output the results.

[0068] Perception module: The input is a single MRI slice of a single sequence (e.g., the 15th slice of the STIR sequence or the 20th slice of the T1WI sequence), ensuring that the model focuses on the details of a single image and avoids redundancy of information from multiple slices.

[0069] In the perception module, the Qwen3-VL-8B visual language model, which is of the same origin as the stage but is independently fine-tuned, is used. After specific fine-tuning, the second visual language model is obtained.

[0070] The difference between the fine-tuning in Phase 1 and Phase 2 is as follows: Phase 1 model fine-tunes macroscopic bone structure features based on MRI sequences in X-ray and MRI image data; while Phase 2's perception module uses a dedicated labeled dataset for single slices of MRI multi-sequences, focusing on microscopic soft tissue pathological signs such as bone marrow edema, fat infiltration, and cartilage damage. It independently fine-tunes the visual branch through LoRA (a parameter-efficient large model fine-tuning technique), learns subtle signal differences between STIR, T2WI, and T1WI sequences, and achieves accurate extraction of single-slice pathological features.

[0071] Module positioning: It is only responsible for image feature extraction and structured output, and does not participate in any decision-making, update memory, or judge positive or negative results. It only provides objective and structured imaging feature evidence for the decision-making module.

[0072] Specifically, in one implementation of this embodiment, step S300 includes the following steps: Step S301: Input a single MRI slice from the MRI image data into the second visual language model; wherein, the second visual language model is a model that is derived from the first visual language model but is independently fine-tuned; Step S302: Extract MRI pathological features based on MRI fine-tuning knowledge, and output the corresponding lesion signals, extent, grade, and involved regions layer by layer to obtain the sacroiliac joint features and corresponding attribute information; The MRI pathological features include: bone marrow edema, fatty infiltration, bone sclerosis, bone erosion, synovitis, and tendonitis.

[0073] In this embodiment, the processing flow of the perception module in stage two is as follows: A single MRI slice from the MRI imaging data is input into the finely tuned Qwen3-VL-8B model. Based on MRI-specific fine-tuning knowledge, the model extracts pathological features: identifying high-signal bone marrow edema in STIR sequences, verifying edema and synovial inflammation in T2WI sequences, and identifying fatty infiltration, bone sclerosis, and erosion in T1WI sequences. Standardized pathological features, such as lesion signal, extent, grade, and affected areas, are output layer by layer as objective auxiliary analytical evidence. After extracting pathological features, we obtained characteristics such as bone marrow edema (high signal on STIR sequence), fatty infiltration (high signal on T1WI), bone sclerosis, bone erosion, synovitis, and tendonitis. For each feature, output the lesion location, extent, severity, and confidence level.

[0074] As an example, the output is: One slice corresponds to one JSON structured lesion information, with fields including: json { "slice_id": "STIR_15", "sequence_type": "STIR", "slice_index": 15, "region_id": "R_UPPER", "lesions": [ { "lesion_type": "edema", "lesion_name": "bone marrow edema", "severity": 2, "confidence": 0.94, "bbox": [160, 230, 270, 420], "mask_path": ". / mask / STIR_15_edema.png" } ], "positive_flag": true, "perception_time": "2026-04-07 10:30:00"} Table 2. Differences between the perception module and the Phase 1 module:

[0075] It is worth mentioning that the two modules are not the same functional unit. They only reuse the Qwen3-VL-8B model architecture and are completely independent in terms of input, task, features, and output. They are the two core links in the "global coarse localization → local fine auxiliary analysis" process, which are connected and complementary.

[0076] like Figure 2 As shown, the image processing method further includes the following steps: Step S400: Based on the sacroiliac joint zoning map, the sacroiliac joint features, and the corresponding attribute information, structured auxiliary analysis information is generated using a decision module that combines a large language model and a knowledge graph.

[0077] In this embodiment, the decision-making module is used to make decisions based on the outputs of the Phase 1 module and the Phase 2 perception module, generating structured auxiliary analysis information.

[0078] The input to the decision-making module in Phase Two of this embodiment includes two types of core information: The current state space consists of the checked partitions, unchecked partitions, positive lesion partitions, negative normal partitions, current iteration round, current scan sequence, current slice level, effective memory snapshot, and excluded memory snapshot. The state space is fully updated immediately after each round of the perception module returns the results, serving as the basis for the next decision.

[0079] AS Sacroiliac Joint Imaging Analysis Knowledge Graph Rules: Includes clinical guidelines for AS auxiliary analysis, sequence priority, examination order of zones, sign verification rules, convergence termination rules, etc.

[0080] The decision module (LLM model + knowledge graph, i.e., large language model and knowledge graph) uniquely determines the specific slice to be input to the perception module. The complete process is as follows: 1) The decision module reads the list of abnormal partition priorities output in Phase 1: Dual memory bank status acquisition and generation: After each round of sensory module outputs slice auxiliary analysis results, the corresponding partitions and features are automatically assigned to the effective memory bank according to the positive high confidence rule, or the negative normal areas are assigned to the exclusion memory bank. The partitions, lesion types, and sign confidence information in the memory bank are updated in real time to form the current full dual memory bank snapshot status. Candidate examination action generation: Combining the current iteration round, the information of the double memory that has been investigated / diagnosed, and the priority of the lesions in the region, the sequence priority rules of the AS-assisted analysis knowledge graph are matched to output standardized candidate examination actions in an orderly manner, including sequence type, target anatomical region, and precise slice level range.

[0081] 2) Constrain, score, and correct candidate inspection actions based on AS-assisted knowledge graph analysis: Sequence priority constraints: STIR sequences are preferred (for optimal visualization of edema), followed by T1WI sequences (for visualization of fat infiltration and bone structure). Partition order constraint: Prioritize checking high-confidence anomalous partitions marked in Phase 1, and then supplement with checking negative partitions; Layer range constraint: For the target partition, select the layer corresponding to the anatomical location to avoid checking irrelevant layers; 3) Generate the final inspection instruction, specifying "which sequence, which partition, and which slice layer", and send it to the perception module to perform feature extraction; 4) After the perception module returns the result, the decision module updates the state space and enters the next iteration; iteration stops when any of the following termination conditions are met: First item: All six key anatomical regions of the sacroiliac joint underwent multi-sequence full-coverage examination; Second: No new positive lesion signs were found in two consecutive iterations, and the auxiliary analysis features converged completely; Thirdly: All positive abnormal regions underwent cross-validation using STIR, T2WI, and T1WI sequences; The fourth item: After reaching the preset maximum safe iteration rounds (default 5 rounds), terminate the loop to avoid invalid repeated scans.

[0082] In this embodiment, the decision module adopts the Qwen-7B Large Language Model (LLM), which requires Lightweight Instruction Fine-tuning (SFT). The specific solution is as follows: 1) Fine-tuning Dataset Construction: Construct a dedicated instruction dataset for AS-assisted analysis, with the following sample format: Input: Current state: Partitions checked [R_UPPER, R_MIDDLE], valid memory [R_UPPER edema level 2], excluded memory [L_LOWER normal], iteration round 2; Knowledge graph rules: STIR sequence priority, abnormal partition priority; Output: Next action: (STIR sequence, L_UPPER partition, layers 14-16, continue iteration); 2) Fine-tuning strategy: The LoRA low-rank adapter parameter high-efficiency fine-tuning strategy is adopted. The weights of the Qwen-7B model backbone network are frozen throughout the process and do not participate in the update. Only the low-rank adapter matrix is ​​added to the model attention layer and a small number of parameters are trained on the instruction output adaptation layer. Without destroying the model's general language ability, the number of trainable parameters is greatly reduced, and the memory usage and training computing power consumption are significantly reduced. 3) Fine-tuning objectives: Enable the LLM model to learn the AS clinical auxiliary analysis logic, generate decision instructions that conform to knowledge graph rules and fit clinical reading habits, reduce rule modification costs, and improve decision accuracy; 4) Knowledge Graph Collaboration: After the fine-tuned LLM model generates candidate examination actions, it still needs to be verified a second time through the knowledge graph to ensure that the decision is 100% in line with clinical standards and to avoid model illusion.

[0083] Specifically, in one implementation of this embodiment, step S400 includes the following steps: Step S401: Based on the sacroiliac joint zoning map, the priority list of suspected abnormal zoning areas, the sacroiliac joint features, and the corresponding attribute information, generate a dual memory state and candidate examination actions.

[0084] In one implementation of this embodiment, step S401 includes the following steps: Step S401a: Based on the sacroiliac joint zoning map, the priority list of suspected abnormal zoning areas, the sacroiliac joint features and corresponding attribute information, the corresponding zoning areas and features are automatically assigned to the effective memory bank according to the positive high confidence rule, or the negative normal areas are assigned to the exclusion memory bank, and the zoning areas, lesion types, and sign confidence information in the bank are updated in real time to generate the dual memory bank status. Step S401b: Combining the current iteration round, the information of the double memory database that has been investigated / diagnosed, and the priority of the lesions in the partition, the sequence priority rules of the knowledge graph of ankylosing spondylitis auxiliary analysis are matched, and candidate examination actions containing sequence type, target anatomical partition, and slice level range are output.

[0085] In one implementation of this embodiment, step S400 further includes the following steps: Step S402: Based on the dual memory bank state and the candidate examination actions, the ankylosing spondylitis auxiliary analysis knowledge graph is used to constrain, score and correct the candidate examination actions, and actions that do not conform to clinical norms are eliminated. Step S403: Output the next check action for the current dual memory state to obtain the structured auxiliary analysis information.

[0086] In this embodiment, after specific fine-tuning, the processing flow of the Qwen-7B large language model is as follows: 1) The LLM model generates candidate auxiliary analysis actions (sequence, partition, level, termination) based on the current state space; 2) Based on the AS-assisted analysis knowledge graph, candidate examination actions are constrained, scored, and corrected to eliminate actions that do not conform to clinical standards; 3) Output the final next action in the format of a quadruple: sequence type, anatomical region, slice range, termination marker; 4) Receive the feature results returned by the perception module, update the state space and dual memory, and enter the next iteration.

[0087] As an example, the standardized quadruple analysis instruction for the final next action is: (STIR sequence, upper left partition, layers 14-16, continue iteration); (T1WI sequence, upper right partition, layers 15-17, continuing iteration); (None, none, none, terminate iteration).

[0088] Specifically, in the above example, the auxiliary analysis action space (quadruple) is represented as follows: Each decision-making step outputs a fixed structure: (sequence type, anatomical region, slice range, termination marker). Sequence type: STIR sequence > T2WI sequence > T1WI sequence (clinical priority, defined by knowledge graph); Anatomical regions: left upper / middle / lower, right upper / middle / lower, subchondral region (6 regions in total); Layer range: The start and end indexes of the slice corresponding to the partition, such as "layer 12-18"; Termination flag: True / False, determined by the knowledge graph convergence rules.

[0089] Specifically, in the above example, the structured memory unit (field details + example) is as follows: Table 3. Each memory entry contains fixed fields, explained in detail below:

[0090] Complete JSON example: json { "round_id": 3, "sequence_type": "STIR", "anatomy_region": "Lower part of the right sacroiliac joint", "slice_range": "Levels 12-18", "lesion_type": "bone marrow edema", "severity": "moderate" "confidence": 0.94, "is_positive": true, "timestamp": "2026-04-07 10:30:00", "is_conflict": false, "slice_path": " / data / MRI / STIR_15.png", "mask_path": " / data / mask / STIR_15_mask.png"}.

[0091] Specifically, in the above example, the dual-memory mechanism used is as follows: 1) Effective memory bank: storing positive signs, high-confidence lesion areas, and key evidence slides; Inclusion criteria: lesion confidence ≥ 0.8; the same lesion validated on 2 or more sequences / slices; meets the ASAS imaging assessment criteria (a standard developed by the International Association for the Assessment of Spondyloarthritis). The role of an effective evidence storage bank: to store key evidence to aid in analysis, for use in generating the final report, tracing the chain of evidence, and avoiding duplicate judgments.

[0092] 2) Exclude memory: Store areas that have been confirmed to be free of anomalies; Inclusion criteria: Partition / sequence has been checked and no positive signs are found; the confidence level of the check is ≥0.9; and it meets the clinical negative judgment criteria.

[0093] The purpose of excluding the memory bank is to avoid redundant checks, compress the context length, and improve reasoning efficiency.

[0094] The management logic of the dual memory is as follows: the dual memory is automatically updated after each iteration, and deduplication, conflict detection and confidence verification are performed. The valid memory is stored first, and the excluded memory is only used for filtering duplicate checks. When the same sequence in the same partition has a judgment conflict, the result of the high confidence (>0.9) + high priority sequence (STIR sequence) shall prevail.

[0095] Specifically, in the example above, the AS-assisted analysis knowledge graph constraint rules are as follows: Knowledge graphs can be implemented using rule tables, decision trees, JSON configuration files, ontology libraries, etc., all of which are equivalent implementation solutions.

[0096] 1) Knowledge graph entities, relations, and attributes: Entities: Imaging entities (X-ray and MRI data), anatomical entities (sacroiliac joint, regions), lesion entities (edema / sclerosis / erosion, etc.), auxiliary analysis entities (AS, sacroiliitis, grading), and rule entities (sequence priority, convergence criteria).

[0097] Relationships: sequence-partition correspondence, lesion-partition prevalence, lesion-sequence signal relationship, partition-adjacency relationship, lesion-auxiliary analysis association.

[0098] Attributes: sequence priority (STIR=3, T2=2, T1=1), lesion severity, confidence level, zonal examination status, ASAS score.

[0099] 2) Core constraint rules: Sequence-partition matching rule: If a partition is STIR negative, the priority of T2 / T1 testing is reduced by 50%; Lesion expansion rule: If a certain region is positive, the examination priority of adjacent regions is increased by 100%; Verification rules for signs: Edema requires verification from at least 2 sequences / 2 layers; otherwise, it will not be written into the valid memory. Conflict handling rules: When there is a conflict between the same partition and the same sequence, the result of the higher confidence (>0.9) and higher priority sequence (STIR) shall prevail; Convergence termination rule: The iteration terminates when there are no new positive signs for two consecutive rounds, or when all partitions have been checked.

[0100] It is worth mentioning that the above-mentioned structured auxiliary analysis information only provides medical image auxiliary analysis and does not directly diagnose diseases. The final output is an objective description of imaging features, zonal location information, and a formatted report that conforms to clinical standards.

[0101] like Figure 2 As shown, the image processing method further includes the following steps: Step S500: Integrate all the structured auxiliary analysis information, generate and output a structured radiological auxiliary analysis report and visualization results.

[0102] In this embodiment, stage three is the auxiliary analysis convergence and output. This stage integrates all structured auxiliary analysis information in the effective memory bank and the excluded memory bank. Based on the ASAS image evaluation standard, all information is integrated to generate a structured auxiliary analysis report for subsequent auxiliary analysis.

[0103] Specifically, in one implementation of this embodiment, step S500 includes the following steps: Step S501: For all the structured auxiliary analysis information in the dual memory bank, the structured radiological auxiliary analysis report is integrated and generated according to the ankylosing spondylitis imaging detection standard; wherein, the dual memory bank includes: an effective memory bank for storing positive signs, high-confidence lesion areas and key evidence slices, and an exclusion memory bank for storing areas confirmed to be without abnormalities; Step S502: Output the structured radiological analysis report and the corresponding visualization results.

[0104] In this embodiment, the generated structured radiological analysis report and visualization results are as follows: The output includes: a summary of imaging assessment, an assessment reference for AS grading, a map of the sacroiliac joint zonation, a chain of evidence for the core lesions, and an auxiliary analysis path for each iterative step; In the sacroiliac joint zoning diagram, the colors are defined as follows: Red: Positive lesion; Yellow: Suspected lesion; Green: Negative result, normal; Gray: Not checked.

[0105] The JSON structured output of Phase 3 is as follows: { "report_id": "AS_20260407_001", "final_assessment": { "summary": "Visible positive signs", "as_grade_reference": "Level II (Recommended based on comprehensive clinical assessment)", "confidence": 0.92 }, "positive_regions":["R_UPPER", "R_MIDDLE"], "evidence_chain":["round1_STIR_R_UPPER_edema","round2_T1WI_R_UPPER_sclerosis"] }

[0106] Through the technical solutions described above in the three stages, this embodiment is expected to reduce the amount of slice processing by 70% to 85%, significantly improving efficiency. The context is compressed from thousands of tokens to hundreds of tokens, breaking through the length limit of the VLM model. Furthermore, the decision-making is fully aligned with the clinical pathway, making it understandable and traceable for doctors. The knowledge graph ensures the compliance of auxiliary analysis and reduces the risk of misjudgment. The structured output can be directly used for medical records, scientific research, and follow-up.

[0107] Those skilled in the art will understand that the technical solution of this embodiment can be improved or replaced in the following ways: 1) Change in the number of stages: The three stages will be merged into two stages or split into four stages, while the core iteration logic remains unchanged.

[0108] 2) Changes in storage memory format: Different storage formats such as single storage memory, triple storage memory, and database / cachate are adopted, while the structured storage and retrieval logic remain unchanged.

[0109] 3) Knowledge graph implementation changes: Replace the knowledge graph with rule tables, decision trees, and condition configuration files to implement clinical constraint functions.

[0110] 4) Model backbone replacement: Use different networks such as CNN model, Transformer model, VLM model, and hybrid model to implement perception / decision, while keeping the process and framework unchanged.

[0111] 5) Changes to the division of the anatomical partitions: The number, naming and range of partitions have been adjusted, but the "partition iterative screening" approach is still adopted.

[0112] 6) Change of termination conditions: Adjust the round, threshold and convergence rules, but the overall iteration termination mechanism remains unchanged.

[0113] 7) Application scenario expansion: The framework can be used for image-assisted analysis of other bone and joint conditions such as rheumatoid arthritis, spondyloarthritis, and hip arthritis.

[0114] Any improvements or modifications made based on the above description shall fall within the protection scope of the embodiments of the present invention.

[0115] This embodiment achieves the following technical effects through the above technical solution: 1) This embodiment abandons static full input and adopts the process of "X-ray preliminary judgment → MRI directional iteration → analysis information convergence output", which can automatically focus on key areas step by step according to clinical logic; and, it stores structured auxiliary analysis information in the form of effective memory bank / exclusion memory bank, solves the problem of long sequence MRI context exceeding the limit, and realizes information reusability and traceability.

[0116] 2) This embodiment uses the AS sacroiliac joint auxiliary analysis knowledge graph constraint model action selection, which can ensure that the decision-making conforms to the clinical pathway and auxiliary analysis specifications, improve interpretability and safety, and achieve "knowable, credible and verifiable" by providing clinically interpretable output partition state diagram, evidence chain and decision basis for each step.

[0117] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include both non-volatile and volatile memory.

[0118] In summary, this invention provides a dynamic interactive sacroiliac joint image-assisted analysis system and image processing method that simulates a clinical image reading path. The system includes: an image data acquisition and preprocessing module for acquiring and preprocessing X-ray and MRI image data; an X-ray initial localization module for feature extraction and zonal localization of X-ray and MRI image data based on a first visual language model; an MRI-oriented iterative auxiliary analysis module for feature extraction of MRI image data based on a second visual language model; a decision module combining a large language model and a knowledge graph to generate structured auxiliary analysis information; and an auxiliary analysis convergence and output module for integrating the auxiliary analysis information and generating and outputting a structured radiological auxiliary analysis report and visualization results. This invention can significantly reduce the amount of slide processing and improve the efficiency, accuracy, and interpretability of auxiliary analysis.

[0119] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A dynamic interactive sacroiliac joint image-assisted analysis system that simulates a clinical image reading path, characterized in that, include: The image data acquisition and preprocessing module is used to acquire the captured X-ray image data and MRI image data; The X-ray image data and the MRI image data are annotated using a medical image annotation tool, and the annotated image data are then filtered to obtain the filtered X-ray image data and MRI image data. The X-ray initial positioning module is used to perform feature extraction and zonal positioning processing on the X-ray image data and the MRI image data based on the first visual language model to obtain a sacroiliac joint zonal positioning map. The MRI-guided iterative auxiliary analysis module is used to extract features from the MRI image data based on a second visual language model to obtain sacroiliac joint features and corresponding attribute information. The attribute information includes location, extent, degree, and confidence level. It also generates structured auxiliary analysis information based on the sacroiliac joint zoning map, the sacroiliac joint features, and the corresponding attribute information, using a decision module combining a large language model and a knowledge graph. The MRI-guided iterative auxiliary analysis module includes a dual-memory state generation unit, used to, based on the sacroiliac joint zoning map, a priority list of suspicious abnormal zoning areas, the sacroiliac joint features, and the corresponding attribute information, classify the corresponding zoning areas and features according to a positive high-confidence rule. Signs are automatically added to the valid memory bank, or negative normal areas are added to the exclusion memory bank, and the partitions, lesion types, and sign confidence information within the bank are updated in real time to generate a dual memory bank status. The candidate examination action generation unit is used to combine the current iteration round, the information of the dual memory banks that have been investigated / diagnosed, and the priority of the lesions in the partitions, match the sequence priority rules of the ankylosing spondylitis auxiliary analysis knowledge graph, and output candidate examination actions that include sequence type, target anatomical partition, and slice level range. The candidate examination action optimization unit is used to constrain, score, and correct the candidate examination actions based on the dual memory bank status and the candidate examination actions, using the ankylosing spondylitis auxiliary analysis knowledge graph, and eliminate actions that do not conform to clinical standards. The auxiliary analysis convergence and output module is used to integrate all structured auxiliary analysis information, generate and output structured radiological auxiliary analysis reports and visualization results; The X-ray initial positioning module includes: The first input unit is used to input the X-ray image data and the MRI image data into the first visual language model; wherein, the first visual language model is a fine-tuned model; A radiological lesion feature recognition unit is used to identify radiological lesion features and their corresponding location, type, severity, confidence level, and anatomical region; wherein, the radiological lesion features include: bone sclerosis features, joint space narrowing features, and bone erosion features; The anatomical partition unit is used to locate the entire area of ​​the sacroiliac joint through target detection and automatically divide it into multiple anatomical partitions according to the anatomical structure, obtaining the detection box and pixel-level semantic segmentation mask for each anatomical partition. An abnormal partitioning unit is used to mark anatomical partitions with abnormalities based on the correspondence between the identified radiological lesion features and the located anatomical partitions, and to score them according to the priority scoring rules, sorting them from high to low based on the comprehensive score to generate the priority list of the suspected abnormal partitions. The first output unit is used to output the sacroiliac joint zoning map and the priority list of the suspected abnormal zoning. The MRI-guided iterative analysis module further includes: The second input unit is used to input a single MRI slice from the MRI image data into the second visual language model; wherein the second visual language model is a model that is derived from the first visual language model but is independently fine-tuned; The MRI pathological feature extraction unit is used to extract MRI pathological features based on MRI fine-tuning knowledge, and output the corresponding lesion signals, extent, grade, and affected areas layer by layer to obtain the sacroiliac joint features and corresponding attribute information; wherein, the MRI pathological features include: bone marrow edema features, fat infiltration features, bone sclerosis features, bone erosion features, synovitis features, and tendonitis features. The second output unit is used to output the next check action of the current dual memory state to obtain the structured auxiliary analysis information.

2. The dynamic interactive sacroiliac joint image-assisted analysis system simulating clinical image reading path according to claim 1, characterized in that, The auxiliary analysis convergence and output module includes: The auxiliary analysis report generation unit is used to integrate and generate the structured radiological auxiliary analysis report based on the ankylosing spondylitis imaging detection standards, using all the structured auxiliary analysis information in the dual memory bank. The dual memory bank includes: an effective memory bank for storing positive signs, high-confidence lesion areas, and key evidence slices, and an exclusion memory bank for storing areas confirmed to be without abnormalities. The output and visualization unit outputs the structured radiological analysis report and the corresponding visualization results.

3. An image processing method based on the dynamic interactive sacroiliac joint image-assisted analysis system simulating clinical image reading path as described in any one of claims 1 to 2, characterized in that, include: Acquire the captured X-ray and MRI image data; Based on the first visual language model, feature extraction and zoning localization processing are performed on the X-ray image data and the MRI image data to obtain a sacroiliac joint zoning localization map. Feature extraction is performed on the MRI image data based on a second visual language model to obtain sacroiliac joint features and corresponding attribute information; wherein, the attribute information includes: location, range, degree, and confidence level; Based on the sacroiliac joint zoning map, the sacroiliac joint features, and the corresponding attribute information, a structured auxiliary analysis information is generated using a decision module that combines a large language model and a knowledge graph. All structured auxiliary analysis information is integrated to generate and output structured radiological auxiliary analysis reports and visualization results.

4. The image processing method according to claim 3, characterized in that, The acquisition of the captured X-ray and MRI image data includes: Acquire X-ray images of the pelvis in the anteroposterior position and MRI images for anatomical spatial alignment and zonal localization; The X-ray image data and the MRI image data are annotated using a medical image annotation tool, and the annotated image data are then filtered to obtain the filtered X-ray image data and MRI image data.

5. The image processing method according to claim 3, characterized in that, The process of extracting features and performing zonal localization on the X-ray and MRI image data based on a first visual language model to obtain a sacroiliac joint zonal localization map includes: The X-ray image data and the MRI image data are input into the first visual language model; wherein, the first visual language model is a fine-tuned model; Identify radiographic lesion features and their corresponding location, type, severity, confidence level, and anatomical region; wherein, the radiographic lesion features include: bone sclerosis features, joint space narrowing features, and bone erosion features; The entire area of ​​the sacroiliac joint is located by target detection and automatically segmented into multiple anatomical regions according to the anatomical structure, resulting in a detection box and pixel-level semantic segmentation mask for each anatomical region. Based on the correspondence between the identified radiological lesion features and the located anatomical regions, the anatomical regions with abnormalities are marked, and scores are assigned according to priority scoring rules. The regions are then sorted from high to low based on their comprehensive scores to generate a priority list of suspected abnormal regions. Output the sacroiliac joint zonal location map and the priority list of the suspected abnormal zonal regions.

6. The image processing method according to claim 5, characterized in that, The feature extraction of the MRI image data based on the second visual language model yields sacroiliac joint features and corresponding attribute information, including: A single MRI slice from the MRI image data is input into the second visual language model; wherein, the second visual language model is a model that is of the same origin as the first visual language model but is independently fine-tuned; Based on MRI fine-tuning knowledge, MRI pathological features are extracted, and the corresponding lesion signals, extent, grade, and involved regions are output layer by layer to obtain the sacroiliac joint features and corresponding attribute information. The MRI pathological features include: bone marrow edema, fatty infiltration, bone sclerosis, bone erosion, synovitis, and tendonitis.

7. The image processing method according to claim 6, characterized in that, The process involves generating structured auxiliary analysis information based on the sacroiliac joint localization map, sacroiliac joint features, and corresponding attribute information, using a decision module that combines a large language model and a knowledge graph. This includes: Based on the sacroiliac joint zoning map, the priority list of suspected abnormal zoning areas, the sacroiliac joint features and corresponding attribute information, the corresponding zoning areas and features are automatically classified into the effective memory bank according to the positive high confidence rule, or the negative normal areas are classified into the exclusion memory bank, and the zoning areas, lesion types, and sign confidence information in the bank are updated in real time to generate a dual memory bank status. Combining the current iteration round, the information of the double memory database that has been investigated / diagnosed, and the priority of the lesions in the partition, the sequence priority rules of the knowledge graph of the ankylosing spondylitis auxiliary analysis are matched, and candidate examination actions containing sequence type, target anatomical partition, and slice level range are output. Based on the dual memory bank state and the candidate examination actions, the ankylosing spondylitis auxiliary analysis knowledge graph is used to constrain, score, and correct the candidate examination actions, eliminating actions that do not conform to clinical norms. Output the next check action for the current dual memory state to obtain the structured auxiliary analysis information.

8. The image processing method according to claim 3, characterized in that, The process involves integrating all structured auxiliary analysis information to generate and output a structured radiological auxiliary analysis report and visualization results, including: For all the structured auxiliary analysis information in the dual memory bank, the structured radiological auxiliary analysis report is integrated and generated according to the imaging detection standards for ankylosing spondylitis; wherein, the dual memory bank includes: an effective memory bank for storing positive signs, high-confidence lesion areas and key evidence slices, and an exclusion memory bank for storing areas confirmed to be without abnormalities; Output the structured radiological analysis report and the corresponding visualization results.

Images