Image abnormal point identification method and system applied to orthopedic diagnosis and treatment

Abstract

This invention relates to the field of medical image processing technology and discloses a method and system for identifying image anomalies in orthopedic diagnosis and treatment. The method includes the following steps: multimodal image acquisition and preprocessing, multimodal image collaboration and feature alignment, anatomically constrained feature learning, hybrid supervised learning optimization, and anomaly identification and result output. The system includes a multimodal image acquisition and preprocessing module, a multimodal image collaboration and feature alignment module, an anatomical constraint knowledge base module, a hybrid supervised learning engine module, an anomaly identification core network module, and an interpretability mapping and clinical workflow interface module. To improve the accuracy and comprehensiveness of identification, this method and system acquire CT and MRI dual-modal images through an adaptive acquisition platform and perform purification and calibration. Feature-level spatial alignment and adaptive weighted fusion are achieved through a differentiable affine spatial transformation network, enabling precise capture of various anomalies such as fracture lines and osteophytes.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method and system for identifying abnormal points in images used in orthopedic diagnosis and treatment. Background Technology

[0002] With the rapid development of medical image processing technology, the demand for accurate and efficient image abnormality identification technology in the field of orthopedic diagnosis and treatment is becoming increasingly urgent. The diagnosis and treatment of orthopedic diseases are highly dependent on the results of imaging examinations. The early detection and accurate localization of abnormalities such as fracture lines, osteophytes, and bone tumors directly affect the formulation of treatment plans, surgical risk assessment, and prognosis.

[0003] Currently, CT imaging excels in displaying bone morphology and structure due to its high-resolution imaging advantage of hard bone tissue; MRI imaging, on the other hand, has unique advantages in soft tissue, bone marrow lesions, and early inflammatory infiltration. The collaborative application of dual-modal imaging has become an important development direction for precision diagnosis and treatment in orthopedics. At the same time, the widespread application of deep learning technology in tasks such as medical image segmentation and target detection provides technical support for the automated identification of abnormal points in orthopedic images, promoting the transformation of orthopedic diagnosis and treatment from traditional experience-based reliance to data-driven and precision-oriented approaches.

[0004] However, existing orthopedic image anomaly recognition technologies still have certain shortcomings: First, single-modal image recognition has inherent limitations. Relying solely on CT images makes it difficult to accurately capture soft tissue-related lesions, while relying solely on MRI images is insufficient in terms of detail at bone edges and the definition of hard tissue abnormalities. Furthermore, traditional multimodal fusion methods are mostly simple pixel-level superpositions, failing to fully consider the confidence differences of different modalities in different anatomical regions, resulting in poor fusion effects. Second, model training is highly dependent on large-scale labeled data. Orthopedic image annotation requires professional physicians to spend a significant amount of time completing the work, and high-quality labeled data is scarce. Traditional supervised learning methods have poor generalization ability in small sample scenarios and are unable to cope with complex and diverse orthopedic anatomical variations and lesion types. Third, the clinical applicability and interpretability of the recognition results are insufficient. Existing technologies mostly only output the approximate area of ​​the anomaly, lacking quantitative descriptions of abnormal parameters, anatomical rule tracing, and effective adaptation to downstream clinical workflows. This makes it difficult for physicians to directly use the recognition results in actual diagnosis and treatment processes such as surgical planning and implant design. In addition, metal artifacts, motion artifacts, and positional deviations during image acquisition also affect the accuracy and stability of anomaly recognition.

[0005] In view of this, we propose a method and system for identifying abnormal points in images for use in orthopedic diagnosis and treatment. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for identifying abnormal points in images used in orthopedic diagnosis and treatment, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A method for identifying abnormal points in images used in orthopedic diagnosis and treatment includes the following steps: S1. Multimodal image acquisition and preprocessing: Two modal image data related to orthopedics of the patient are acquired through an adaptive acquisition platform. The modal image data includes CT image sequences and MRI image sequences. The modal image data is purified to remove metal implant artifacts and motion artifacts, and standardized calibration is performed based on a parameter calibration benchmark. S2. Multimodal Image Collaboration and Feature Alignment: The preprocessed CT image sequence and MRI image sequence are input into the corresponding depth feature encoder branches respectively. Feature maps are extracted in the middle layer of the encoder. Through the differentiable affine spatial transformation network submodule, based on the bony landmark features extracted from the CT images, a non-rigid deformation field is calculated and applied to the MRI feature map to achieve feature-level spatial alignment of key anatomical regions. The aligned MRI feature map and the CT feature map of the same level are weighted and fused using an adaptive feature-gated fusion unit. The fusion weight is dynamically generated by the confidence of the two modal features. S3. Feature Learning Guided by Anatomical Constraints: Constructing a computable anatomical constraint knowledge base, which includes a statistical shape model and an anatomical relationship rule graph. The expression of the statistical shape model is as follows:

[0008] in For the generated shape, For average shape, Main component, For adjustable weights, The number of principal components is used to input the fused features into the anomaly detection core network, yielding preliminary keypoint prediction results or segmented contours. ,Will Input the anatomical constraint knowledge base, calculate the shape projection difference and rule violation penalty value to form the anatomical rationality loss, and combine the anatomical rationality loss as a regularization term with the network main task loss to perform backpropagation to train the network; S4. Hybrid supervised learning optimization: Design a self-supervised pre-training task based on anatomical context recovery, and pre-train the network using a large number of unlabeled normal bone images; fine-tune the pre-trained network on limited initial labeled data; initiate an active learning labeling iteration loop, screen samples to be labeled based on model prediction uncertainty and anatomical morphological differences, and add them to the training set after expert labeling to optimize the model; S5. Anomaly Identification and Result Output: The network is trained and optimized to infer multimodal fusion features and identify anomalies in orthopedic images. An interpretable mapping module generates anomaly region heatmaps, anatomical rule tracing information, and a deviation quantification report from the normal statistical model. The output is structured data containing the three-dimensional spatial coordinates, topological relationships, and anomaly parameters of the key points of the anomaly. The structured data is used in downstream clinical workflows.

[0009] In a further embodiment, the adaptive acquisition platform in step S1 includes a base, a lifting and adjusting bracket, a rotating platform, and an adaptive positioning fixture. The lifting and adjusting bracket is driven by a ball screw pair, and the adaptive positioning fixture is driven by a cylinder. The clamping force is adjusted in real time by feedback from a pressure sensor.

[0010] In a further embodiment, the weight generation formula for the adaptive feature gating fusion unit in step S2 is:

[0011] in For CT feature weights, For MRI feature weights, Confidence level of CT features MRI feature confidence level, in the bone edge region Higher than In soft tissue areas Higher than .

[0012] In a further embodiment, the nodes of the anatomical relationship rule diagram in step S3 are key anatomical points, including the center of the vertebral body and the midpoint of the articular surface; the edges represent anatomical relationships including adjacent, parallel, symmetrical and distance range constraints, and each edge is attached with a differentiable loss function, and the penalty value for rule violation is the weighted sum of the calculation results of the loss functions of each edge.

[0013] In a further scheme, the sample selection criteria of the active learning annotation iteration loop in step S4 satisfy the following: the entropy value of the selected sample is higher than the preset entropy threshold, and the anatomical distance with the existing training set is greater than the preset distance threshold, wherein the anatomical distance is calculated by a statistical shape model.

[0014] In a further embodiment, the abnormal points mentioned in step S5 include fracture lines, osteophytes, osteolysis, bone tumors, and early osteoarthritis lesion areas. The downstream clinical workflow includes three-dimensional reconstruction, surgical path planning, personalized implant design, and vertebroplasty simulation.

[0015] An image anomaly recognition system applied in orthopedic diagnosis and treatment includes: A multimodal image acquisition and preprocessing module is used to acquire orthopedic image data of two modalities and perform purification and standardization processing. The modal image data includes CT image sequences and MRI image sequences. The module includes an adaptive acquisition platform and a sample purification processing component. The multimodal image collaboration and feature alignment module includes two parallel deep feature encoder branches, a differentiable affine spatial transformation network submodule, and an adaptive feature gating fusion unit, which are used to realize feature extraction, spatial alignment, and weighted fusion of multimodal images. The anatomical constraint knowledge base module includes a statistical shape model construction unit and an anatomical relationship rule diagram generation unit. The statistical shape model expression is as follows:

[0016] in For the generated shape, For average shape, Main component, For adjustable weights, The number of principal components is used to provide anatomical constraints and calculate the loss of anatomical rationality. The hybrid supervised learning engine module includes a self-supervised pre-training unit and an active learning annotation iteration unit, which are used to optimize model training based on unlabeled data and limited labeled data; The core network module for anomaly recognition is used to learn features based on fused features and anatomical constraints, and output preliminary key point prediction results or segmented contours. The interpretability mapping and clinical workflow interface module is used to generate abnormal area heatmaps, anatomical rule traceability reports and structured data, and provides an interface with downstream clinical systems.

[0017] In a further embodiment, the adaptive acquisition platform includes a cast iron base, a lifting and adjusting bracket driven by a ball screw pair, a rotating platform driven by a worm gear, and an adaptive positioning fixture driven by a cylinder.

[0018] In a further embodiment, the sample purification processing component includes a switchable filter module, an artifact masking adjustment mechanism with a multi-degree-of-freedom linkage structure, and a parameter calibration reference plate. The switchable filter module has three sets of filters with different wavelengths built in, and the parameter calibration reference plate has gradient blocks and a bone simulation prosthesis of known size built in.

[0019] In a further embodiment, the interpretability mapping and clinical workflow interface module includes a feature contribution heatmap generation unit, an anatomical attribution unit, a structured data output unit, and a clinical workflow interface. The structured data output unit outputs a JSON format file.

[0020] Compared with the prior art, the present invention provides a method and system for identifying abnormal points in images used in orthopedic diagnosis and treatment, which has the following beneficial effects: 1. This method and system for identifying abnormal points in orthopedic images aims to improve the accuracy and comprehensiveness of abnormal point identification in orthopedic images. It acquires dual-modal images of CT and MRI through an adaptive acquisition platform and performs purification and calibration. Then, it achieves feature-level spatial alignment and adaptive weighted fusion through a differentiable affine spatial transformation network. This fully leverages the modal advantages of CT in the bone edge region and MRI in the soft tissue region, thereby achieving accurate capture of various abnormal points such as fracture lines and osteophytes.

[0021] 2. This image anomaly point recognition method and system applied to orthopedic diagnosis and treatment aims to reduce the dependence of model training on large-scale labeled data and improve generalization ability. It utilizes unlabeled data through self-supervised pre-training based on anatomical context recovery, and combines active learning labeling iteration loop to screen high-value samples to supplement the training set. At the same time, it introduces an anatomical constraint knowledge base to construct a rationality loss regularization network, thereby achieving continuous optimization of model recognition performance in small sample scenarios.

[0022] 3. This image anomaly point recognition method and system applied to orthopedic diagnosis and treatment, in order to enhance the clinical applicability and interpretability of the anomaly recognition results, generates anomaly area heatmaps, anatomical rule traceability information and deviation quantification reports through an interpretability mapping module, outputs structured data including the three-dimensional coordinates and topological relationships of anomalies, and is adapted to downstream clinical workflows such as three-dimensional reconstruction and surgical path planning, thereby providing intuitive, accurate and directly applicable decision support for orthopedic diagnosis and treatment. Attached Figure Description

[0023] Figure 1 This is a flowchart of the image abnormality point recognition method of the present invention applied to orthopedic diagnosis and treatment; Figure 2 This is a block diagram of the image abnormality recognition system of the present invention applied to orthopedic diagnosis and treatment; Figure 3 This is a block diagram of the adaptive data acquisition platform of the present invention; Figure 4 This is a flowchart of the feature learning sub-process guided by anatomical constraints in this invention. Figure 5 This is a flowchart of the hybrid supervised learning optimization sub-process of the present invention; Figure 6 This is a flowchart of the anomaly identification and result output sub-process of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Please see Figures 1-6 This invention provides a technical solution: a method for identifying abnormal points in images used in orthopedic diagnosis and treatment, comprising the following steps: S1. Multimodal Image Acquisition and Preprocessing: Two modal image data related to orthopedics are acquired through an adaptive acquisition platform. The modal image data includes CT image sequences and MRI image sequences. The modal image data is purified to remove artifacts of metal implants and motion artifacts, and standardized based on a parameter calibration benchmark. In addition, the adaptive acquisition platform includes a base, a lifting and adjusting bracket, a rotating stage, and an adaptive positioning fixture. The lifting and adjusting bracket is driven by a ball screw pair, with a lifting stroke of 0-800mm and a positioning accuracy of ±0.1mm. The rotating stage has a rotation angle range of 0-360° and a rotation angle accuracy of ±0.05°. The adaptive positioning fixture is driven by a cylinder, and the clamping force is adjusted in real time through feedback from a pressure sensor. S2. Multimodal Image Collaboration and Feature Alignment: Preprocessed CT and MRI image sequences are input into their respective depth feature encoder branches. Feature maps are extracted from the intermediate layers of the encoder. Using a differentiable affine spatial transformation network submodule, a non-rigid deformation field is calculated based on bony landmark features extracted from the CT images and applied to the MRI feature map, achieving feature-level spatial alignment of key anatomical regions. An adaptive feature-gated fusion unit then performs weighted fusion of the aligned MRI feature map and the CT feature map at the same level. The fusion weights are dynamically generated based on the confidence levels of the two modalities. Furthermore, the weight generation formula for the adaptive feature-gated fusion unit is as follows:

[0026] in For CT feature weights, For MRI feature weights, Confidence level of CT features MRI feature confidence level, in the bone edge region Higher than In soft tissue areas Higher than ; S3. Feature Learning Guided by Anatomical Constraints: Constructing a computable anatomical constraint knowledge base, which includes a statistical shape model and an anatomical relationship rule graph. The expression for the statistical shape model is:

[0027] in For the generated shape, For average shape, Main component, For adjustable weights, The number of principal components is used to input the fused features into the anomaly detection core network, yielding preliminary keypoint prediction results or segmented contours. ,Will The anatomical constraint knowledge base is input, and the difference in shape projection and the penalty value for rule violation are calculated to form the anatomical rationality loss. The anatomical rationality loss is used as a regularization term and combined with the main task loss of the network for backpropagation training. In addition, the nodes of the anatomical relationship rule graph are key anatomical points, including the center of the vertebral body and the midpoint of the articular surface; the edges represent anatomical relationships including adjacent, parallel, symmetrical and distance range constraints. Each edge is attached with a differentiable loss function, and the penalty value for rule violation is the weighted sum of the calculation results of the loss functions of each edge. S4. Hybrid Supervised Learning Optimization: A self-supervised pre-training task is designed based on anatomical context recovery, and the network is pre-trained using a large number of unlabeled normal bone images; the pre-trained network is fine-tuned on limited initial labeled data; an active learning labeling iteration loop is initiated, and samples to be labeled are selected based on model prediction uncertainty and anatomical morphological differences. After expert labeling, the samples are added to the training set to optimize the model. In addition, the sample selection criteria of the active learning labeling iteration loop meet the following requirements: the entropy value of the selected samples is higher than the preset entropy value threshold, and the anatomical morphological distance from the existing training set is greater than the preset distance threshold. The anatomical morphological distance is calculated by a statistical shape model. S5. Anomaly Identification and Output: The network, after training and optimization, infers from multimodal fusion features to identify anomalies in orthopedic images. The interpretability mapping module generates heatmaps of abnormal regions, anatomical rule tracing information, and a deviation quantification report from the normal statistical model. The output is structured data containing the three-dimensional spatial coordinates, topological relationships, and abnormal parameters of the key abnormal points. The structured data is used in downstream clinical workflows. In addition, anomalies include fracture lines, osteophytes, osteolysis, bone tumors, and early osteoarthritis lesions. Downstream clinical workflows include three-dimensional reconstruction, surgical path planning, personalized implant design, and vertebroplasty simulation.

[0028] An image anomaly recognition system applied in orthopedic diagnosis and treatment includes: The multimodal image acquisition and preprocessing module is used to acquire orthopedic image data in two modalities and perform purification and standardization processing. The modal image data includes CT image sequences and MRI image sequences. The module includes an adaptive acquisition platform and sample purification processing components. The multimodal image collaboration and feature alignment module includes two parallel deep feature encoder branches, a differentiable affine spatial transformation network submodule, and an adaptive feature gating fusion unit, which are used to realize feature extraction, spatial alignment, and weighted fusion of multimodal images. The anatomical constraint knowledge base module includes a statistical shape model construction unit and an anatomical relationship rule diagram generation unit. The statistical shape model expression is as follows:

[0029] in For the generated shape, For average shape, Main component, For adjustable weights, The number of principal components is used to provide anatomical constraints and calculate the loss of anatomical rationality. The hybrid supervised learning engine module includes a self-supervised pre-training unit and an active learning annotation iteration unit, which are used to optimize model training based on unlabeled data and limited labeled data; The core network module for anomaly recognition is used to learn features based on fused features and anatomical constraints, and output preliminary key point prediction results or segmented contours. The interpretability mapping and clinical workflow interface module is used to generate abnormal area heatmaps, anatomical rule traceability reports and structured data, and provides an interface with downstream clinical systems.

[0030] In addition, the adaptive acquisition platform includes a cast iron base, a lifting and adjusting bracket driven by a ball screw pair, a rotating platform driven by a worm gear, and an adaptive positioning fixture driven by a cylinder. The bottom of the base is equipped with adjustable horizontal support feet, and the inner side of the adaptive positioning fixture is attached with a medical silicone pad. It is equipped with a pressure sensor for real-time feedback of clamping force.

[0031] In addition, the sample purification and processing components include a switchable filter module, an artifact masking adjustment mechanism with a multi-degree-of-freedom linkage structure, and a parameter calibration reference plate. The switchable filter module has three sets of filters with different wavelengths built in, and the switching response time is ≤0.5s. The parameter calibration reference plate has 256 levels of grayscale gradient blocks and a bone simulation prosthesis of known size built in.

[0032] In addition, the interpretability mapping and clinical workflow interface module includes a feature contribution heatmap generation unit, an anatomical attribution unit, a structured data output unit, and a clinical workflow docking interface. The structured data output unit outputs JSON format files, and the clinical workflow docking interface can connect with computer-aided design software and 3D printer driver software.

[0033] Example 1: Application of spinal fracture line identification 1.1 Experimental Preparation Hardware configuration: The adaptive acquisition platform uses a cast iron base (dimensions 800mm×600mm×150mm), a ball screw drive for lifting and adjusting the bracket (lifting stroke 0-800mm, positioning accuracy ±0.1mm), a worm gear drive for rotating stage (0-360° rotation, angular accuracy ±0.05°), and a cylinder drive for adaptive positioning fixture (equipped with a pressure sensor, clamping force adjustment range 5-30N); the CT equipment uses a 64-slice spiral CT (slice thickness 0.625mm, pixel size 0.4mm×0.4mm), and the MRI equipment uses a 3.0T superconducting MRI (T1 weighted sequence, slice thickness 1mm, matrix 256×256). Software environment: The network model is built based on the PyTorch 1.12 framework, and the hardware acceleration uses an NVIDIA A100 GPU (80GB of video memory). The operating system is Ubuntu 20.04LTS. Dataset: 1000 spinal imaging data were collected, including 800 unlabeled normal spinal images (used for self-supervised pre-training) and 200 data with fracture line annotations (150 cases for training set and 50 cases for test set). The annotation was completed by three orthopedic experts with the title of associate chief physician or above. The fracture types covered compression fractures, burst fractures, transverse process fractures, etc. Anatomical constraint knowledge base configuration: The statistical shape model is constructed based on 500 normal spinal CT images, with 30 principal components, covering the average morphology of cervical vertebrae C1-C7, thoracic vertebrae T1-T12, and lumbar vertebrae L1-L5; the anatomical relationship rule diagram nodes include 12 types of key anatomical points such as vertebral body center, midpoint of superior and inferior articular surfaces, and pedicle center; the edge constraints include the distance between adjacent vertebral body centers (normal range 15-20mm), articular surface parallelism (angle ≤5°), and spinal physiological curvature (Cervical lordosis Cobb angle 20-40°, lumbar lordosis Cobb angle 30-50°), etc.

[0034] 1.2 Implementation Steps S1: Multimodal image acquisition and preprocessing The patient's position was fixed using an adaptive acquisition platform, and spinal CT image sequences (scanning range from the skull base to the sacrum) and MRI image sequences (focusing on areas of suspected injury) were acquired simultaneously. The sample purification and processing component activated the switchable filter module (using a 550nm wavelength filter), and combined with a multi-degree-of-freedom linkage structure artifact masking adjustment mechanism, removed artifacts of metal implants (such as spinal internal fixation rod systems) and motion artifacts. Standardized calibration was performed based on a parameter calibration benchmark plate (with built-in 256-level grayscale gradient blocks and a spinal simulation prosthesis), mapping the pixel values ​​of CT and MRI images to the range of [0,255], with a unified resolution of 1mm×1mm×1mm. S2: Multimodal Image Collaboration and Feature Alignment The preprocessed CT and MRI image sequences were input into the ResNet50 depth feature encoder branch, respectively. A 64-channel feature map was extracted at the third layer of the encoder. A differentiable affine spatial transformation network submodule was used to extract bony landmarks (20 landmarks in total) such as vertebral body edges and pedicles from the CT images. A non-rigid deformation field was calculated and applied to the MRI feature map to achieve feature-level spatial alignment of key anatomical regions (such as suspected fractured vertebrae and two adjacent vertebrae), with an alignment error ≤0.5mm. Weighted fusion was performed using an adaptive feature-gated fusion unit, and the confidence level of CT features in the bone edge region was determined. =0.85, MRI feature confidence level =0.15, the weight is calculated as follows:

[0035] soft tissue area =0.3, =0.7, weight is =0.3, =0.7; S3: Feature Learning Guided by Anatomical Constraints The fused features are input into the anomaly detection core network (an improved U-Net++ model with an attention mechanism added to the decoder) to obtain the preliminary fracture line segmentation contour. ,Will Input the anatomical constraint knowledge base, calculate the shape projection difference (Euclidean distance between the normal spinal shape and the predicted shape) and the rule violation penalty value (e.g., when a fracture causes the distance between the centers of adjacent vertebrae to exceed the range of 15-20mm, a penalty coefficient λ=2.0 is applied), which constitutes the loss of anatomical rationality. The network primary task loss adopts Dice loss. The total loss function is: The learning rate was set to 1e-4, and the training was iterated for 200 epochs. S4: Hybrid Supervised Learning Optimization Based on anatomical context restoration, a self-supervised pre-training task (skeleton morphology completion in occluded areas) was designed. The network was pre-trained for 100 epochs using 800 unlabeled normal images. It was then fine-tuned for 50 epochs on a set of 150 labeled training images. An active learning annotation iteration loop was initiated, with an entropy threshold of 0.8 and an anatomical morphology distance threshold of 1.2 (calculated using a statistical shape model). Twenty high-value unlabeled samples were selected, labeled by experts, and added to the training set. The network was then fine-tuned for another 30 epochs. S5: Anomaly Identification and Result Output The optimized network performs inference on images from 50 cases in the test set. The interpretability mapping module generates a heatmap of the fracture area (red areas indicate high-confidence fracture sites), anatomical rule traceability information (such as "abnormal parallelism between the superior articular surface of L3 vertebra and the inferior articular surface of L2 vertebra, with an angle of 12°, indicating fracture displacement") and a deviation quantification report. It outputs structured JSON data, including parameters such as the three-dimensional spatial coordinates of the fracture line (accuracy ±0.2mm), fracture length, displacement distance, and topological relationship with adjacent vertebrae. Downstream, it interfaces with spinal three-dimensional reconstruction software (such as Mimics) and surgical path planning system.

[0036] 1.3 Implementation Results In the test set of 50 samples, the fracture line recognition accuracy reached 96.8%, recall rate 95.2%, precision rate 97.5%, and the average recognition time was 8.3 seconds per case. Compared with the traditional single-modal CT recognition method, this embodiment improved the accuracy of occult fracture (such as micro compression fracture) recognition by 12.3%, and reduced the fracture displacement distance measurement error from ±1.5mm to ±0.3mm.

[0037] Example 2: Identification of Knee Osteophytes and Early Osteoarthritis Lesions 2.1 Experimental Preparation Hardware configuration: The parameters of the adaptive acquisition platform are the same as in Example 1. The CT equipment is a 128-slice spiral CT (0.5mm slice thickness), and the MRI equipment is a 3.0T superconducting MRI (T2-weighted fat suppression sequence, 0.8mm slice thickness). Dataset: 800 knee joint imaging data were collected, including 600 unlabeled normal data and 200 labeled data (120 training data and 80 test data). The labels included the location and size of osteophytes and the lesion areas of early osteoarthritis (cartilage wear and bone marrow edema). Anatomical constraint knowledge base configuration: The statistical shape model is constructed based on 400 normal knee joint images (n=25). The nodes of the anatomical relationship rule diagram include 10 types of key points such as the femoral condyle center, tibial plateau center, and meniscus attachment point. The edge constraints include the gap between the femoral condyle and tibial plateau (normal range 3-5mm) and the distance of osteophyte protrusion (normal ≤2mm).

[0038] 2.2 Implementation Steps S1: Multimodal image acquisition and preprocessing The knee joint was fixed in a 30° flexion position using an adaptive acquisition platform, and CT and MRI images were acquired simultaneously. The sample purification and processing component removed the metal prosthesis artifacts after knee replacement surgery. The images were standardized using a parameter calibration benchmark plate to achieve a uniform image resolution of 0.8mm×0.8mm×0.8mm. S2: Multimodal Image Collaboration and Feature Alignment EfficientNet-B4 was used as the depth feature encoder to extract 48-channel feature maps. Non-rigid deformation fields were calculated based on bony landmarks of the femoral condyle margin and tibial plateau in CT images, and aligned with the MRI feature maps with an alignment error ≤0.4mm. During adaptive fusion, bone and osteophyte regions were considered. =0.9, =0.1, soft tissue and cartilage region =0.2, =0.8; S3: Feature Learning Guided by Anatomical Constraints The anomaly recognition core network adopts a dual-branch structure to handle osteophyte and cartilage lesion recognition tasks separately. The anatomical rationality loss includes a penalty for osteophyte protrusion distance (λ=1.8 when it exceeds 2mm) and a penalty for joint space distance (λ=2.2 when it deviates from the range of 3-5mm). The total loss function is: Training for 180 epochs; S4: Hybrid Supervised Learning Optimization The self-supervised pre-training task was the rotational recovery of the knee joint anatomical structure. It used 600 unlabeled data for 80 epochs of pre-training; 120 labeled data for 40 epochs of fine-tuning; and actively learned to select 30 samples with entropy value ≥0.75 and anatomical morphology distance ≥1.0, labeled them to supplement the training set, and fine-tuned them for 25 epochs. S5: Anomaly Identification and Result Output Generates heat maps of osteophyte areas and cartilage wear tracing reports (such as "medial femoral condyle cartilage thickness 2.1mm, below the normal range of 3-4mm, consistent with early osteoarthritis manifestations"), outputs structured data including osteophyte three-dimensional coordinates, volume, cartilage wear degree score (0-4 points), etc., and interfaces with personalized knee joint prosthesis design software.

[0039] 2.3 Implementation Results In the test set of 80 samples, the accuracy rate of osteophyte identification was 94.5%, the accuracy rate of early osteoarthritis lesion identification was 92.8%, and the accuracy rate of cartilage wear area identification was 15.7% higher than that of traditional MRI single-modal identification method. The error of osteophyte volume measurement was ≤5%, and the average processing time was 9.1 seconds / case.

[0040] Example 3: Identification of Pelvic Bone Tumors and Osteolysis Zones 3.1 Experimental Preparation Hardware configuration: The adaptive acquisition platform adds a special positioning mold for the pelvis; the CT equipment is a 256-slice spiral CT (0.3mm slice thickness); and the MRI equipment is a 3.0T superconducting MRI (diffusion weighted sequence, b-value = 800s / mm²). Dataset: 600 cases of pelvic imaging data were collected, 500 cases of unlabeled normal data, and 100 cases of labeled data (70 cases in the training set and 30 cases in the test set). The labels include the location, boundary and extent of bone resorption area of ​​bone tumors. Tumor types include osteosarcoma, giant cell tumor of bone, metastatic tumors, etc. Anatomical constraint knowledge base configuration: The statistical shape model is constructed based on 300 normal pelvic images (n=35). The nodes of the anatomical relationship rule diagram include 15 types of key points such as the anterior superior iliac spine, the center of the pubic symphysis, and the ischial tuberosity. The edge constraints include the angle between the ilium and the ischium (normal range 130-150°), the smoothness constraint of the boundary between the osteolysis area and normal bone tissue, etc.

[0041] 3.2 Implementation Steps S1: Multimodal image acquisition and preprocessing The patient's position is fixed by a pelvic mold of the adaptive acquisition platform, and CT and MRI images are acquired simultaneously. The sample purification and processing component removes artifacts such as metal contraceptive rings in the pelvis. After standardization by the parameter calibration benchmark plate, the image resolution is 0.5mm×0.5mm×0.5mm. S2: Multimodal Image Collaboration and Feature Alignment VisionTransformer (ViT) was used as the depth feature encoder to extract 768-channel feature maps. Based on CT bony landmarks, MRI feature maps were aligned with an alignment error ≤0.3mm. During adaptive fusion, the bone parenchyma region was considered. =0.88, =0.12, area of ​​tumor and soft tissue infiltration =0.25, =0.75; S3: Feature Learning Guided by Anatomical Constraints The core network for anomaly detection adopts a Transformer-U-Net hybrid structure. The anatomical rationality loss includes a bone morphology angle penalty (λ=2.5 when deviating from the 130-150° range) and a boundary smoothness penalty (λ=1.6 when the boundary roughness of the osteolysis area exceeds the normal range). The total loss function is: Training for 220 epochs; S4: Hybrid Supervised Learning Optimization The self-supervised pre-training task was pelvic anatomical region mask recovery. It used 500 unlabeled data for 100 epochs of pre-training; 70 labeled data for 50 epochs of fine-tuning; and actively learned to select 20 samples with entropy values ​​≥0.82 and anatomical morphology distances ≥1.3, labeled them to supplement the training set, and fine-tuned them for 30 epochs. S5: Anomaly Identification and Result Output Generates heat maps of bone tumor / bone resorption areas and anatomical rule tracing reports (such as "the bone morphology angle of the right iliac bone region is 115°, which is lower than the normal range, indicating that the tumor invasion has caused bone destruction"). Outputs structured data including the three-dimensional coordinates, volume, and distance relationship with surrounding organs of the lesion area, and connects with the tumor puncture path planning and radiotherapy target delineation system. 3.3 Implementation Results In the test set of 30 samples, the accuracy rate of bone tumor identification was 93.7%, and the accuracy rate of osteolysis area identification was 91.5%, which improved the diagnostic efficiency by 80% compared with the traditional CT+MRI manual image reading. The error of lesion boundary delineation was ≤0.6mm, providing accurate reference for the formulation of clinical treatment plans.

[0042]

[0043] Table 1: Multi-dimensional performance comparison between the present invention and traditional methods As shown in Table 1, the present invention has the following effects: Enhanced Precision: By adaptively fusing CT and MRI in a dual-modal manner, the advantages of CT in imaging hard bone tissue and MRI in imaging soft tissue and lesion infiltration areas are fully utilized. Combined with the regularization effect of the anatomical constraint knowledge base, the accuracy of abnormal point identification is improved by 8-13 percentage points compared with traditional methods, and the measurement error is controlled within the range of 0.3-0.6mm, meeting the needs of precise clinical diagnosis and treatment. Significant efficiency improvements: The hybrid supervised learning model reduces the reliance on large-scale labeled data. Self-supervised pre-training uses unlabeled data to improve the basic performance of the model, and active learning selects high-value samples, reducing the amount of labeled data required by more than 60%. At the same time, the automated identification process reduces the processing time of a single image from 18-25 seconds in the traditional method to 8-10 seconds, greatly improving the efficiency of clinical work. Wide clinical applicability: The three implementations cover core orthopedic areas such as the spine, knee joint, and pelvis, and can identify various orthopedic abnormalities such as fracture lines, osteophytes, bone tumors, osteolysis, and early arthritis. The output structured data can be seamlessly integrated with downstream clinical workflows such as 3D reconstruction, surgical path planning, and personalized implant design, adapting to the entire process of orthopedic diagnosis, treatment, and rehabilitation. Enhanced interpretability: Through abnormal area heatmaps, anatomical rule tracing reports, and deviation quantification analysis, AI recognition results are no longer "black box outputs." Clinicians can intuitively understand the differences between abnormal points and normal anatomical structures and the anatomical rules violated, thereby increasing their trust in AI results and promoting the clinical translation of the technology. Excellent robustness: The high-precision positioning and sample purification of the adaptive acquisition platform effectively reduce the impact of artifacts and body position deviations on the recognition results; the anatomical constraint-guided feature learning enables the model to maintain stable performance when facing complex anatomical variation scenarios. The three embodiments all show high robustness in different locations and different lesion types.

[0044] The present invention has been described in detail above. However, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, any modifications or improvements that do not depart from the spirit of the present invention are within the scope of protection of the present invention.

Claims

1. A method for identifying abnormal points in images used in orthopedic diagnosis and treatment, characterized in that, Includes the following steps: S1. Multimodal image acquisition and preprocessing: Two modal image data related to orthopedics of the patient are acquired through an adaptive acquisition platform. The modal image data includes CT image sequences and MRI image sequences. The modal image data is purified to remove metal implant artifacts and motion artifacts, and standardized calibration is performed based on a parameter calibration benchmark. S2. Multimodal Image Collaboration and Feature Alignment: The preprocessed CT image sequence and MRI image sequence are input into the corresponding depth feature encoder branches respectively. Feature maps are extracted in the middle layer of the encoder. Through the differentiable affine spatial transformation network submodule, based on the bony landmark features extracted from the CT images, a non-rigid deformation field is calculated and applied to the MRI feature map to achieve feature-level spatial alignment of key anatomical regions. The aligned MRI feature map and the CT feature map of the same level are weighted and fused using an adaptive feature-gated fusion unit. The fusion weight is dynamically generated by the confidence of the two modal features. S3. Feature Learning Guided by Anatomical Constraints: Constructing a computable anatomical constraint knowledge base, which includes a statistical shape model and an anatomical relationship rule graph. The expression of the statistical shape model is as follows: in For the generated shape, For average shape, Main component, For adjustable weights, The number of principal components is used to input the fused features into the anomaly detection core network, yielding preliminary keypoint prediction results or segmented contours. ,Will Input the anatomical constraint knowledge base, calculate the shape projection difference and rule violation penalty value to form the anatomical rationality loss, and combine the anatomical rationality loss as a regularization term with the network main task loss to perform backpropagation to train the network; S4. Hybrid supervised learning optimization: A self-supervised pre-training task is designed based on anatomical context recovery, and the network is pre-trained using a large number of unlabeled normal bone images; the pre-trained network is then fine-tuned on limited initial labeled data. Initiate an active learning annotation iteration loop, select samples to be annotated based on model prediction uncertainty and anatomical morphology differences, and add them to the training set to optimize the model after expert annotation; S5. Anomaly Identification and Result Output: The network is trained and optimized to infer multimodal fusion features and identify anomalies in orthopedic images. An interpretable mapping module generates anomaly region heatmaps, anatomical rule tracing information, and a deviation quantification report from the normal statistical model. The output is structured data containing the three-dimensional spatial coordinates, topological relationships, and anomaly parameters of the key points of the anomaly. The structured data is used in downstream clinical workflows.

2. The image abnormality identification method applied to orthopedic diagnosis and treatment according to claim 1, characterized in that: The adaptive acquisition platform described in step S1 includes a base, a lifting and adjusting bracket, a rotating platform, and an adaptive positioning fixture.

3. The image abnormality identification method applied to orthopedic diagnosis and treatment according to claim 1, characterized in that: The weight generation formula for the adaptive feature gating fusion unit in step S2 is: in For CT feature weights, For MRI feature weights, Confidence level of CT features MRI feature confidence level, in the bone edge region Higher than In soft tissue areas Higher than .

4. The image abnormality identification method applied to orthopedic diagnosis and treatment according to claim 1, characterized in that: The nodes of the anatomical relationship rule diagram in step S3 are key anatomical points, including the center of the vertebral body and the midpoint of the articular surface; the edges represent anatomical relationships including adjacent, parallel, symmetrical and distance range constraints, and each edge is attached with a differentiable loss function. The penalty value for violating the rule is the weighted sum of the calculation results of the loss functions of each edge.

5. The image abnormality identification method applied to orthopedic diagnosis and treatment according to claim 1, characterized in that: The sample selection criteria for the active learning annotation iteration loop in step S4 are: the entropy value of the selected sample is higher than the preset entropy threshold, and the anatomical distance between the selected sample and the existing training set is greater than the preset distance threshold. The anatomical distance is calculated by a statistical shape model.

6. The image abnormality identification method applied to orthopedic diagnosis and treatment according to claim 1, characterized in that: The abnormal points mentioned in step S5 include fracture lines, osteophytes, osteolysis, bone tumors, and early osteoarthritis lesion areas. The downstream clinical workflow includes three-dimensional reconstruction, surgical path planning, personalized implant design, and vertebroplasty simulation.

7. An image abnormality recognition system applied in orthopedic diagnosis and treatment, characterized in that, include: A multimodal image acquisition and preprocessing module is used to acquire orthopedic image data of two modalities and perform purification and standardization processing. The modal image data includes CT image sequences and MRI image sequences. The module includes an adaptive acquisition platform and a sample purification processing component. The multimodal image collaboration and feature alignment module includes two parallel deep feature encoder branches, a differentiable affine spatial transformation network submodule, and an adaptive feature gating fusion unit, which are used to realize feature extraction, spatial alignment, and weighted fusion of multimodal images. The anatomical constraint knowledge base module includes a statistical shape model construction unit and an anatomical relationship rule diagram generation unit. The statistical shape model expression is as follows: in For the generated shape, For average shape, Main component, For adjustable weights, The number of principal components is used to provide anatomical constraints and calculate the loss of anatomical rationality. The hybrid supervised learning engine module includes a self-supervised pre-training unit and an active learning annotation iteration unit, which are used to optimize model training based on unlabeled data and limited labeled data; The core network module for anomaly recognition is used to learn features based on fused features and anatomical constraints, and output preliminary key point prediction results or segmented contours. The interpretability mapping and clinical workflow interface module is used to generate abnormal area heatmaps, anatomical rule traceability reports and structured data, and provides an interface with downstream clinical systems.

8. The image abnormality recognition system applied in orthopedic diagnosis and treatment according to claim 7, characterized in that: The adaptive acquisition platform includes a cast iron base, a lifting and adjusting bracket driven by a ball screw pair, a rotating platform driven by a worm gear, and an adaptive positioning fixture driven by a cylinder.

9. The image abnormality recognition system applied in orthopedic diagnosis and treatment according to claim 7, characterized in that: The sample purification and processing component includes a switchable filter module, an artifact masking adjustment mechanism with a multi-degree-of-freedom linkage structure, and a parameter calibration reference plate. The switchable filter module has three sets of filters with different wavelengths built in, and the parameter calibration reference plate has gradient blocks and a bone simulation prosthesis of known size built in.

10. The image abnormality recognition system applied in orthopedic diagnosis and treatment according to claim 7, characterized in that: The interpretability mapping and clinical workflow interface module includes a feature contribution heatmap generation unit, an anatomical attribution unit, a structured data output unit, and a clinical workflow interface.

Images