A method and system for evaluating rheumatic joint lesions based on image recognition
By employing a dual-modal joint degradation assessment and local deformation correction method, high-confidence feature regions are identified, resolving the spatial localization ambiguity problem between ultrasound and magnetic resonance images in the context of pathological degradation. This enables high-precision joint analysis of cross-modal lesion features, improving the accuracy and reliability of rheumatic joint disease assessment.
Patent Information
- Application Number
- CN202611295471.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-25
- Publication Date
- 2026-11-10
AI Technical Summary
Existing ultrasound and magnetic resonance multimodal images suffer from spatial localization ambiguity in the context of pathological degradation, leading to low accuracy in the joint analysis of cross-modal lesion features and affecting the accuracy and reliability of clinical assessment.
By employing an image recognition-based approach, a dual-modal joint degradation assessment is performed. High-confidence feature regions are identified and used as the primary spatial constraints for candidate spatial pose search. Furthermore, local deformation correction and weighted joint statistical analysis are used to eliminate localization ambiguities caused by pathological bone erosion, thereby improving assessment accuracy.
It effectively resolves spatial localization ambiguities in multimodal images under pathological degradation backgrounds, enables high-precision joint analysis of cross-modal lesion features, and improves the accuracy and reliability of clinical assessment.
Smart Images

Figure CN123071297A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image recognition, in particular to a rheumatic joint lesion evaluation method and system based on image recognition. BACKGROUND
[0002] In the clinical evaluation of knee rheumatoid arthritis, multi-modal statistical analysis of ultrasound and magnetic resonance is a common method to realize cross-modal quantitative verification of synovial hyperplasia. In actual clinical operation, doctors usually detect the synovial inflammatory area under the standard section and record the approximate angle of the section in the magnetic resonance coordinate system. Then, the bone boundary in the ultrasound image is matched with the corresponding structure in the magnetic resonance image by using image similarity measurement, and the position of the section in the magnetic resonance space is inversely solved, so as to realize the fusion of ultrasound measurement value and magnetic resonance statistical data.
[0003] However, the existing ultrasound section space positioning technology highly depends on the integrity of the bone reference. When the joint part is diseased, for example, when the femoral trochlear groove appears bone erosion, the bone marker points in the ultrasound image often appear intermittent or blurred due to sound wave scattering and shielding, and the same area in the magnetic resonance image also shows uneven signal due to pathological changes. This asymmetric visibility degradation under the dual modal leads to the loss of key local bone reference, so that the positioning algorithm based on single similarity measurement cannot obtain a unique matching solution when searching for the best position due to the degradation of the marker points, and then multiple candidate spatial poses are generated, causing mapping ambiguity.
[0004] When the algorithm mistakenly selects the position of the non-real section among multiple candidate poses, the ultrasound synovial region will be incorrectly mapped to other joint recess positions in the magnetic resonance statistical space. The deviation of spatial positioning causes the joint statistical analysis of synovial thickness and magnetic resonance signal to be mismatched in anatomical position, which further conceals or fabricates the real relationship between the lesion and the anatomical structure, seriously affecting the accuracy and reliability of clinical evaluation. Therefore, how to solve the spatial positioning ambiguity of multi-modal images in the pathological degradation background and realize the high-precision cross-modal joint analysis of lesion characteristics is a technical problem to be solved in the field of medical image processing.
[0005] In view of the above problems, the existing technology needs to be improved. SUMMARY
[0006] The purpose of this invention is to provide an image recognition-based method and system for assessing rheumatic joint lesions. It aims to solve the technical problems of spatial localization ambiguity and low accuracy of joint analysis of lesion features caused by the lack of bony reference in multimodal images under pathological degeneration background. It can automatically identify and shield low confidence features based on bimodal degeneration assessment, use high confidence region constraint search and supplement with local deformation correction, effectively eliminate localization ambiguity caused by pathological bone erosion, and significantly improve the accuracy of cross-modal lesion assessment.
[0007] In a first aspect, the present invention provides an image recognition-based method for assessing rheumatic joint lesions, comprising the following steps: S1. Acquire a first modal image and a second modal image containing the target monitoring object; S2. Extract local reference features from the first modality image and the second modality image respectively; S3. By performing a dual-modal joint degradation assessment on the local reference features, an assessment result is obtained, and based on the assessment result, high-confidence feature regions and low-confidence feature regions are identified; S4. Using the high-confidence feature region as the main spatial constraint, search for multiple candidate spatial poses of the first modality image in the spatial coordinate system corresponding to the second modality image; S5. Determine the target spatial pose by verifying each of the candidate spatial poses; S6. Based on the target spatial pose, the first modal image is mapped onto the second modal image, and according to the distribution characteristics of the low confidence feature region, the intermodal mapping boundary under the target spatial pose is locally deformed to obtain each local target region after mapping; S7. Based on the evaluation results, assign weight labels to each of the local target regions, and based on the weight labels, perform weighted joint statistical analysis on the lesion features in the first modal image and the second modal image to comprehensively evaluate the degree of rheumatic joint disease.
[0008] The image recognition-based assessment method for rheumatic joint diseases provided by this invention has the advantages of effectively resolving spatial localization ambiguities in multimodal images under pathological degeneration backgrounds, achieving high-precision joint analysis of cross-modal lesion features, and improving the accuracy and reliability of clinical assessment.
[0009] Secondly, the present invention provides an image recognition-based assessment system for rheumatic joint diseases, comprising: The image acquisition module is used to acquire a first modal image and a second modal image containing the target monitoring object; The feature extraction module is used to extract local reference features from the first modality image and the second modality image, respectively; The region identification module is used to obtain an evaluation result by performing a dual-modal joint degradation evaluation on the local reference features, and to identify high-confidence feature regions and low-confidence feature regions based on the evaluation result; The search module is used to search for multiple candidate spatial poses of the first modality image in the spatial coordinate system corresponding to the second modality image, using the high-confidence feature region as the main spatial constraint. The verification module is used to determine the target spatial pose by verifying each of the candidate spatial poses; The mapping correction module is used to map the first modal image to the second modal image based on the target spatial pose, and to perform local deformation correction on the intermodal mapping boundary under the target spatial pose according to the distribution characteristics of the low confidence feature region, so as to obtain each local target region after mapping. The statistical analysis module is used to assign weight labels to each of the local target regions based on the evaluation results, and to perform weighted joint statistical analysis on the lesion features in the first modality image and the second modality image based on the weight labels, so as to comprehensively evaluate the degree of rheumatic joint disease.
[0010] As can be seen from the above, the image recognition-based rheumatic arthritis assessment method provided by this invention effectively identifies and quantifies the degeneration of bony references in medical images through a dual-modal joint degradation assessment mechanism, providing reliable constraints for subsequent spatial localization. During the localization process, the problem of pose ambiguity caused by lesions is successfully resolved through high-confidence feature-guided search and multi-dimensional consistency verification (geometric, pathological, and soft tissue distribution). Furthermore, the local deformation correction and contribution-weighted statistical method proposed in this application ensures the accuracy of cross-modal feature mapping, avoiding anatomical mismatches in the lesion area caused by traditional rigid registration, and greatly improving the clinical accuracy and reference value of rheumatic arthritis assessment results.
[0011] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description
[0012] Figure 1 This is a flowchart of an image recognition-based assessment method for rheumatic joint diseases provided in an embodiment of the present invention.
[0013] Figure 2This is a schematic diagram of a rheumatic joint disease assessment system based on image recognition provided in an embodiment of the present invention.
[0014] Label Explanation: 100. Image acquisition module; 200. Feature extraction module; 300. Region recognition module; 400. Search module; 500. Verification module; 600. Mapping correction module; 700. Statistical analysis module. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0016] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0017] In the clinical evaluation of rheumatoid arthritis of the knee, multimodal statistical analysis combining ultrasound and magnetic resonance imaging (MRI) is a routine method for cross-modal quantitative verification of the degree of synovial hyperplasia. In actual clinical practice, physicians typically probe the inflammatory area of the synovium under a standard section and record the approximate angle of this section in the MRI coordinate system. Subsequently, image similarity measures are used to match the bony boundaries in the ultrasound image with the corresponding structures in the MRI image to inversely determine the position of the section in MRI space, thereby achieving the fusion of ultrasound measurements and MRI statistical data. However, existing ultrasound section spatial localization techniques are highly dependent on the integrity of the bony reference. When lesions occur in the joint, such as bone erosion in the trochlear groove of the femur, the bony markers in the ultrasound image often appear discontinuous or blurred due to sound wave scattering and obstruction, and the same area in the MRI image may also show signal heterogeneity due to pathological changes. This asymmetric visibility degradation in the bimodal context leads to the loss of key local skeletal references. As a result, localization algorithms based on a single similarity metric cannot obtain a unique matching solution when searching for the optimal location due to the degradation of marker points, thus generating multiple candidate spatial poses and causing mapping ambiguity.
[0018] Specifically, in the scenario of assessing synovial inflammation in the trochlear groove region of the femur, when the ultrasound probe acquires an image containing bone erosion lesions, the cortical bone contour in the ultrasound image appears discontinuous due to the scattering of sound waves on the irregular bone surface. Simultaneously, the signal intensity in the MRI image in this region changes due to inflammation and edema, blurring the originally clear bony anatomical structure and the boundary with the surrounding soft tissue. At this point, when the localization algorithm performs image registration, the lack of continuous and clear bony reference points prevents it from locking onto a unique corresponding section in the MRI space, resulting in the algorithm outputting multiple candidate poses. If the system mistakenly selects a location that is not the true section, the ultrasound synovial region will be incorrectly mapped to other joint recess locations in the MRI statistical space, leading to anatomical mismatch.
[0019] If the aforementioned spatial positioning ambiguity is not resolved, the joint statistical analysis of ultrasound synovial thickness and magnetic resonance signals will produce biases. These biases will obscure or fabricate the true relationship between lesions and anatomical structures, causing clinical assessment results to fail to reflect the true state of joint lesions and seriously affecting the accuracy and reliability of medical image processing.
[0020] For reference, see the appendix. Figure 1 This invention provides an image recognition-based method for assessing rheumatic joint lesions, comprising the following steps: S1. Acquire a first modal image and a second modal image containing the target monitoring object; the first modal image is an ultrasound image; the second modal image is a magnetic resonance image; S2. Extract local reference features from the first modality image and the second modality image respectively; S3. By performing a dual-modal joint degradation assessment on local reference features, the assessment results are obtained, and based on the assessment results, high-confidence feature regions and low-confidence feature regions are identified; S4. Using high-confidence feature regions as the main spatial constraints, search for multiple candidate spatial poses of the first modality image in the spatial coordinate system corresponding to the second modality image; S5. Determine the target spatial pose by verifying each candidate spatial pose; S6. Based on the target spatial pose, the first modal image is mapped to the second modal image, and according to the distribution characteristics of the low confidence feature region, the intermodal mapping boundary under the target spatial pose is locally deformed to obtain each local target region after mapping. S7. Based on the evaluation results, assign weight labels to each local target region, and based on the weight labels, perform weighted joint statistical analysis on the lesion features in the first modality image and the second modality image to comprehensively evaluate the degree of rheumatic joint disease.
[0021] For ease of understanding, the following explains some key terms in this embodiment: First modality image: refers to image data acquired through a specific imaging technique, such as ultrasound image in this application. Ultrasound image generates an image by utilizing the reflection and scattering characteristics of ultrasound waves in tissues, and can display soft tissue structure and blood flow information in real time, but its image quality is easily affected by operator experience and tissue attenuation.
[0022] Second-modal images refer to image data acquired through another imaging technique, such as magnetic resonance images in this application. Magnetic resonance images generate images by utilizing the resonance phenomenon of atomic nuclei under the action of strong magnetic fields and radio frequency pulses. They can provide high-resolution soft tissue contrast and enable three-dimensional imaging, but their acquisition time is long and the cost is high.
[0023] Local reference features: These are anatomical or pathological features that are identifiable in both the first and second modal images and are used for image registration and structural localization. These features are usually bony structures or their boundaries, but their morphology and signal may degenerate in pathological conditions.
[0024] Bimodal joint degradation assessment refers to the process of comprehensively analyzing the degree of degradation of local reference features in the first and second modal images to determine their reliability. This assessment aims to identify which feature regions become unreliable due to lesions, thereby avoiding the introduction of errors in subsequent processing.
[0025] High-confidence feature regions refer to local reference feature regions that, after dual-modal joint degradation evaluation, are determined to be morphologically intact, signal-stable, and reliably reflect anatomical structures. These regions have high weight and reference value in image registration.
[0026] Low-confidence feature regions: These refer to local reference feature regions that are identified as having damaged morphology, uneven signal, and low reliability after dual-modal joint degradation evaluation. These regions require special processing during image registration, such as weight reduction or local deformation correction.
[0027] Candidate spatial pose: refers to the multiple potential spatial locations and poses that the first modality image may have within the spatial coordinate system corresponding to the second modality image. Due to lesions or image noise, there may be multiple similar matching results, forming a set of candidate poses.
[0028] Target spatial pose: refers to the unique spatial pose that, after verifying the candidate spatial poses, is determined to be the one that most accurately reflects the true position and orientation of the first modal image in the second modal image.
[0029] Intermodal mapping boundary: This refers to the boundary formed by the first modal image in the space of the second modal image after mapping the first modal image to the second modal image. This boundary defines the corresponding regions of the two modal images.
[0030] Local deformation correction: refers to the process of making local non-rigid adjustments to the intermodal mapping boundary for low confidence feature regions, so as to make it more accurately fit the actual anatomical structure or lesion distribution and compensate for the local structural differences caused by the lesion.
[0031] Weighting labels: These are numerical values or labels assigned to each local target region based on the results of the bimodal joint degradation assessment, indicating its reliability or importance. High-weighting labels are typically assigned to high-confidence regions, and low-weighting labels are assigned to low-confidence regions.
[0032] Weighted joint statistical analysis: This refers to adjusting the contribution of each local target region to the statistical results based on its weight label when performing statistical analysis of lesion features in multimodal images, in order to improve the accuracy and reliability of the assessment.
[0033] This application proposes an image recognition-based method for assessing rheumatic joint lesions, aiming to address the problems of spatial localization ambiguity and inaccurate joint analysis of cross-modal lesion features in traditional methods under the background of pathological degeneration.
[0034] In the steps of acquiring first and second modal images containing the target monitoring object, the system first receives ultrasound image sequences and magnetic resonance imaging (MRI) volumetric data from the same examination cycle for the patient. The ultrasound images are acquired by experienced sonographers by probing and freezing the inflammatory synovial region under standard sections. These images not only contain information about the soft tissue of the synovial hyperplasia but also record the reflection signals of the underlying bony boundaries. Simultaneously with ultrasound acquisition, the system obtains the approximate angle and initial pose range of the ultrasound section in the human anatomical coordinate system through spatial positioning devices or the physician's rough recording. The MRI data is a three-dimensional high-resolution volumetric sequence covering the entire knee joint, typically including multiple sequences such as T1-weighted, T2-weighted, and proton density-weighted sequences to comprehensively reflect the pathological state of the cortical bone, medullary cavity, and intra-articular synovium. After acquiring the raw data from these two modalities, the system initiates a basic preprocessing module. For ultrasound images, due to the inherent speckle noise of their imaging physics mechanism, which severely interferes with subsequent bony boundary extraction, the system employs a smoothing algorithm based on anisotropic diffusion filtering to denoise the ultrasound images. This algorithm effectively preserves and enhances bony reflection boundaries with high gradients while smoothing noise in uniform soft tissue regions. For MRI data, the system first performs bias field correction to eliminate local signal intensity variations in the image caused by radio frequency coil inhomogeneities, ensuring statistical consistency of signal intensity for the same anatomical structure at different spatial locations. Subsequently, the system maps the two-dimensional pixel coordinate system of the ultrasound image and the three-dimensional voxel coordinate system of the MRI image to a global reference coordinate system, constructing an initial search space using the initial pose range recorded during ultrasound acquisition. This preprocessing not only standardizes the grayscale distribution and spatial scale of the dual-modal images but also defines reasonable computational boundaries for finding specific two-dimensional sections in a massive number of three-dimensional voxels, avoiding the huge computational overhead and mismatch risk caused by blind global searches. Simultaneously, the system establishes a contextual information association table for multimodal data, aligning and storing metadata such as ultrasound image acquisition time, probe frequency, and depth settings with MRI scanning parameters, slice thickness, and slice spacing, so that this contextual information can be readily accessed to calibrate matching parameters during subsequent joint statistical analysis.
[0035] In the step of extracting local reference features from the first and second modal images respectively, the system needs to extract local bony reference information that can support the localization of the same anatomical section from ultrasound images and magnetic resonance imaging data, and establish a comparable multimodal feature space between the two modalities. Because the imaging principles of ultrasound and magnetic resonance are fundamentally different, the apparent features of the same bony structure differ greatly between the two images. In ultrasound images, the cortical bone appears as a continuous or discontinuous bright line with strong echoes, usually accompanied by acoustic shadowing; while in magnetic resonance images, normal cortical bone appears as a dark band with extremely low signal, contrasting sharply with the high signal of the joint fluid or fat pad. Therefore, the system cannot directly use raw grayscale pixels for matching, but must extract high-level geometric and semantic features. The system first applies a phase consistency-based edge detection algorithm to the ultrasound image. Unlike traditional grayscale gradient-based edge detection, phase consistency is insensitive to changes in local image contrast and can more stably extract bony boundaries in ultrasound images where brightness is uneven due to scattering and attenuation. The extracted bony boundaries are represented as a set of two-dimensional point clouds. The system further calculates the local normal vector, principal curvature, and local geometric descriptor based on shape context for each boundary point. On the magnetic resonance imaging (MRI) data side, the system uses a three-dimensional level set-based method to perform three-dimensional segmentation of the femoral condyle, tibial plateau, and patellar cortical surfaces of the knee joint, extracting a continuous three-dimensional bone surface mesh model. For each vertex on this three-dimensional model, the system also calculates its three-dimensional normal vector, Gaussian curvature, mean curvature, and three-dimensional shape context descriptor. To establish a comparable feature space between two-dimensional ultrasound features and three-dimensional MRI features, the system constructs a cross-modal feature mapping operator. This operator projects the geometric features of the two-dimensional ultrasound boundary points and the geometric features of the three-dimensional MRI surface vertices into a unified high-dimensional feature vector space. In this high-dimensional space, the distance between feature vectors directly reflects the geometric similarity of the anatomical structures, no longer affected by local gray-level inversion and imaging modal differences. In addition, the system also extracts the contour of the synovial region adjacent to the bony boundaries as auxiliary reference features. The hypoechoic boundaries of the synovial hyperplasia region were extracted using ultrasound, while the enhanced high-signal synovial volume surface was extracted using magnetic resonance imaging. These bony references and adjacent soft tissue features together constitute the basic elements of the multimodal feature space, providing rich and structured input information for subsequent identification of degenerative states and spatial localization.
[0036] In the steps of performing a dual-modal joint degradation assessment of local reference features, obtaining assessment results, and identifying high-confidence and low-confidence feature regions based on these results, in the pathological state of rheumatoid arthritis, bone erosion and synovitis lead to asymmetric degradation of local bony structures. Directly using the features extracted in the previous step for matching would inevitably introduce spurious correspondences. Therefore, the system must quantitatively assess the degradation state of the dual-modal local references before spatial localization, clearly distinguishing between stable and usable regions and low-confidence regions. The system traverses the two-dimensional bony boundary point cloud extracted from ultrasound images, evaluating the integrity and continuity of each local boundary segment. Specifically, the system calculates the frequency of interruptions and the variance of echo intensity within a fixed-length window. If the bony boundary of a certain region exhibits frequent interruptions and abnormal attenuation of echo intensity, the system determines that the region may be severely obscured by synovial hyperplasia or has already experienced cortical bone erosion defects. Simultaneously, on the three-dimensional bone surface model extracted by magnetic resonance imaging, the system evaluates the uniformity of the cortical bone signal and surface smoothness along the candidate regions corresponding to the initial search space. The system calculates the normal vector divergence of the local bone surface and the probability of abnormally high signal intensity in the adjacent medullary cavity. Based on the above single-modal degradation characteristics, the system constructs a dual-modal joint degradation assessment model. For any pair of potential corresponding points in the feature space, the system calculates their joint degradation index. The calculation of the joint degradation index comprehensively considers the boundary continuity score on the ultrasound side, the signal homogeneity score on the MRI side, and the consistency of their expression forms. When a local reference appears as a blurred outline on ultrasound and the corresponding region appears as cortical interruption or signal heterogeneity on MRI, the joint degradation index will increase significantly. The system sets a dynamically adaptive degradation threshold, dividing all extracted bony reference features into two sets: feature points with a joint degradation index below the threshold are assigned to the stable usable region set, which represents healthy bone references that have not been significantly eroded or damaged by pathology and can truly reflect anatomical geometry; feature points with a joint degradation index above or equal to the threshold are assigned to the low-confidence region set, which represents damaged references that have decreased visibility and morphological distortion due to pathological degradation. Through this preliminary identification mechanism, the system successfully distinguishes between structural defects caused by actual pathological defects and conventional apparent defects, thus preventing the risk of degradation features dominating subsequent spatial matching and enabling the system to selectively utilize reliable information for cross-sectional localization.
[0037] In the step of searching for multiple candidate spatial poses of the first modality image within the spatial coordinate system corresponding to the second modality image, using high-confidence feature regions as the main spatial constraints, after completing the availability identification of local references, the system uses the selected set of stable and usable regions as the main localization constraints to search for candidate positions of ultrasound sections in the 3D space of magnetic resonance imaging (MRI). Because low-confidence regions are eliminated, the number of effective features participating in matching is reduced. While this improves the matching's resistance to pathological interference, it also makes the spatial constraints relatively loose, easily generating multiple local similarity extrema in the vast 3D search space. To efficiently and comprehensively find all potential true corresponding positions, the system adopts a restricted parameter space search strategy based on hierarchical sampling. The system first discretizes the translation and rotation parameters of the 3D volume of the MRI within the constraints of the initial coarse pose, generating a series of virtual sections in 3D space. For each virtual section, the system extracts its intersection line with the 3D bone surface model of the MRI, using it as the MRI lateral bony reference projection in that pose. Subsequently, the system uses only the 2D boundary point cloud of the stable and usable region on the ultrasound side to perform feature matching with the intersection line on the virtual cross-section. During the matching process, the system calculates the point-to-point Euclidean distance and the cosine value of the angle between the normal vectors of the stable ultrasound boundary points and the magnetic resonance intersection line, and uses a Gaussian kernel function to convert these distance and angle differences into local similarity scores. Since low-confidence regions are explicitly restricted from participating in dominant matching, their weights in the similarity calculation are set to minimum values or directly zero, thus ensuring that the similarity score is entirely contributed by stable and healthy anatomical structures. The system traverses all sampled virtual cross-section parameters and records the global similarity score corresponding to each parameter combination. In the parameter space, the system searches for local maxima of the similarity score. Due to the complexity of the knee joint anatomy in rheumatoid arthritis patients and the locality of effective reference features, the system usually detects multiple local peaks with similar scores. The spatial poses corresponding to these local peaks are the candidate spatial solutions. The system extracts all local peaks with similarity scores exceeding a preset baseline threshold to construct a set of candidate spatial solutions. Each candidate solution not only contains a specific 3D spatial transformation matrix, but also includes the mapping residual distribution of the stable feature point pairs participating in the matching at that location. This step transforms the blind mismatch problem caused by the simultaneous degradation of the bimodal reference into a process of generating multiple high-confidence candidate solutions under constrained and reliable feature constraints, laying the foundation for subsequent resolution of multiple solution ambiguities.
[0038] In the step of determining the target spatial pose by verifying each candidate spatial pose, the system must provide clear ambiguity resolution criteria for the multiple candidate spatial solutions generated in the previous step, and inversely solve for the unique and true ultrasound section position to avoid the system falling into spatial mapping errors based solely on a single similarity peak. The system introduces cross-regional geometric relationship consistency and relative position consistency of adjacent structures as in-depth discrimination criteria. For each candidate position in the candidate spatial solution set, the system not only examines the matching fit of stable and usable regions, but also extends the evaluation scope to the overall anatomical topology inside and outside the section. The system extracts the residual contours of low-confidence regions in the ultrasound image and the boundaries of adjacent synovial hyperplasia soft tissue, and maps them to the magnetic resonance space through the transformation matrix of the current candidate solution. The system calculates the spatial overlap rate between these mapped degenerate contours and the actual lesion regions in the magnetic resonance space. If a candidate solution is the true anatomical corresponding position, then even in low-confidence regions, the pathological absence or abnormal hyperplasia reflected by ultrasound should be highly consistent with the lesion distribution reflected by magnetic resonance in spatial topology. To quantify these judgment criteria, the system constructs a comprehensive scoring function. The function comprises three core terms: the first is a local similarity fidelity term based on stable and usable regions, used to measure the alignment accuracy of healthy anatomical structures; the second is a consistency term based on the relative positions of neighboring structures, used to penalize candidate solutions that disrupt the normal spatial topological relationships of the synovium, cartilage, and cortical bone; and the third is a pathological distribution consistency term for degenerative regions, used to reward poses that make ultrasound pathological features coincide with magnetic resonance pathological features in space. The system calculates a comprehensive score for all elements in the candidate spatial solution set. By comparing the comprehensive scores, the system can effectively distinguish which candidate solutions are supported by true anatomical correspondences and which candidate solutions are merely pseudo-correspondences generated by accidental geometric approximations of locally incomplete contours. The system selects the candidate solution with the highest comprehensive score as the final valid candidate solution, thereby uniquely determining the three-dimensional position and pose of the ultrasound section in magnetic resonance space. This process completely resolves the problem of multiple solutions for spatial localization caused by degenerative feature coupling, restoring the uniqueness and reliability of spatial mapping.
[0039] In the steps of mapping the first modal image to the second modal image based on the target spatial pose, and performing local deformation correction on the intermodal mapping boundary under the target spatial pose according to the distribution characteristics of low-confidence feature regions to obtain the mapped local target regions, after uniquely determining the three-dimensional position of the ultrasound section, the system needs to map the synovial thickness measurement value and other lesion features in the ultrasound image to the magnetic resonance statistical space. However, since low-confidence regions are downweighted or excluded during the spatial localization stage, these regions may still have local mapping deviations or slight misalignments under the determined global pose. If the ultrasound features are directly and rigidly projected, it may cause millimeter-level friction or boundary crossing between the local lesion boundary and the magnetic resonance anatomy. Therefore, the system must strictly limit and locally correct the mapping boundary based on the distribution of low-confidence regions. The system first projects the pixel grid of the ultrasound image completely onto the three-dimensional voxel space of the magnetic resonance image according to the inverse solution of the section position, forming an initial mapping mask. Subsequently, the system retrieves the previously identified set of low-confidence regions and marks the local spatial range corresponding to these degenerate references on the initial mapping mask. For these marked local areas, the system initiates an elastic relaxation correction mechanism. Using the mapping results of stable, usable regions as rigid anchors, the system allows non-rigid deformation of the mapping mesh in low-confidence regions within a very small range. The driving force for this deformation comes from the local fine-tuning matching of the ultrasonic synovial boundary pixels with the soft tissue contrast gradient on the corresponding MRI cross-section. Through this local elastic correction, the system fine-tunes the mapping coordinates of degraded regions, making them more closely match the actual soft tissue boundaries in the MRI image, thereby compensating for the local positioning uncertainty caused by bone reference degradation. Simultaneously, the system performs boundary constraint operations. For extreme regions that are severely degraded in both modes and completely lose their structural correspondence, the system assesses mapping risk by calculating the divergence of the local spatial deformation field. If the deformation divergence of a local region exceeds a safety threshold, indicating that the mapping of that region is extremely unreliable, the system will forcibly shrink the mapping boundary of that region or perform a smooth transition, preventing the ultrasonic synovial thickness value from projecting beyond the boundary to locations in the MRI that clearly belong to healthy cortical bone or unrelated joint recesses. By using this targeted correction for the effects of degraded areas, the system effectively prevents the ultrasonic synovial thickness measurement from being projected to an erroneous location drawn by the degraded reference, ensuring the spatial accuracy of the lesion distribution map in local details and blocking the transmission of local mapping benchmark drift to subsequent statistical analysis.
[0040] In the process of assigning weight labels to each local target region based on the evaluation results, and performing weighted joint statistical analysis on the lesion features in the first and second modal images based on these weight labels to comprehensively assess the severity of rheumatoid arthritis, the system formally enters the multi-image joint statistical analysis and lesion quantification fusion stage after receiving the mapping result package. The task of this stage is to perform joint probabilistic modeling of the synovial thickness value measured by ultrasound with the synovial signal intensity, bone marrow edema range, and bone erosion volume extracted by magnetic resonance within a unified magnetic resonance statistical space, in order to comprehensively assess the severity of rheumatoid arthritis. The system reads the spatial coordinates, synovial thickness value, and corresponding mapping confidence label from the mapping result package. When constructing the joint statistical model, the system introduces the mapping confidence label as a prior weight for the statistical samples. The system establishes a weighted spatial mixture effect model, inputting the ultrasound synovial thickness and magnetic resonance signal intensity at each mapping location as multivariate observations into the model. For high-confidence regions where the mapping confidence level approaches one, the system assigns extremely high statistical weights to the observed values, allowing them to play a dominant role in estimating the mean, variance, and cross-modal correlation coefficient of synovial hyperplasia. These high-confidence data constitute the core benchmark of the joint statistical model. For low-confidence regions with lower mapping confidence levels, the system performs weight reduction processing, significantly reducing the impact of these observed values on the estimation of global statistical parameters. If the mapping confidence level of a local region is lower than the system's set safety threshold, the system will perform a restrictive statistical operation on that region, i.e., it will not calculate the joint probability distribution of ultrasound and MRI in that region, but will only retain the lesion measurement results of the MRI single-modal model, and mark that region as a manual review area. Through this confidence-based triage and weighting mechanism, the system effectively avoids treating highly ambiguous mapping results and normal mapping results with equal weight, preventing the accumulation and amplification of errors caused by spatial mismatch in the statistical model. Finally, based on the optimized joint statistical model, the system generates a cross-modal lesion quantification map containing the degree of synovial hyperplasia, the distribution of bone erosion, and their joint pathological probabilities. This quantitative map is displayed overlaid on the patient's original 3D MRI anatomical image using a visualization rendering engine with different pseudo-colors and transparency. Simultaneously, the system generates a detailed lesion assessment report, recording the quantitative indicators of each joint recess and the distribution of low-confidence areas due to degeneration. This final output not only provides clinicians with accurate, reliable, and confidence-indicating multimodal joint diagnostic evidence but also fully realizes a closed-loop technology chain from degeneration identification and disambiguation to statistical fusion, completely overcoming the interference of rheumatoid arthritis pathological degeneration on multi-image joint statistical analysis.
[0041] The following example will provide a more detailed explanation of the above technical solution: Suppose a patient presents with knee pain, and the doctor suspects rheumatoid arthritis, ordering both ultrasound and MRI scans. In traditional assessment methods, doctors typically attempt to match bony boundaries in the ultrasound image with corresponding structures in the MRI image to determine the location of the ultrasound section in MRI space. However, if the patient's knee joint has lesions such as bone erosion, bony markers in the ultrasound image may be blurred due to sound wave scattering and occlusion, and the same area in the MRI image may also exhibit signal inhomogeneity due to pathological changes. This asymmetric visibility degradation in the dual-modal context leads to the loss of key local bony references. Consequently, localization algorithms based on a single similarity metric cannot obtain a unique matching solution when searching for the optimal location due to marker degradation, resulting in multiple candidate spatial poses and mapping ambiguity. When the algorithm mistakenly selects a non-true section location from multiple candidate poses, the ultrasound synovial region may be incorrectly mapped to other joint recess locations in the MRI statistical space. This spatial positioning error leads to anatomical mismatch in the subsequent joint statistical analysis of synovial thickness and magnetic resonance signals, thereby obscuring or fabricating the true relationship between the lesion and the anatomical structure, seriously affecting the accuracy and reliability of clinical assessment.
[0042] To address this, this application proposes an image recognition-based method for assessing rheumatic joint disease. Specifically, the method first acquires a first modal image (ultrasound image) and a second modal image (magnetic resonance image) containing the target monitoring object. For example, the system receives a sequence of ultrasound images of the patient's knee joint and three-dimensional magnetic resonance imaging volume data. These raw data undergo preprocessing, such as ultrasound image denoising and magnetic resonance image bias field correction, to standardize image quality and unify the coordinate system.
[0043] Subsequently, the system extracts local reference features from the first and second modal images, respectively. For example, it extracts two-dimensional geometric features of bony boundaries from ultrasound images, including the spatial location, orientation, and curvature of boundary points; and extracts three-dimensional surface geometric features of bony structures from magnetic resonance images, including the spatial location of surface vertices, normal direction, and curvature. These features provide the foundation for subsequent cross-modal matching.
[0044] Next, the system performs a dual-modal joint degradation assessment of local reference features to obtain evaluation results, and based on these results, identifies high-confidence and low-confidence feature regions. For example, the system evaluates the morphological integrity and signal continuity of bony boundaries in ultrasound images, and the signal homogeneity and structural smoothness of bony structures in magnetic resonance images. Simultaneously, it assesses the consistency of expression of corresponding local reference features in both modalities. By calculating a joint degradation index, the system divides bony reference features into high-confidence regions (joint degradation index below a preset threshold) and low-confidence regions (joint degradation index above or equal to a preset threshold). This step effectively distinguishes regions whose reliability is reduced due to lesions, avoiding their introduction of errors in subsequent registration.
[0045] Then, using high-confidence feature regions as the primary spatial constraints, the system searches for multiple candidate spatial poses of the first modality image within the spatial coordinate system corresponding to the second modality image. For example, the system uses only the feature point set of high-confidence regions in the ultrasound image to match a series of virtual cross-sections generated in the magnetic resonance image. By calculating similarity scores, the system identifies multiple spatial poses with similarity scores not less than a preset threshold as candidates. Since interference from low-confidence regions is eliminated, this step can more accurately find potential matching locations.
[0046] Furthermore, the target spatial pose is determined by verifying each candidate spatial pose. For example, the system evaluates the consistency of the geometric relationship between high-confidence regions in the ultrasound image and the corresponding anatomical structures in the MRI image, the consistency of the pathological distribution between low-confidence regions and the corresponding lesion regions in the MRI image, and the consistency of the relative positions between adjacent soft tissue structures under each candidate pose. By weighted summing of these evaluation results, a comprehensive score is calculated for each candidate pose, and the candidate pose corresponding to the highest comprehensive score is determined as the target spatial pose. This multi-dimensional verification mechanism effectively resolves spatial positioning ambiguities caused by local feature degradation, ensuring the uniqueness and accuracy of the final positioning.
[0047] Subsequently, based on the target spatial pose, the first modal image is mapped onto the second modal image. According to the distribution characteristics of low-confidence feature regions, local deformation correction is performed on the intermodal mapping boundary under the target spatial pose to obtain the mapped local target regions. For example, the system completely projects the pixel grid of the ultrasound image onto the three-dimensional voxel space of the magnetic resonance image to form the initial mapping boundary. For the previously identified low-confidence feature regions, the system drives a deformable mesh or local deformation field for local optimization based on their geometric contours and signal intensity gradients. This makes the mapping boundary more accurately fit the real soft tissue boundary in the magnetic resonance image, compensating for the local positioning uncertainty caused by bone reference degradation.
[0048] Finally, based on the evaluation results, weight labels are assigned to each local target region. Based on these weight labels, a weighted joint statistical analysis is performed on the lesion features in the first and second modal images to comprehensively assess the severity of rheumatic joint disease. For example, the system assigns weight labels to each mapped local target region based on the previous bimodal joint degenerative assessment results. For lesion features in high-weight regions, their contribution is maintained or enhanced; for lesion features in low-weight regions, their contribution is reduced or their inclusion is restricted. Through this weighted processing, the system can more accurately statistically analyze lesion features such as ultrasound synovial thickness, magnetic resonance synovial signal intensity, bone marrow edema extent, and bone erosion volume, thereby comprehensively assessing the severity of rheumatic joint disease.
[0049] This application's technical solution achieves high-precision spatial registration and lesion assessment even in the context of blurred image features caused by lesions by introducing a dual-modal joint degradation assessment mechanism. Unlike traditional ultrasound sectional spatial localization techniques that heavily rely on the integrity of bony references, this application, after extracting local reference features, first performs a dual-modal joint degradation assessment on these features to identify high-confidence and low-confidence feature regions. This innovative step effectively solves the problem in traditional methods where pathological degradation leads to the loss of key local bony references, resulting in spatial localization ambiguity.
[0050] For example, in the aforementioned patient knee joint lesion assessment, traditional methods may result in blurred or discontinuous bony landmarks in ultrasound and MRI images due to bone erosion, leading to multiple similar candidate spatial poses and making it difficult to determine the true ultrasound cross-sectional location. This application, through dual-modal joint degradation assessment, can clearly distinguish which bony regions are reliable (high confidence) and which are damaged (low confidence). When searching for candidate spatial poses, this application uses only high-confidence feature regions as the primary spatial constraints, effectively avoiding interference from low-confidence regions and thus improving the accuracy of preliminary localization.
[0051] Furthermore, this application determines a unique target spatial pose by performing multi-dimensional verification on each candidate spatial pose, including geometric consistency, pathological distribution consistency, and relative position consistency, and calculating a comprehensive score. Compared to traditional methods that rely on only a single similarity metric, this application's verification mechanism can more comprehensively and robustly evaluate the reliability of candidate poses, completely resolving the problem of multiple interpretations and ambiguity. In the above example, even if there are multiple candidate poses with high local similarity, this application can accurately identify the target pose that best matches the actual anatomical structure and lesion distribution through comprehensive evaluation, avoiding the mismapping of the ultrasound synovial region to other joint recess locations in the magnetic resonance statistical space.
[0052] Furthermore, during the mapping process, this application performs local deformation correction on the mapping boundaries between modalities based on the distribution characteristics of low-confidence feature regions, ensuring the spatial accuracy of the lesion distribution map in local details. Finally, by assigning weight labels to each local target region and performing weighted joint statistical analysis, this application effectively avoids treating highly ambiguous mapping results and normal mapping results with equal weight, preventing the accumulation and amplification of errors caused by spatial mismatch in the statistical model. This makes the final lesion assessment results more accurate and reliable, providing clinicians with multimodal joint diagnostic evidence with confidence level indications. The overall technical concept of this application, through refined processing and intelligent assessment of multimodal images in the context of lesions, significantly improves the accuracy and reliability of rheumatic joint lesion assessment, overcomes the limitations of existing technologies in processing complex pathological images, and has made a significant technical contribution.
[0053] In some embodiments, the specific steps in step S2 include: S21. Extract the two-dimensional geometric features of the bony boundary from the ultrasound image as local reference features; the two-dimensional geometric features include the spatial location of the boundary point, the orientation of the boundary point, and the degree of curvature of the boundary point; S22. Extract the three-dimensional surface geometric features of the bony structure from the magnetic resonance image as local reference features; the three-dimensional geometric features include the spatial position of the surface vertex, the normal direction of the surface vertex, and the curvature of the surface vertex.
[0054] Specifically, extracting two-dimensional geometric features of bony boundaries from ultrasound images refers to identifying and quantifying the geometric properties of bone edges in ultrasound images, providing bone contour information in a two-dimensional plane as a reference for subsequent spatial localization. This feature extraction can be achieved by using edge detection algorithms (e.g., Canny, Sobel, or phase consistency algorithms) to identify regions in the image with drastic grayscale or phase changes, and then refining them into bony boundaries through morphological operations (e.g., dilation, erosion, skeleton extraction); alternatively, deep learning models (e.g., U-Net, Mask R-CNN) can be used to perform semantic segmentation on the ultrasound image, directly outputting a pixel-level mask of the bony boundaries, from which geometric features can then be extracted. The spatial location of boundary points refers to the specific coordinates of each pixel or sampling point on the bony boundary in the image coordinate system. This is the fundamental data constituting the bony boundary contour, used to describe the overall shape and relative position of the boundary. The location information can be obtained by traversing the binary image output by the edge detection algorithm and recording the coordinates of all bony boundary pixels; alternatively, it can be obtained by curve fitting (e.g., B-spline curve, Bézier curve) to the bony boundary and then sampling points at equal intervals from the fitted curve. The direction of the boundary points refers to the tangent or normal direction at each point on the bony boundary in the ultrasound image. It describes the orientation and shape of the bony boundary in the local region and helps distinguish boundary segments of different shapes. This direction can be approximated by calculating the gradient direction in the neighborhood of the boundary point; alternatively, it can be obtained by performing local polynomial fitting on the boundary point sequence and then differentiating the derivative. The curvature of the boundary points refers to the curvature at each point on the bony boundary in the ultrasound image. It reflects the degree of curvature of the boundary at that point and is an important parameter describing the local morphology of the bony boundary, crucial for identifying features such as bulges and depressions in bones. The degree of curvature can be obtained by calculating the second derivative of the boundary point sequence or by using discrete curvature estimation methods (e.g., curvature calculation based on a three-point circle); alternatively, it can be obtained by fitting a local curve to the boundary and then calculating the curvature from the mathematical expression of the fitted curve.
[0055] Simultaneously, extracting the three-dimensional surface geometric features of bony structures from magnetic resonance images refers to identifying and quantifying the geometric properties of the three-dimensional surface of bones (e.g., femoral condyles, tibial plateau) from magnetic resonance images, providing bone surface information in three-dimensional space for subsequent spatial localization. This feature extraction can be achieved by using image segmentation algorithms (e.g., level set methods, graph cut algorithms, deep learning segmentation models) to segment the magnetic resonance image into three-dimensional bones, and then extracting the three-dimensional surface mesh model from the segmentation results; alternatively, multi-scale feature extractors (e.g., 3D SIFT, Hessian matrix features) can be used to directly detect key points on the bone surface in the magnetic resonance image and extract their local geometric descriptors. The spatial position of surface vertices refers to the specific coordinate values of each vertex on the three-dimensional surface mesh model of the bony structure in the magnetic resonance image in a three-dimensional voxel coordinate system. This is the fundamental data constituting the three-dimensional bone surface model, used to describe the overall three-dimensional morphology and spatial distribution of the bone. This location information can be obtained directly from the voxel coordinates or mesh vertex coordinates output by the 3D segmentation algorithm; alternatively, it can be obtained by parametrically representing the 3D surface (e.g., parametric surface) and then sampling points from the parametric surface to obtain its spatial position. The normal direction of a surface vertex refers to the surface normal vector at each vertex of the 3D surface mesh model of the bony structure in the magnetic resonance image. It describes the orientation of the bone surface in a local region and is crucial for identifying surface features such as concavity, convexity, and slope. This normal direction can be estimated by calculating the average normal vector of the triangular faces in the vertex's neighborhood or using principal component analysis (PCA); alternatively, it can be calculated by fitting a local surface to the 3D surface and then extracting the normal vector from the mathematical expression of the fitted surface. The curvature of a surface vertex refers to the Gaussian curvature, mean curvature, or principal curvature at each vertex of the 3D surface mesh model of the bony structure in the magnetic resonance image. It reflects the degree of curvature of the surface at that point and is an important parameter describing the local morphology of the 3D bone surface, helping to identify key anatomical structures such as articular surfaces and edges of the bone. The curvature can be estimated by calculating the geometric properties of the mesh in the vertex neighborhood (e.g., angles, side lengths); or it can be calculated by fitting a local surface to the three-dimensional surface and then calculating the curvature from the mathematical expression of the fitted surface.
[0056] This application's solution constructs a cross-modal spatial alignment geometric benchmark by extracting differentiated features from images of different modalities. For ultrasound images, leveraging their sensitivity to bony boundaries, two-dimensional geometric features are extracted. By analyzing the spatial position, direction, and curvature of boundary points, the bony contours under the ultrasound cross-section are accurately depicted, providing geometric constraints based on a two-dimensional plane for subsequent pose search. For magnetic resonance imaging (MRI) images, utilizing their three-dimensional imaging advantages, three-dimensional surface geometric features of bony structures are extracted. By analyzing the spatial position, normal direction, and curvature of surface vertices, the three-dimensional morphology of joint bony structures is fully described. This method of extracting corresponding two-dimensional boundary features from ultrasound and three-dimensional surface features from MRI solves the matching problem caused by the inconsistency of data dimensions across different modalities. It ensures that subsequent joint degradation assessments can be compared based on physically meaningful bony structural features, thereby effectively improving the accuracy and robustness of cross-modal mapping. In this way, in step S2, this application extracts modally specific yet geometrically comparable local reference features for both ultrasound and magnetic resonance images, laying a solid foundation for the subsequent bimodal joint degradation assessment in step S3 and the spatial pose search in step S4. This differentiated and complementary feature extraction strategy effectively overcomes the inherent differences between ultrasound and magnetic resonance images in imaging mechanisms and data dimensions. Even under pathological conditions, where the bony reference exhibits local degradation, reliable cross-modal correspondences can still be established through these carefully designed geometric features, thereby avoiding assessment and localization errors caused by feature mismatch.
[0057] As a specific implementation method, the system first applies a phase-consistency-based edge detection algorithm to ultrasound images. Unlike traditional edge detection based on gray-level gradients, phase consistency is insensitive to changes in local image contrast and can more stably extract bony boundaries in ultrasound images where brightness is uneven due to scattering and attenuation. The extracted bony boundaries are represented as a two-dimensional point cloud set, and the system further calculates the local normal vector, principal curvature, and shape context-based local geometric descriptor for each boundary point. On the magnetic resonance imaging data side, the system uses a three-dimensional level set-based method to perform three-dimensional segmentation of the femoral condyle, tibial plateau, and patellar cortical bone surfaces of the knee joint, extracting a continuous three-dimensional bone surface mesh model. For each vertex on this three-dimensional model, the system also calculates its three-dimensional normal vector, Gaussian curvature, mean curvature, and three-dimensional shape context descriptor.
[0058] Through the above technical solutions, this application constructs a cross-modal spatial alignment geometric benchmark by extracting differentiated features from images of different modalities. For ultrasound images, leveraging their sensitivity to bony boundaries, two-dimensional geometric features are extracted. By analyzing the spatial position, direction, and curvature of boundary points, the bony contour under the ultrasound section is accurately depicted, providing geometric constraints based on a two-dimensional plane for subsequent pose search. For magnetic resonance imaging (MRI) images, utilizing their three-dimensional imaging advantages, three-dimensional surface geometric features of bony structures are extracted. By analyzing the spatial position of surface vertices, normal direction, and curvature, the three-dimensional morphology of joint bony structures is fully described. This method of correspondingly extracting two-dimensional boundary features from ultrasound and three-dimensional surface features from MRI solves the matching problem caused by inconsistent data dimensions across different modalities. It ensures that subsequent joint degradation assessments can be compared based on physically meaningful bony structural features, thereby effectively improving the accuracy and robustness of cross-modal mapping.
[0059] In some embodiments, the specific steps in step S3 include: S31. Evaluate the morphological integrity and signal continuity of the corresponding local reference features in the first modality image, and the signal uniformity and structural smoothness of the corresponding local reference features in the second modality image, respectively, to obtain the single-modality evaluation results; S32. Evaluate the consistency between the representation of the corresponding local reference features in the first modality image and the corresponding local reference features in the second modality image to obtain the intermodal consistency evaluation result; S33. Based on the single-modal evaluation results and the intermodal consistency evaluation results, the evaluation results are obtained by calculating the joint degradation index of the local reference features; S34. Based on the evaluation results, and according to the joint degradation index and the preset degradation threshold, identify the high-confidence feature regions and low-confidence feature regions in the local reference features.
[0060] This scheme achieves refined quantification of image feature quality by constructing a multi-dimensional evaluation system. First, by separately evaluating the morphological integrity and signal continuity of ultrasound images, and the signal homogeneity and structural smoothness of magnetic resonance images, it can identify local signal degradation or structural loss caused by pathological changes from a single-modal perspective, laying the foundation for subsequent evaluation. Second, by introducing intermodal consistency evaluation, comparing the expression differences of the same anatomical structure in two modalities, it can effectively identify asymmetric degradation caused by different imaging mechanisms, thereby more accurately defining the reliability of features. Based on this, by calculating a joint degradation index, the quality indicators of single modalities are organically integrated with the matching consistency between modalities, comprehensively reflecting the true usability of local reference features. Finally, based on the comparison between the joint degradation index and the preset threshold, the system automatically identifies and classifies high-confidence and low-confidence feature regions. This process not only solves the bias caused by a single evaluation dimension, but also provides a reliable constraint for subsequent spatial pose search. It ensures that in the context of pathological degradation, the system can prioritize the use of high-quality features for localization and perform targeted processing on low-quality regions, thereby significantly improving the robustness of cross-modal mapping and the reliability of clinical evaluation.
[0061] Specifically, in step S31, the morphological integrity and signal continuity of the corresponding local reference features in the first modality image (ultrasound image) and the signal uniformity and structural smoothness of the corresponding local reference features in the second modality image (magnetic resonance image) are evaluated to identify the degradation status of features within their respective modalities. Morphological integrity and signal continuity refer to whether the geometry of the bony boundary in the ultrasound image is complete and continuous, and whether its echo signal is stable and uninterrupted. This can be achieved by calculating the interruption frequency and variance of the echo intensity of the local boundary within a fixed-length window; high interruption frequency or high variance indicates incomplete morphology or signal discontinuity. Alternatively, the smoothness and continuity of the signal can be determined by analyzing the topological structure of the bony boundary, such as using morphological operators to detect boundary breakpoints or cavities and quantifying their number and size; or by calculating the local gray-level gradient change rate of the boundary pixels. Signal uniformity and structural smoothness refer to whether the intensity distribution of the bone cortex signal in the magnetic resonance image is uniform, and whether the bone surface structure is smooth without obvious depressions or protrusions. This can be achieved by calculating the normal divergence of the local bone surface and the probability of abnormally high signal intensity in the adjacent medullary cavity. High normal divergence or abnormally high signal intensity indicates structural unevenness or signal inhomogeneity. Alternatively, signal uniformity can be assessed by calculating the statistical characteristics of the gray-level histogram of the local region (such as standard deviation and entropy); or by performing curvature analysis on the bone surface, such as calculating local variations in Gaussian curvature or mean curvature, to quantify structural smoothness.
[0062] In step S32, the expression consistency between the corresponding local reference features in the first modality image and the corresponding local reference features in the second modality image is evaluated. This aims to identify intermodal inconsistencies caused by different imaging principles or asymmetric pathological manifestations. Expression consistency refers to the degree of similarity between local reference features in ultrasound images and corresponding local reference features in magnetic resonance images in terms of geometric morphology, topology, or pathological manifestations. This can be quantified by projecting the two-dimensional bony boundary in the ultrasound image onto the three-dimensional space of the magnetic resonance image, and then calculating the geometric distance or shape similarity between the projected two-dimensional boundary and the corresponding three-dimensional bone surface model slice in the magnetic resonance image. Alternatively, it can be evaluated through feature descriptor matching. For example, shape context descriptors or Local Binary Pattern (LBP) features of corresponding anatomical structures in the two modalities can be extracted, and then the distance between these feature descriptors (such as Euclidean distance or cosine similarity) can be calculated as a consistency measure.
[0063] In step S33, based on the single-modal evaluation results and the intermodal consistency evaluation results, the evaluation result is obtained by calculating the joint degradation index of the local reference features. The joint degradation index is a comprehensive quantitative indicator used to reflect the overall degradation degree of local reference features in the bimodal image. Its calculation can comprehensively consider the boundary continuity score on the ultrasound side, the signal uniformity score on the magnetic resonance side, and the consistency of their expression forms. This can be a weighted sum model, for example: Joint degradation index = w1*(ultrasound boundary interruption frequency) + w2*(ultrasound echo intensity variance) + w3*(magnetic resonance normal vector divergence) + w4*(magnetic resonance abnormal high signal probability) + w5*(intermodal geometric distance), where w1, w2, w3, w4, and w5 are preset weights. Alternatively, a machine learning model, such as a support vector machine (SVM) or a neural network, can be used, with the single-modal evaluation results and the intermodal consistency evaluation results as input features, to train the model and output a continuous degradation index.
[0064] In step S34, based on the evaluation results, high-confidence and low-confidence feature regions in the local reference features are identified according to the joint degradation index and a preset degradation threshold. The preset degradation threshold is a critical value used to distinguish the confidence level of feature regions. This threshold can be dynamically adjusted according to the statistical distribution of the training dataset (such as the mean and standard deviation of the joint degradation index), or learned based on the degree of degradation annotated by clinical experts. Alternatively, it can be a fixed threshold, pre-set by experienced clinicians or image processing experts based on a large amount of case data. The identification process involves classifying feature points with a joint degradation index below the threshold into a stable usable region set (high-confidence feature regions), and classifying feature points with a joint degradation index above or equal to the threshold into a low-confidence region set (low-confidence feature regions). Alternatively, fuzzy logic or clustering algorithms can be used to divide feature points into multiple confidence levels based on the joint degradation index, then classifying the highest level as a high-confidence region and the rest as low-confidence regions.
[0065] This approach, through its multi-dimensional and comprehensive evaluation mechanism, addresses the challenge of traditional methods that rely solely on a single dimension to comprehensively capture complex image features. It accurately distinguishes between structural defects caused by actual pathological flaws and conventional apparent defects, thus providing reliable constraints for subsequent steps (such as spatial pose search in S4 and intermodal mapping boundary correction in S6). When combined with steps S1 and S2, this approach ensures effective quality screening of extracted features, allowing subsequent spatial localization and mapping processes to prioritize high-quality, high-confidence features while addressing low-confidence regions. This significantly improves the robustness of cross-modal mapping and the reliability of clinical assessment.
[0066] The following is a specific example to illustrate this. In step S31, for the first modality image (ultrasound image), the system can traverse the extracted two-dimensional bony boundary point cloud. For each local boundary segment, the system calculates its interruption frequency within a fixed-length window, for example, by detecting whether the distance between adjacent points in the boundary point sequence exceeds a preset threshold to determine interruption. Simultaneously, the system calculates the variance of the echo intensity within this window to quantify signal fluctuation. A high interruption frequency or high variance value indicates poor morphological integrity or signal continuity of the ultrasound local reference feature. For the second modality image (magnetic resonance image), the system evaluates the uniformity of the bone cortex signal and surface smoothness along the candidate region corresponding to the initial search space on the extracted three-dimensional bone surface model. Specifically, the system can calculate the normal vector of each vertex on the local bone surface and statistically analyze the divergence of the normal vectors in its neighborhood; a larger divergence indicates a less smooth surface. Furthermore, the system can analyze whether there are abnormally high signal regions (such as bone marrow edema) in the adjacent medullary cavity and calculate their probability. These calculation results collectively constitute the single-modality evaluation result. In step S32, to assess intermodal consistency, the system can project the two-dimensional bony boundaries extracted from the ultrasound image into the three-dimensional space of the magnetic resonance image. Then, the system calculates the average Euclidean distance between the projected two-dimensional boundary and the corresponding three-dimensional bone surface model slice in the magnetic resonance image. The smaller the distance, the higher the intermodal consistency. Furthermore, the geometric consistency can be further quantified by comparing the local curvature of the ultrasound boundary points with the local curvature of the corresponding points on the magnetic resonance surface. In step S33, the system calculates the joint degradation index of local reference features based on the above single-modal evaluation results and intermodal consistency evaluation results. For example, a linear weighted model can be used: Joint degradation index = w_us * (ultrasound interruption frequency + ultrasound echo variance) + w_mr * (magnetic resonance normal divergence + magnetic resonance abnormal high signal probability) + w_inter * (intermodal geometric distance). Here, w_us, w_mr, and w_inter are weighting coefficients obtained based on clinical experience or machine learning training, used to balance the contributions of different evaluation dimensions to the degradation index. In step S34, the system can set a dynamically adaptive degradation threshold. This threshold can be determined based on the overall joint lesion severity of the current patient or through statistical analysis of a large amount of patient data. For example, it can be set as a percentile (e.g., the 75th percentile) of the distribution of the joint degradation index of all local reference features. Then, the system identifies all local reference features with a joint degradation index below the threshold as high-confidence feature regions, while identifying local reference features with a joint degradation index above or equal to the threshold as low-confidence feature regions. In this way, the system can automatically and objectively classify local reference features, providing a basis for subsequent accurate matching and correction.
[0067] Through the aforementioned technical solution, this application's approach constructs a multi-dimensional evaluation system, achieving refined quantification of image feature quality. This effectively solves the problem that traditional methods, under pathological degradation, struggle to comprehensively capture complex image features using only a single dimension, leading to inaccurate feature region segmentation. This multi-dimensional evaluation method makes the judgment of the degradation state of local reference features more comprehensive and accurate. Furthermore, by calculating a joint degradation index, the quality indicators of a single modality are organically integrated with the matching consistency between modalities, comprehensively reflecting the true usability state of local reference features. Finally, based on the comparison between the joint degradation index and a preset threshold, automatic identification and classification of high-confidence and low-confidence feature regions are achieved. This process not only resolves the bias caused by a single evaluation dimension but also provides a reliable constraint for subsequent spatial pose search, ensuring that under pathological degradation, the system can prioritize the use of high-quality features for localization and perform targeted processing on low-quality regions. Therefore, in step S3, this scheme performs a dual-modal joint degradation assessment of local reference features and identifies high-confidence and low-confidence feature regions, which significantly improves the accuracy and robustness of spatial pose search in the subsequent step S4, avoids mismatch and mapping ambiguity caused by degradation features, and thus ensures the accuracy and reliability of the entire rheumatic joint disease assessment method.
[0068] In some embodiments, the specific steps in step S4 include: S41. Extract a set of feature points from the high-confidence feature regions of the first modality image; S42. Generate a series of virtual cross planes in the spatial coordinate system corresponding to the second modality image; S43. Match the virtual section with the feature point set based on feature correspondence; S44. Based on the matching results, identify multiple spatial poses with a similarity of not less than a preset similarity threshold, and use them as candidate spatial poses.
[0069] The extraction of feature point sets from high-confidence feature regions of the first modality image aims to select only those regions in the ultrasound image deemed reliable and unaffected by pathology after dual-modality joint degradation assessment, extracting geometric feature points for subsequent matching. This ensures that the matching process is based on stable, anatomically significant data, avoiding interference with localization accuracy caused by feature distortion or loss due to lesions. This feature point set can be generated using image processing-based corner detection algorithms, such as the Harris corner detector or the Shi-Tomasi corner detector, to identify pixels with significant local grayscale changes within the high-confidence feature regions of the ultrasound image. Alternatively, a pre-trained deep learning model, such as a convolutional neural network-based feature extractor, can be used to analyze the high-confidence regions and automatically identify and extract representative geometric feature points, such as key points on bony boundaries.
[0070] A series of virtual cross-sections are generated within the spatial coordinate system corresponding to the second modality image. The purpose is to simulate the two-dimensional cross-sections that the ultrasound probe might scan in the three-dimensional magnetic resonance image space, thereby transforming the complex search problem in three-dimensional space into a matching problem between two-dimensional cross-sections and feature point sets, reducing computational complexity and providing a clear geometric reference. These virtual cross-sections can be generated by parametrically sampling the three-dimensional voxel space of the magnetic resonance image. For example, within a preset translation and rotation range, a series of planar equations are generated with a fixed step size, each plane representing a virtual cross-section. Alternatively, based on an initial coarse pose estimate, an adaptive sampling strategy can be used to generate a series of virtual cross-sections in the magnetic resonance image. For example, denser sampling can be performed near the estimated ultrasound cross-section, while sparser sampling can be performed in regions far from the estimated position.
[0071] Matching virtual cross-sections with feature point sets based on feature correspondence aims to evaluate the geometric similarity between each virtual cross-section and the feature point set extracted from the ultrasound image, thereby quantifying the potential projection position of the ultrasound image in magnetic resonance space. Matching based on feature correspondence directly reflects the geometric fit between the ultrasound imaging plane and the magnetic resonance anatomical structure. This matching can be evaluated by calculating the geometric distance (e.g., the average point-to-line distance or Hausdorff distance) and normal vector consistency between the intersection of the ultrasound feature point set and the projection of the virtual cross-section onto the magnetic resonance bone surface model. Alternatively, the Iterative Closest Point (ICP) algorithm or its variants can be used to register the ultrasound feature point set with the projection intersection of the virtual cross-section onto the magnetic resonance bone surface model to minimize their geometric errors, thus obtaining a matching score.
[0072] Based on the matching results, multiple spatial poses with similarity no less than a preset similarity threshold are identified and used as candidate spatial poses. The aim is to filter out spatial positions and poses with high similarity to the ultrasonic feature point set from a large number of virtual cross-sections, forming a candidate set. This helps narrow the search range for subsequent refined localization and provides high-quality input for resolving multiple-solution ambiguity problems. This identification can be achieved by sorting the similarity scores obtained after matching each virtual cross-section with the feature point set, and selecting the spatial poses corresponding to all virtual cross-sections with scores higher than the preset similarity threshold as candidate spatial poses. Alternatively, all local maxima points can be identified in the parameter space of the similarity scores, and the spatial poses corresponding to these local maxima points can be used as candidate spatial poses, while ensuring that the similarity of these local maxima points is no less than the preset similarity threshold.
[0073] This application's solution introduces a virtual cross-section matching mechanism, providing a clear geometric benchmark for cross-modal spatial pose search. First, a feature point set is extracted from the high-confidence feature region of the first modality image, ensuring the data source upon which subsequent matching calculations rely has high anatomical stability and avoiding interference from noise in low-confidence regions. Second, a series of virtual cross-sections are generated within the spatial coordinate system corresponding to the second modality image, transforming the complex search problem in three-dimensional space into a matching problem between two-dimensional cross-sections and feature point sets, effectively reducing the computational dimensionality. By matching the virtual cross-sections and feature point sets based on feature correspondences, the projection position of the ultrasound image in magnetic resonance space can be quantitatively evaluated. This feature correspondence-based matching method directly reflects the geometric fit between the ultrasound imaging plane and the magnetic resonance anatomical structure. Finally, by setting a similarity threshold, multiple candidate spatial poses are selected, not only narrowing the search range but also providing a high-quality candidate set for further verification and refined localization, ensuring that the localization process possesses both global search coverage and local matching accuracy. This scheme, closely integrated with step S3 (which involves performing a dual-modal joint degradation assessment on local reference features to obtain evaluation results, and identifying high-confidence and low-confidence feature regions based on these results), jointly solves the spatial localization challenge in the context of pathological degradation. By pre-identifying high-confidence feature regions in step S3, this scheme can focus on using this reliable anatomical information for preliminary spatial pose search, thereby avoiding interference from low-confidence regions (i.e., lesion areas) in the matching process. This combination enables the scheme to efficiently and comprehensively find all potential true corresponding locations even when facing feature degradation caused by lesions, overcoming the localization ambiguity and insufficient matching accuracy problems caused by feature degradation in traditional methods. Specifically, step S3 provides high-quality input for this scheme, allowing it to search in a "cleaner" feature space, thereby improving the efficiency and accuracy of the search.
[0074] As a specific implementation, when extracting feature point sets from the high-confidence feature regions of the first modality image (ultrasound image), the Scale Invariant Feature Transform (SIFT) algorithm or the Accelerated Robust Feature Transform (SURF) algorithm can be used to detect and describe these feature points. These algorithms can extract feature points that are robust to changes in image scale, rotation, and brightness, thereby enhancing the stability of cross-modal matching. When generating a series of virtual cross-sections in the spatial coordinate system corresponding to the second modality image (magnetic resonance image), a region of interest (ROI) containing the knee joint can be defined first. Then, within this ROI, a series of virtual planes can be generated in three orthogonal directions (e.g., sagittal, coronal, and transverse planes) with a translation step of 1 mm and a rotation step of 5 degrees. For each virtual cross-section, the system can calculate its intersection line with a pre-segmented surface model of bony structures (such as the femoral condyle and tibial plateau) in the magnetic resonance image. When matching the virtual cross-sections with the feature point set based on feature correspondence, a descriptor-based matching method can be used. For example, the descriptors of SIFT or SURF feature points extracted from ultrasound images are compared with the descriptors of corresponding points on the intersection line of virtual sections, using Euclidean distance or Hamming distance as similarity measures. To improve matching efficiency, the Random Sample Consensus (RANSAC) algorithm can be used to eliminate outlier matching point pairs. Based on the matching results, when identifying multiple spatial poses with similarity no less than a preset similarity threshold, a similarity threshold can be set. For example, the percentage of matching point pairs to the total number of feature points can be no less than 70%, or the root mean square error (RMSE) of the matching error can be lower than a certain preset value. All spatial poses corresponding to virtual sections that meet this condition are identified as candidate spatial poses.
[0075] Through the above technical solution, this application can efficiently and comprehensively search for multiple potential spatial poses of ultrasound images in the three-dimensional space corresponding to magnetic resonance images. Specifically, by extracting feature point sets using only high-confidence feature regions, the interference of pathologically degraded regions on the matching process is effectively avoided, significantly improving the reliability of features. Generating virtual cross-sections in the magnetic resonance space and performing matching transforms the complex three-dimensional search problem into two-dimensional matching, greatly reducing computational complexity and improving search efficiency. Identifying multiple candidate spatial poses with similarity no less than a preset similarity threshold provides high-quality input for subsequent refined verification, avoiding mismatches that may occur due to a single similarity measure in a pathological context. This effectively solves the problem of how to efficiently and accurately screen out candidate poses with practical clinical significance from a massive number of spatial possibilities when pathological degradation leads to complex feature point distribution, and establishes a reliable geometric correspondence between ultrasound images and magnetic resonance images, avoiding the problems of blind search or insufficient matching accuracy.
[0076] In some embodiments, the specific steps in step S5 include: S51. Evaluate the geometric relationship consistency between the high-confidence feature regions in the first modality image and the corresponding anatomical structures in the second modality image, and obtain the geometric relationship consistency evaluation result; S52. Evaluate the consistency of pathological distribution between low-confidence feature regions in the first modality image and the corresponding lesion regions in the second modality image to obtain the pathological distribution consistency evaluation results; S53. Evaluate the relative positional consistency between the soft tissue structure of the adjacent local reference feature in the first modality image and the corresponding soft tissue structure in the second modality image, and obtain the relative positional consistency evaluation result; S54. For each candidate spatial pose, calculate the comprehensive score corresponding to each candidate spatial pose based on the corresponding geometric relationship consistency assessment result, pathological distribution consistency assessment result, and relative position consistency assessment result; the specific steps include: for each candidate spatial pose, normalize the corresponding geometric relationship consistency assessment result, pathological distribution consistency assessment result, and relative position consistency assessment result, and obtain the comprehensive score by weighted summation of the normalized geometric relationship consistency assessment result, normalized pathological distribution consistency assessment result, and normalized relative position consistency assessment result based on preset weights; S55. Determine the candidate spatial pose corresponding to the highest comprehensive score as the target spatial pose.
[0077] The evaluation process assesses the geometric consistency between high-confidence feature regions in the first modality image and their corresponding anatomical structures in the second modality image. This assessment quantifies the degree of geometric matching between the geometry of identified high-confidence feature regions in the first modality image (ultrasound image) and their corresponding anatomical structures (such as cortical bone) in the second modality image (magnetic resonance image). Its purpose is to provide a fundamental verification of the accuracy of candidate spatial poses using reliable anatomical information. For example, geometric consistency can be assessed by calculating the average Hausdorff distance or point-to-surface distance between the high-confidence bony boundary point cloud in the ultrasound image and the surface projection of the corresponding bony structure in the magnetic resonance image. Alternatively, the evaluation can be obtained by comparing the similarity between the shape descriptors (such as principal component analysis axes and invariant moments) of the high-confidence feature regions in the ultrasound image and the shape descriptors of the corresponding anatomical structures in the magnetic resonance image.
[0078] The consistency of pathological distribution between low-confidence feature regions in the first modality image and corresponding lesion regions in the second modality image is evaluated to obtain a pathological distribution consistency assessment result. This result measures the degree of spatial agreement between the pathological features (such as synovial hyperplasia and bone erosion) reflected by low-confidence feature regions in the first modality image (ultrasound image) whose visibility has decreased due to pathological degeneration, and the actual lesion regions (such as synovitis, bone edema, and bone erosion) detected in the second modality image (magnetic resonance image) under the current candidate spatial pose. Its function is to use pathological information to impose secondary constraints on the candidate pose, especially providing auxiliary judgment when bony references are incomplete. For example, the spatial overlap rate can be assessed by calculating the Dice similarity coefficient or Jaccard index between the pathological features (such as synovial hyperplasia regions) contained in the mapped low-confidence regions in the ultrasound image and the corresponding lesion regions (such as enhanced synovium) in the magnetic resonance image. Alternatively, the pathological distribution consistency assessment results can be obtained by analyzing the correlation or mutual information between the abnormal signal intensity distribution in low-confidence areas of ultrasound images and the abnormal signal intensity distribution in the corresponding lesion areas of magnetic resonance images.
[0079] The relative positional consistency between soft tissue structures with adjacent local reference features in the first modality image and their corresponding soft tissue structures in the second modality image is evaluated to obtain a relative positional consistency assessment result. This aims to verify whether the spatial relative positional relationship between soft tissue structures (such as synovium and cartilage) adjacent to local reference features (such as bony boundaries) in the first modality image (ultrasound image) and their corresponding soft tissue structures in the second modality image (magnetic resonance image) remains consistent. Its function is to introduce anatomical information beyond bony structures, further enhancing the dimension of pose verification and ensuring the coherence and authenticity of the mapping results at the soft tissue anatomical level. For example, the distance error between the mapped soft tissue boundary (such as the synovial boundary) in the ultrasound image and the corresponding soft tissue boundary in the magnetic resonance image can be extracted and evaluated in conjunction with their geometric relationship relative to the bony reference. Alternatively, the relative positional consistency assessment result can be obtained by calculating the difference between the deformation field of the soft tissue region in the ultrasound image and the deformation field of the corresponding soft tissue region in the magnetic resonance image, or by comparing their relative position vectors in the local anatomical coordinate system.
[0080] For each candidate spatial pose, a comprehensive score is calculated based on the corresponding geometric consistency assessment results, pathological distribution consistency assessment results, and relative position consistency assessment results. This aims to integrate the assessment results from these three dimensions to generate a quantitative comprehensive evaluation index for each candidate spatial pose. Its function is to transform qualitative anatomical and pathological features into quantitative evaluation indicators, enabling the system to objectively select the optimal solution from multiple candidate poses. Normalization ensures the comparability of different assessment results across numerical ranges, while weighted summation allows adjustments to the contribution of different assessment dimensions to the final score. For example, normalization can employ linear normalization (Min-Max Scaling) to map the assessment results to the [0,1] interval. Preset weights can be obtained based on clinical experience or machine learning methods to reflect the importance of different assessment items. Alternatively, Z-score standardization can be used to convert the assessment results into a distribution with a mean of 0 and a standard deviation of 1. During weighted summation, weights can be set based on expert knowledge; for example, geometric consistency has a higher weight, followed by pathological distribution consistency, and then relative position consistency.
[0081] The final decision-making step involves identifying the candidate spatial pose corresponding to the highest overall score as the target spatial pose. Based on the overall score calculated in the previous step, the pose with the highest score among all candidate spatial poses is selected as the final target spatial pose. This process accurately pinpoints the unique true pose from multiple fuzzy candidate solutions, significantly improving the reliability of cross-modal image mapping and the accuracy of clinical assessment. For example, one can directly select the pose with the highest score by iterating through all candidate spatial poses and their corresponding overall scores. Alternatively, a score threshold or score difference threshold can be set when selecting the highest-scoring pose. If the difference between the highest and second-highest scores is less than a certain threshold, it may indicate the need for manual review or further refined matching.
[0082] This method first acquires a first-modal image and a second-modal image, and extracts local reference features from each. Then, through dual-modal joint degradation assessment, high-confidence and low-confidence feature regions are identified. Based on this, using the high-confidence feature regions as the primary spatial constraints, multiple candidate spatial poses of the first-modal image are searched within the spatial coordinate system corresponding to the second-modal image. To address the problem that relying solely on a single similarity metric often results in multiple candidate spatial poses leading to mapping ambiguity when pathological degradation causes feature blurring, this method introduces a multi-dimensional verification mechanism. Specifically, for each candidate spatial pose, the system not only examines the matching fit of stable and usable regions but also extends the evaluation scope to the overall anatomical topology inside and outside the cross-section. The system extracts the residual contours of low-confidence regions in the ultrasound image and the boundaries of adjacent synovial hyperplasia soft tissue, mapping them to the magnetic resonance space through the transformation matrix of the current candidate solution. The system calculates the spatial overlap rate between these mapped degraded contours and the actual lesion regions in the magnetic resonance space. If a candidate solution corresponds to a true anatomical location, then even in low-confidence regions, the pathological deficiencies or abnormal proliferations reflected by ultrasound should maintain a high degree of spatial topological consistency with the lesion distribution reflected by MRI. To quantify these criteria, the system constructs a comprehensive scoring function. This function contains three core terms: the first is a local similarity fidelity term based on stable and usable regions, used to measure the alignment accuracy of healthy anatomical structures; the second is a consistency term based on the relative positions of adjacent structures, used to penalize candidate solutions that disrupt the normal spatial topological relationships of the synovium, cartilage, and cortical bone; the third is a pathological distribution consistency term for degenerative regions, used to reward poses that make ultrasound pathological features coincide with MRI pathological features in space. The system calculates a comprehensive score for all elements in the candidate spatial solution set. By comparing the comprehensive scores, the system can effectively distinguish which candidate solutions are supported by true anatomical correspondences and which are merely spurious correspondences generated by accidental geometric approximations of local incomplete contours. The system selects the candidate solution with the highest comprehensive score as the final valid candidate solution, thus uniquely determining the three-dimensional position and orientation of the ultrasound section in MRI space. This process completely resolves the problem of multiple interpretations of spatial localization caused by degenerate feature coupling, restoring the uniqueness and reliability of spatial mapping.
[0083] In one specific implementation, during the determination of the target spatial pose, the system first receives multiple candidate spatial poses generated in step S4. For each candidate spatial pose, the system performs a series of refined checks. For example, when evaluating the consistency of the geometric relationship between high-confidence feature regions in the first modality image and the corresponding anatomical structures in the second modality image, the system can extract a two-dimensional point set of high-confidence bony boundaries from the ultrasound image and project it onto the three-dimensional space of the magnetic resonance image using the transformation matrix of the current candidate pose. Subsequently, the system can calculate the average Euclidean distance between these projected points and the pre-segmented cortical bone surface in the magnetic resonance image. The smaller the distance, the higher the geometric consistency. Simultaneously, when evaluating the consistency of the pathological distribution between low-confidence feature regions in the first modality image and the corresponding lesion regions in the second modality image, the system can identify synovial hyperplasia regions within the mapped low-confidence regions in the ultrasound image and perform spatial overlap analysis with the synovial inflammation regions identified by T2-weighted sequences in the magnetic resonance image. For example, the Dice similarity coefficient between the two can be calculated; the higher the coefficient, the better the consistency of the pathological distribution. Furthermore, when evaluating the relative positional consistency between soft tissue structures with adjacent local reference features in the first modality image and their corresponding soft tissue structures in the second modality image, the system can analyze the relative positional deviation between the mapped articular cartilage edge in the ultrasound image and the corresponding articular cartilage edge in the magnetic resonance image. For example, the average distance between the two in a direction perpendicular to the bone surface can be measured and evaluated in conjunction with their relative positional relationship with the bony reference. When calculating the comprehensive score corresponding to each candidate spatial pose, the system normalizes the above three evaluation results. For example, the average Euclidean distance, Dice similarity coefficient, and average distance deviation can be mapped to the range [0,1], respectively. Subsequently, the system can perform a weighted summation of the normalized results according to preset weights, for example, the weight of the geometric relationship consistency evaluation result can be set to 0.5, the weight of the pathological distribution consistency evaluation result can be set to 0.3, and the weight of the relative position consistency evaluation result can be set to 0.2, thereby obtaining the comprehensive score of each candidate spatial pose. Finally, the system compares the comprehensive scores of all candidate spatial poses and determines the candidate spatial pose with the highest score as the target spatial pose. For example, if the overall score of candidate pose A is 0.85, the overall score of candidate pose B is 0.78, and the overall score of candidate pose C is 0.89, then the system will determine candidate pose C as the target spatial pose.
[0084] Through the above technical solution, this application effectively solves the problem of feature blurring caused by pathological degradation in multimodal image mapping, leading to multiple candidate spatial poses and mapping ambiguities. Building upon the basic solution's identification of multiple candidate spatial poses, this solution introduces a multi-dimensional verification mechanism, constructing a comprehensive evaluation system from three levels: geometry, pathology, and relative soft tissue position. Specifically, by evaluating the consistency of the geometric relationship between high-confidence feature regions in the first modality image and the corresponding anatomical structures in the second modality image, the spatial matching accuracy of candidate poses based on bony structures is ensured, providing a robust anatomical benchmark for pose verification. This allows the system to perform preliminary screening based on undamaged reliable regions even when local bony references degenerate due to lesions. Furthermore, by evaluating the consistency of the pathological distribution between low-confidence feature regions in the first modality image and the corresponding lesion regions in the second modality image, this solution cleverly utilizes the distribution patterns of pathological features to impose secondary constraints on the pose. Even when bony landmarks are blurred, the consistency of lesion distribution can help determine the rationality of the pose, thus avoiding misjudgments caused by the degradation of a single geometric feature. Furthermore, by evaluating the relative positional consistency between soft tissue structures of adjacent local reference features in the first modality image and their corresponding soft tissue structures in the second modality image, this scheme introduces soft tissue spatial information beyond bony structures, further enhancing the dimensionality of pose verification and ensuring the coherence and authenticity of the mapping results at the soft tissue anatomical level, thereby improving the overall robustness of the mapping. By normalizing the above three evaluation results and calculating a weighted sum to obtain a comprehensive score, this scheme transforms qualitative anatomical and pathological features into quantitative evaluation indicators, enabling the system to objectively select the optimal solution from multiple candidate poses. Finally, the pose corresponding to the highest comprehensive score is determined as the target spatial pose, achieving the goal of accurately locking the unique true pose from multiple fuzzy candidate solutions. This significantly improves the reliability of cross-modal image mapping and the accuracy of clinical assessment, providing a more precise spatial basis for subsequent joint statistical analysis of lesion features.
[0085] In some embodiments, the specific steps in step S6 include: S61. Based on the target spatial pose, the pixel grid of the ultrasound image is completely projected into the three-dimensional voxel space of the magnetic resonance image to obtain the initial intermodal mapping boundary. S62. Based on the geometric contour and signal intensity gradient of the low-confidence feature region, local optimization is performed by driving a pre-constructed deformable mesh or local deformation field to perform local deformation correction on the intermodal mapping boundary so that the intermodal mapping boundary fits the actual shape of the low-confidence feature region and obtains each local target region after mapping.
[0086] The target spatial pose refers to the precise position and orientation of the ultrasound image in the three-dimensional space of the magnetic resonance image, determined after verifying multiple candidate spatial poses in step S5. It provides the basic rigid transformation parameters for subsequent image projection. Projecting the pixel grid of the ultrasound image completely into the three-dimensional voxel space of the magnetic resonance image means transforming each pixel of the ultrasound image into the three-dimensional coordinate system of the magnetic resonance image according to the rotation and translation relationship defined by the target spatial pose. This can be understood as "cutting out" a two-dimensional plane or thin layer corresponding to the ultrasound image in the three-dimensional space of the magnetic resonance image; its boundary is the initial intermodal mapping boundary. This initial boundary is obtained based on a global rigid transformation, providing a spatial reference for subsequent fine-tuning. The geometric contour and signal intensity gradient of the low-confidence feature region are key driving information for local deformation correction. The geometric contour can refer to the irregular edges and fracture morphology exhibited by the low-confidence region in the ultrasound image, as well as the actual shape of the corresponding lesion region in the magnetic resonance image. The signal intensity gradient reflects the rate and direction of change in pixel or voxel grayscale values in the image, indicating tissue boundaries or lesion edges. Pre-constructed deformable meshes or local deformation fields are techniques for achieving non-rigid deformation. For example, a deformable mesh can be a mesh model based on the finite element method (FEM) or free deformation (FFD), whose nodes can move under external forces. Local deformation fields can be defined based on B-splines or radial basis functions (RBF), achieving non-rigid deformation of local regions through control points. Local optimization refers to fine-tuning low-confidence regions based on the initial mapping boundary to reduce local mapping errors. Local deformation correction refers to non-rigid local adjustments to the initial intermodal mapping boundary to better match the actual anatomical morphology of the lesion region. The final mapped local target regions are more accurately mapped regions of the lesion region in the ultrasound image in magnetic resonance space after precise correction.
[0087] This application's solution addresses the problem of rigid transformations failing to adapt to differences in pathological tissue morphology during cross-modal mapping by introducing a local deformation correction mechanism. First, a preliminary mapping benchmark is established by projecting the pixel grid of the ultrasound image onto the three-dimensional space of magnetic resonance imaging (MRI), providing a spatial reference for subsequent fine-tuning. This initial projection is based on the target spatial pose determined in previous steps S1 to S5, obtained by identifying high-confidence feature regions and verifying multiple candidate poses, thus ensuring high-precision rigid alignment at the global level. Building upon this, the geometric contours and signal intensity gradients of low-confidence feature regions are used as driving forces, and the initial mapping boundary is dynamically adjusted using a deformable mesh or local deformation field. This method of local optimization based on low-confidence region features effectively compensates for nonlinear deformation of anatomical structures caused by lesions, ensuring that the mapping boundary accurately conforms to the actual morphology of the lesion. Through this progressive processing from global rigid projection to local deformation correction, precise alignment of ultrasound and MRI images in the lesion region is achieved. This scheme makes full use of the evaluation results of feature credibility in the previous steps. In the global localization stage, high credibility features are used first to avoid interference from lesion areas. In the local correction stage, low credibility areas are specifically refined. This overcomes the impact of pathological degradation on the spatial localization and mapping accuracy of multimodal images as a whole, and lays a solid anatomical spatial foundation for subsequent joint statistical analysis of high credibility lesion features.
[0088] In one specific implementation, after uniquely determining the three-dimensional position of the ultrasound section, the system needs to map the synovial thickness measurement and other lesion features from the ultrasound image to the magnetic resonance statistical space. However, because low-confidence regions are downweighted or excluded during the spatial localization stage, these regions may still have local mapping biases or minor misalignments under the determined global pose. If the ultrasound features are directly and rigidly projected, it may cause millimeter-level friction or boundary crossing between the local lesion boundary and the magnetic resonance anatomy. Therefore, the system must strictly limit and locally correct the mapping boundary based on the distribution of low-confidence regions. The system first projects the pixel grid of the ultrasound image completely onto the three-dimensional voxel space of the magnetic resonance image according to the inverse solution of the section position, forming an initial mapping mask. For example, the target spatial pose can be a 4x4 homogeneous transformation matrix containing rotation and translation parameters. By multiplying the coordinates (u,v,0) of the center point of each pixel in the ultrasound image by this transformation matrix, its corresponding coordinates (x,y,z) in the three-dimensional voxel space of the magnetic resonance image can be obtained. These projection points can form an initial two-dimensional plane or thin layer, the boundary of which is the initial intermodal mapping boundary. Subsequently, the system retrieves the previously identified set of low-confidence regions and marks the local spatial extents corresponding to these degenerate references on the initial mapping mask. For these marked local extents, the system initiates an elastic relaxation correction mechanism. Using the mapping results of stable, usable regions as rigid anchors, the system allows non-rigid deformation of the mapping mesh in low-confidence regions within a very small range. The driving force for deformation comes from the local fine-tuning matching of the ultrasound synovial boundary pixels with the soft tissue contrast gradient on the corresponding MRI slice. For example, in an ultrasound image, a bone erosion area may appear as a fracture and irregular contour of the cortical bone boundary, while the synovial tissue inside or around it may show abnormal signal intensity (such as hypoechoic hyperplasia). In an MRI image, the corresponding bone erosion area may appear as a defect in the cortical bone, and abnormal signals in the surrounding soft tissue (such as synovium) (such as high-signal edema). This contour and signal gradient information is extracted as the basis for deformation correction. Through this local elastic correction, the system fine-tunes the mapping coordinates of the degraded region, making them more closely match the actual soft tissue boundaries in the MRI image, thereby compensating for the local positioning uncertainty caused by the degradation of the bony reference. Simultaneously, the system performs boundary constraint operations. For extreme regions that are severely degraded in both modes and completely lose their structural correspondence, the system assesses mapping risk by calculating the divergence of the local spatial deformation field. If the deformation divergence of a local region exceeds a safety threshold, indicating that the mapping of that region is extremely unreliable, the system will forcibly shrink the mapping boundary of that region or perform a smooth transition, preventing the ultrasound synovial thickness value from projecting beyond the boundary to locations in the MRI that clearly belong to healthy cortical bone or unrelated joint recesses.By using this targeted correction for the effects of degraded areas, the system effectively prevents the ultrasonic synovial thickness measurement from being projected to an erroneous location drawn by the degraded reference, ensuring the spatial accuracy of the lesion distribution map in local details and blocking the transmission of local mapping benchmark drift to subsequent statistical analysis.
[0089] Through the above technical solution, this application effectively addresses the problem that relying solely on the target spatial pose for rigid projection during intermodal mapping is insufficient to handle complex anatomical changes in the lesion region. By performing local deformation correction on low-confidence feature regions based on global rigid projection, the intermodal mapping boundary can accurately conform to the actual lesion morphology, thus avoiding mismatch between the mapping boundary and the actual lesion region. This significantly improves the alignment accuracy of ultrasound and magnetic resonance images in the lesion region, ensuring the accuracy and reliability of subsequent statistical analysis of lesion features, and providing a more accurate spatial basis for the comprehensive assessment of rheumatic joint diseases.
[0090] In some embodiments, step S7, which involves performing a weighted joint statistical analysis on the lesion features in the first modality image and the second modality image based on weight labels to comprehensively assess the severity of rheumatic arthritis, includes the following specific steps: S71. Adjust the contribution of lesion features in the first modality image and the second modality image according to the weight label; the contribution adjustment specifically includes steps S711-S712: S711. For lesion features in high-weight regions, maintain or enhance their contribution; S712. For lesion characteristics in low-weight regions, reduce their contribution or restrict their inclusion; S72. Statistical analysis based on contribution-adjusted lesion characteristics is used to comprehensively assess the severity of rheumatic arthritis.
[0091] The weighting identifier is a quantitative indicator used to characterize the reliability or credibility of each local reference feature or mapped region during the bimodal joint degradation assessment. This identifier can be a continuous numerical value, such as a floating-point number from 0 to 1, where higher values indicate higher credibility; or it can be a discrete category label, such as "high credibility" and "low credibility". Its generation can be based on a comprehensive evaluation of multiple dimensions, including the morphological integrity, signal continuity, signal uniformity, structural smoothness, and intermodal expression consistency of the local reference features, or on an assessment of the degree and risk of local deformation correction during the mapping process. Contribution adjustment refers to the process of differentiating the influence of lesion features extracted from the first and second modal images in subsequent statistical analysis according to the weighting identifier. This adjustment aims to ensure that high-credibility lesion features dominate the final assessment while effectively suppressing or eliminating noise and errors that may be introduced by low-credibility lesion features. Contribution adjustment can be achieved in various ways, such as multiplying the original measurement value of the lesion feature by a coefficient related to the weighting identifier, or assigning the lesion feature to different statistical processing groups according to the weighting identifier. For lesion features in high-weight regions, maintaining or enhancing their contribution is an operation aimed at fully utilizing the accurate pathological information provided by high-confidence regions. Maintaining contribution means that the lesion features in this region will participate in statistical analysis with their original or near-original weights, ensuring that the true pathological condition is fully reflected. Enhancing contribution can be achieved by assigning a higher weight coefficient, allowing them to exert a greater influence in the statistical model, thereby highlighting the diagnostic value of key lesion regions. For lesion features in low-weight regions, reducing their contribution or restricting their inclusion is an operation aimed at reducing the potential negative impact of low-confidence regions. Reducing contribution can be achieved by multiplying the measured value of the lesion feature by a weight coefficient less than 1, thereby weakening its role in statistical analysis. Restricting their inclusion means that for regions with extremely low confidence, their lesion features may be completely excluded from certain statistical calculations, or used only as auxiliary references, to avoid misleading the overall assessment results. Statistical analysis refers to the use of statistical methods to quantify and model these features after adjusting their contribution to obtain a comprehensive assessment of the severity of rheumatic joint disease. This analysis can employ various statistical models. For example, a weighted spatial mixing effect model can be established to analyze the relationship between ultrasound synovial thickness and magnetic resonance signal intensity; or machine learning algorithms can be used to classify or regress the adjusted features to predict the lesion grade.
[0092] This application's solution achieves refined screening and weighted processing of multimodal lesion features by introducing a contribution adjustment mechanism. Upon receiving the mapping result package, the system formally enters the multi-image joint statistical analysis and lesion quantification fusion stage. The task of this stage is to perform joint probabilistic modeling within a unified magnetic resonance statistical space, combining the synovial thickness value measured by ultrasound and other lesion features with the synovial signal intensity, bone marrow edema range, and bone erosion volume extracted by magnetic resonance, to comprehensively assess the severity of rheumatoid arthritis. The system reads the spatial coordinates, synovial thickness value, and corresponding mapping confidence identifier from the mapping result package. When constructing the joint statistical model, the system introduces the mapping confidence identifier as a priori weight for the statistical samples. Specifically, based on the weight identifier, the contribution of lesion features in the first and second modal images is adjusted. For high-confidence regions where the mapping confidence identifier approaches one, the system assigns extremely high statistical weights to their observed values, allowing them to play a dominant role in estimating the mean, variance, and cross-modal correlation coefficient of synovial hyperplasia. These highly reliable data form the core benchmark of the joint statistical model, ensuring that the reliable information identified in previous steps through bimodal joint degradation assessment and local deformation correction is fully utilized, thereby effectively avoiding the negative impact of positioning uncertainty caused by bone reference degradation on the final assessment results. For low-confidence regions with low mapping confidence, the system performs weight reduction processing, significantly reducing the impact of these observations on the estimation of global statistical parameters. If the mapping confidence of a local region is lower than the system's set limit safety threshold, the system will perform a restrictive statistical operation on that region, i.e., it will not calculate the joint probability distribution of ultrasound and MRI in that region, but will only retain the lesion measurement results of MRI single-modality, and mark the region as an artificial review area. Through this confidence-based triage and weighting mechanism, the system effectively avoids the equal weighting of highly ambiguous mapping results and normal mapping results, preventing the error caused by spatial mismatch from accumulating and amplifying in the statistical model. Finally, based on the contribution-adjusted lesion features, the system performs statistical analysis and generates a cross-modal lesion quantification map containing the degree of synovial hyperplasia, bone erosion distribution, and their joint pathological probabilities based on the optimized joint statistical model. This process ensures that when assessing the severity of rheumatic joint disease using multimodal image fusion, it can fully utilize the accurate information from high-confidence regions while effectively avoiding errors that may be introduced by low-confidence regions, thereby significantly improving the accuracy and reliability of the assessment.
[0093] In one specific implementation, after mapping the first modal image (ultrasound image) to the second modal image (magnetic resonance image) and identifying high-confidence and low-confidence feature regions, the system assigns a weight identifier to each mapped local target region. For example, for a high-confidence feature region, its weight identifier can be set to a high value between 0.9 and 1.0, indicating that the mapping and feature extraction of this region are highly reliable. For a low-confidence feature region, its weight identifier may be set to a low value between 0.1 and 0.5, indicating that the reliability of this region is low and there may be noise or deformation errors. When performing weighted joint statistical analysis, the system can use a weighted spatial mixing effect model. For lesion features of high-weight regions, for example, if the synovial thickness measurement value of a high-weight region in the ultrasound image is X and the signal intensity of the corresponding region in the magnetic resonance image is Y, then in the statistical model, these values will be multiplied by a weight coefficient close to or greater than 1 (e.g., the weight identifier value itself or its square) to maintain or enhance their contribution to the estimation of model parameters. For example, if the weight is 0.95, its contribution can remain at its original value or be slightly enhanced by multiplying by 1.1. Conversely, for lesion features in low-weight regions, the system reduces their contribution or restricts their inclusion. For instance, if the weight of a low-weight region is 0.2, its lesion feature value may be multiplied by a small coefficient (e.g., 0.2) in the statistical calculation, thus significantly reducing its influence. Furthermore, if the weight of a region is below a preset safety threshold (e.g., 0.1), the system can restrict its inclusion, meaning its ultrasound lesion feature value is not included in the joint probability distribution calculation of ultrasound and MRI, but only the lesion measurement results of the MRI single modality are retained, and it is marked as a region requiring manual review. In this way, the system can ensure that the final statistical analysis results are mainly driven by reliable lesion features, without being interfered with by regions with high uncertainty.
[0094] Through the above technical solution, this application effectively solves the problem of decreased assessment accuracy caused by noise or artifact interference in low-confidence regions in traditional multimodal joint statistical analysis. By introducing weight labels and adjusting the contribution of lesion features, this application ensures that lesion features in high-confidence regions play a dominant role in statistical analysis, thereby accurately reflecting the true situation of key lesion areas. Simultaneously, for low-confidence regions, by reducing their contribution or restricting their inclusion, the misleading influence of noise and errors caused by intermodal mapping bias or local degradation on the overall assessment results is effectively avoided. This makes the final assessment of the severity of rheumatic joint disease more comprehensive, objective, and accurate, significantly improving the robustness and clinical reference value of cross-modal joint analysis, and providing doctors with more reliable diagnostic evidence.
[0095] Reference AppendixFigure 2 This invention provides an image recognition-based assessment system for rheumatic arthritis (this image recognition-based assessment system for rheumatic arthritis adopts the image recognition-based assessment method for rheumatic arthritis described in the above embodiments, and the specific process is described in the corresponding steps above), including: The image acquisition module 100 is used to acquire a first modal image and a second modal image containing the target monitoring object; The feature extraction module 200 is used to extract local reference features from the first modality image and the second modality image, respectively; The region identification module 300 is used to obtain the evaluation result by performing a dual-modal joint degradation evaluation on local reference features, and to identify high-confidence feature regions and low-confidence feature regions based on the evaluation result; The search module 400 is used to search for multiple candidate spatial poses of the first modality image in the spatial coordinate system corresponding to the second modality image, using high-confidence feature regions as the main spatial constraints. The verification module 500 is used to determine the target spatial pose by verifying each candidate spatial pose; The mapping correction module 600 is used to map the first modal image to the second modal image based on the target spatial pose, and to perform local deformation correction on the intermodal mapping boundary under the target spatial pose according to the distribution characteristics of the low confidence feature region, so as to obtain each local target region after mapping. The statistical analysis module 700 is used to assign weight labels to each local target region based on the evaluation results, and to perform weighted joint statistical analysis on the lesion features in the first modality image and the second modality image based on the weight labels, so as to comprehensively evaluate the lesion degree of rheumatic joint disease.
[0096] In this context, the units described as separate components may or may not be physically separate. Similarly, the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.
[0097] Furthermore, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0098] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.
[0099] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for assessing rheumatic joint lesions based on image recognition, characterized in that, Includes the following steps: S1. Acquire a first modal image and a second modal image containing the target monitoring object; S2. Extract local reference features from the first modality image and the second modality image respectively; S3. By performing a dual-modal joint degradation assessment on the local reference features, an assessment result is obtained, and based on the assessment result, high-confidence feature regions and low-confidence feature regions are identified; S4. Using the high-confidence feature region as the main spatial constraint, search for multiple candidate spatial poses of the first modality image in the spatial coordinate system corresponding to the second modality image; S5. Determine the target spatial pose by verifying each of the candidate spatial poses; S6. Based on the target spatial pose, the first modal image is mapped onto the second modal image, and according to the distribution characteristics of the low confidence feature region, the intermodal mapping boundary under the target spatial pose is locally deformed to obtain each local target region after mapping; S7. Based on the evaluation results, assign weight labels to each of the local target regions, and based on the weight labels, perform weighted joint statistical analysis on the lesion features in the first modal image and the second modal image to comprehensively evaluate the degree of rheumatic joint disease.
2. The image recognition-based assessment method for rheumatic joint lesions according to claim 1, characterized in that, The first modal image is an ultrasound image; the second modal image is a magnetic resonance image.
3. The image recognition-based assessment method for rheumatic joint lesions according to claim 2, characterized in that, The specific steps in step S2 include: S21. Extract the two-dimensional geometric features of the bony boundary from the ultrasound image as the local reference features; the two-dimensional geometric features include the spatial position of the boundary point, the direction of the boundary point, and the degree of curvature of the boundary point; S22. Extract the three-dimensional surface geometric features of the bony structure from the magnetic resonance image as the local reference features; the three-dimensional geometric features include the spatial position of the surface vertex, the normal direction of the surface vertex, and the curvature of the surface vertex.
4. The image recognition-based assessment method for rheumatic joint lesions according to claim 1, characterized in that, The specific steps in step S3 include: S31. Evaluate the morphological integrity and signal continuity of the corresponding local reference features in the first modal image, and the signal uniformity and structural smoothness of the corresponding local reference features in the second modal image, respectively, to obtain the single-modal evaluation results; S32. Evaluate the consistency between the representation of the corresponding local reference features in the first modality image and the corresponding local reference features in the second modality image to obtain the intermodal consistency evaluation result; S33. Based on the single-modal evaluation results and the intermodal consistency evaluation results, the evaluation results are obtained by calculating the joint degradation index of the local reference features; S34. Based on the evaluation results, and according to the joint degradation index and the preset degradation threshold, identify the high-confidence feature regions and low-confidence feature regions in the local reference features.
5. The image recognition-based assessment method for rheumatic joint lesions according to claim 1, characterized in that, The specific steps in step S4 include: S41. Extract a set of feature points from the high-confidence feature region of the first modality image; S42. Generate a series of virtual cross planes in the spatial coordinate system corresponding to the second modality image; S43. Match the virtual section with the feature point set based on feature correspondence; S44. Based on the matching results, identify multiple spatial poses with a similarity of not less than a preset similarity threshold, and use them as candidate spatial poses.
6. The image recognition-based assessment method for rheumatic joint lesions according to claim 1, characterized in that, The specific steps in step S5 include: S51. Evaluate the geometric relationship consistency between the high-confidence feature region in the first modality image and the corresponding anatomical structure in the second modality image to obtain the geometric relationship consistency evaluation result; S52. Evaluate the consistency of pathological distribution between the low-confidence feature region in the first modality image and the corresponding lesion region in the second modality image to obtain the pathological distribution consistency evaluation result; S53. Evaluate the relative positional consistency between the soft tissue structure adjacent to the local reference feature in the first modal image and the corresponding soft tissue structure in the second modal image, and obtain the relative positional consistency evaluation result; S54. For each candidate spatial pose, calculate the comprehensive score corresponding to each candidate spatial pose based on the corresponding geometric relationship consistency assessment result, pathological distribution consistency assessment result, and relative position consistency assessment result; S55. The candidate spatial pose corresponding to the highest comprehensive score is determined as the target spatial pose.
7. The image recognition-based assessment method for rheumatic joint lesions according to claim 6, characterized in that, The specific steps in step S54 include: For each candidate spatial pose, the corresponding geometric relationship consistency assessment result, pathological distribution consistency assessment result, and relative position consistency assessment result are normalized. Based on preset weights, the comprehensive score is obtained by weighted summation of the normalized geometric relationship consistency assessment result, the normalized pathological distribution consistency assessment result, and the normalized relative position consistency assessment result.
8. The image recognition-based assessment method for rheumatic joint lesions according to claim 2, characterized in that, The specific steps in step S6 include: S61. Based on the target spatial pose, the pixel grid of the ultrasound image is completely projected into the three-dimensional voxel space of the magnetic resonance image to obtain the initial intermodal mapping boundary. S62. Based on the geometric contour and signal intensity gradient of the low-confidence feature region, local optimization is performed by driving a pre-constructed deformable mesh or local deformation field to perform local deformation correction on the intermodal mapping boundary so that the intermodal mapping boundary fits the actual shape of the low-confidence feature region, and the mapped local target regions are obtained.
9. The image recognition-based assessment method for rheumatic joint lesions according to claim 1, characterized in that, In step S7, the specific steps for performing weighted joint statistical analysis on the lesion features in the first modality image and the second modality image based on the weight identifier to comprehensively assess the severity of rheumatic arthritis include: S71. Based on the weight identifier, adjust the contribution of lesion features in the first modality image and the second modality image; the contribution adjustment specifically includes steps S711-S712: S711. For lesion features in high-weight regions, maintain or enhance their contribution; S712. For lesion characteristics in low-weight regions, reduce their contribution or restrict their inclusion; S72. Statistical analysis based on contribution-adjusted lesion characteristics is used to comprehensively assess the severity of rheumatic arthritis.
10. A rheumatic joint disease assessment system based on image recognition, characterized in that, include: The image acquisition module is used to acquire a first modal image and a second modal image containing the target monitoring object; The feature extraction module is used to extract local reference features from the first modality image and the second modality image, respectively; The region identification module is used to obtain an evaluation result by performing a dual-modal joint degradation evaluation on the local reference features, and to identify high-confidence feature regions and low-confidence feature regions based on the evaluation result; The search module is used to search for multiple candidate spatial poses of the first modality image in the spatial coordinate system corresponding to the second modality image, using the high-confidence feature region as the main spatial constraint. The verification module is used to determine the target spatial pose by verifying each of the candidate spatial poses; The mapping correction module is used to map the first modal image to the second modal image based on the target spatial pose, and to perform local deformation correction on the intermodal mapping boundary under the target spatial pose according to the distribution characteristics of the low confidence feature region, so as to obtain each local target region after mapping. The statistical analysis module is used to assign weight labels to each of the local target regions based on the evaluation results, and to perform weighted joint statistical analysis on the lesion features in the first modality image and the second modality image based on the weight labels, so as to comprehensively evaluate the degree of rheumatic joint disease.
