Joint surgical operation data fusion processing method and system based on multi-modal image

Abstract

The invention discloses a joint surgical operation data fusion processing method and system based on a multi-modal image, and relates to the technical field of image processing. The method comprises the following steps: performing motion state evaluation based on joint trajectory data of each posture point image in a multi-modal image, and dynamically acquiring a reference joint image of the multi-modal image based on a motion state evaluation result; performing image accuracy evaluation based on joint position data in each reference joint image, and dynamically correcting the joint data based on an image accuracy evaluation result; and forming a joint three-dimensional model based on the corrected joint data, performing dynamic region division based on joint evaluation parameters in the joint three-dimensional model, and dynamically adjusting the model geometric accuracy of the joint three-dimensional model based on a region division result and a joint category. Accurate quantitative analysis of the joint movement function, individualized high-precision three-dimensional model reconstruction and systematic improvement of clinical diagnosis efficiency are achieved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for fusion processing of joint surgical data based on multimodal images. Background Technology

[0002] Joint surgery involves complex anatomical structures such as bones, cartilage, and ligaments. A single imaging modality cannot simultaneously meet the needs of preoperative diagnosis for resolving fine structures, intraoperative navigation for real-time accuracy, and postoperative assessment for functional status. It has limitations in terms of information coverage or accuracy. Therefore, the deep integration of joint surgery with multimodal imaging technology has become a core trend in precision orthopedic medicine. Multimodal imaging technology integrates the advantages of multiple imaging modalities such as CT, MRI, ultrasound, and PET-CT. It can clearly present the fine structures of soft tissues such as articular cartilage and ligaments through MRI, accurately reconstruct the anatomical morphology of bones through CT, and combine the real-time performance of ultrasound with the metabolic functional imaging of PET-CT. It provides comprehensive imaging evidence for the accurate preoperative diagnosis, lesion extent assessment, and surgical planning of joint diseases (such as osteoarthritis, joint injuries, and tumors). At the same time, it enables real-time positioning and adjustment of the surgical path through image navigation during surgery. Postoperatively, it enables efficacy evaluation and complication monitoring through multimodal image comparison. It significantly improves the accuracy, safety, and minimally invasiveness of joint surgery and promotes the development of joint surgery towards personalization and precision.

[0003] The core function of joint surgery data fusion processing is to integrate preoperative multimodal imaging data, intraoperative real-time navigation data, patient clinical information, and instrument sensor data, breaking the limitations of a single data dimension. Its core benefit lies in achieving personalized optimization of preoperative surgical plans, precise navigation and risk warning during intraoperative operations, and quantitative assessment and rehabilitation monitoring of postoperative efficacy through complementary and collaborative analysis of multi-source data. This significantly improves surgical safety, minimal invasiveness, and long-term prognosis. Existing technologies mainly use algorithms such as deep learning and machine learning for data denoising, feature extraction, and fusion modeling. Combined with standardized data interaction protocols such as DICOM and FHIR, and relying on image navigation systems, intraoperative sensors, and cloud data platforms, it realizes real-time transmission, integrated processing, and visualization of multi-source data, providing full-process data support for precision diagnosis and treatment in joint surgery.

[0004] For example, the invention patent announcement CN119811686B, which discloses a method and system for processing minimally invasive surgical data based on AR technology, includes: performing three-dimensional reconstruction and lesion segmentation on medical image data and extracting feature points; using feature point data to acquire the position of surgical instruments and perform spatial mapping; reconstructing the light field and generating an augmented reality scene based on feature points and position data; performing trajectory analysis and path planning based on augmented reality; performing real-time monitoring and risk warning of the surgical path; and finally extracting surgical patterns and establishing knowledge mapping relationships.

[0005] However, in the process of implementing the inventive technical solution in the embodiments of this application, it was found that the above-mentioned technology has at least the following technical problems: In existing technologies, during the fusion processing of multimodal images for joint surgery before surgery, patient movement may cause spatial misalignment and registration deviation between image data acquired at different time points or in different modalities. This disrupts the precise spatial correspondence between multi-source information, resulting in inaccurate anatomical structure positioning in the fused 3D model and affecting the accuracy of the surgical plan. Summary of the Invention

[0006] This application provides a method and system for fusion processing of joint surgical data based on multimodal images. It addresses the problem in existing technologies where patient movement during preoperative fusion processing of multimodal images for joint surgical data can lead to spatial misalignment and registration deviations between image data acquired at different time points or in different modalities. This disrupts the precise spatial correspondence between multiple sources, resulting in inaccurate anatomical structure positioning in the fused 3D model and affecting the accuracy of the surgical plan. The method achieves high-precision fusion of multimodal images based on dynamic evaluation and correction, generating a reliable 3D joint model suitable for precise surgical planning.

[0007] This application provides a method for fusing and processing joint surgical data based on multimodal images, including the following steps: acquiring joint trajectory data of each pose point image in the multimodal images; evaluating the motion state based on the joint trajectory data of each pose point image; obtaining reference joint images of the multimodal images based on the motion state evaluation results; acquiring joint position data in each reference joint image; evaluating the image accuracy based on the joint position data in each reference joint image; determining whether to dynamically correct the joint data based on the image accuracy evaluation results; if so, fusing to form a three-dimensional joint model after dynamically correcting the joint data; otherwise, directly fusing to form a three-dimensional joint model; acquiring joint evaluation parameters in the three-dimensional joint model; dynamically dividing the region based on the joint evaluation parameters in the three-dimensional joint model; and dynamically adjusting the model accuracy based on the region division results and joint categories.

[0008] This application provides a multimodal imaging-based joint surgery data fusion processing system, comprising: an image acquisition module, an accuracy calibration module, and a model optimization module. The image acquisition module acquires joint trajectory data from various pose points in the multimodal imaging, performs motion state assessment based on the joint trajectory data, and obtains reference joint images from the multimodal imaging based on the motion state assessment results. The accuracy calibration module acquires joint position data from each reference joint image, performs image accuracy assessment based on the joint position data in each reference joint image, and determines whether to dynamically correct the joint data based on the image accuracy assessment results. If so, the joint data is fused to form a three-dimensional joint model after dynamic correction; otherwise, the three-dimensional joint model is directly fused. The model optimization module acquires joint evaluation parameters from the three-dimensional joint model, performs dynamic region division based on the joint evaluation parameters, and dynamically adjusts the model accuracy based on the region division results and joint categories.

[0009] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By converting dynamic images into quantified joint trajectory deviation indices, accurate identification and classification of motion abnormalities are achieved, and highly reliable individualized reference joint images are intelligently generated based on clinical priorities. Furthermore, through cross-validation and iterative correction mechanisms among multimodal reference images, abnormal data is effectively eliminated and the consistency of joint position data is improved, laying a high-quality data foundation for modeling. In the 3D model generation stage, by dynamically dividing regions and adaptively adjusting geometric accuracy based on core and non-core joint parameters, intelligent model construction is achieved, which preserves high-precision anatomical details in key functional areas and optimizes computational resources in non-critical areas. This realizes the standardization, precision, and efficiency of the entire process from dynamic images to diagnostic-grade 3D models, providing a reliable technical platform for quantitative analysis of joint motion function and personalized diagnosis and treatment.

[0010] 2. By constructing a multi-level data processing mechanism that combines quantitative assessment, graded fault tolerance, and intelligent repair, the deviation position and angle of the joint trajectory are comprehensively transformed into a joint trajectory deviation index, thereby achieving accurate quantitative assessment of motion posture and automatic identification of abnormal posture points. By distinguishing between repairable and unrepairable abnormal data, effective images are preserved to the maximum extent while ensuring the overall reliability of the data. Based on posture point priority, and using high-priority posture points as constraints, a motion trajectory that conforms to physiological laws is reconstructed through an interpolation optimization algorithm with smoothing constraints. This enables the dynamic generation of highly reliable and physiologically accurate individualized reference joint images even under non-ideal acquisition conditions, providing a solid data foundation for subsequent surgical planning and evaluation.

[0011] 3. By constructing a precision control system based on cross-validation of multimodal reference images, and employing a weighted fusion algorithm of three joint measurement data—average distance difference, maximum distance difference, and structural similarity—the system achieves accurate quantitative evaluation and adaptation level classification of the deviation between any two sets of reference joint images. By statistically analyzing the proportion of highly adapted image pairs and identifying abnormal images, and implementing step-by-step iterative correction of abnormal data based on the priority of the joint measurement ratio coefficient, the system achieves synergistic optimization of multimodal image data in three dimensions: joint position accuracy, extreme deviation control, and overall structural consistency. This ensures that the joint data used for subsequent modeling meets the overall fusion quality standard, laying a solid data foundation for generating high-precision and high-reliability 3D joint models.

[0012] 4. A multi-level region division system based on core and non-core parameters of the joint was established, and the 3D joint model was accurately divided into four levels of regions according to the severity of the pathology, thereby realizing differentiated identification of regions of different clinical importance. By mapping the region classification results, the number of abnormal parameters and the joint category in a multi-dimensional way, the sampling rate and overall mesh density of each region were dynamically adjusted. In the end, while ensuring that the key lesion areas obtain the highest geometric accuracy to support precise surgical planning, the model data volume of non-critical areas was effectively optimized, which significantly improved the practical value and computational efficiency of the 3D model in clinical diagnosis and surgical navigation. Attached Figure Description

[0013] Figure 1 A flowchart of a joint surgery data fusion processing method based on multimodal images provided in an embodiment of this application; Figure 2 A flowchart illustrating the joint data correction process of the joint surgery data fusion processing method based on multimodal images provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of a joint surgery data fusion processing system based on multimodal images provided in an embodiment of this application. Detailed Implementation

[0014] This application provides a method and system for fusing and processing joint surgical data based on multimodal imaging. This solves the problem in existing technologies where patient movement during preoperative fusing and processing of multimodal joint surgical data can lead to spatial misalignment and registration deviations between image data acquired at different time points or in different modalities. This disrupts the precise spatial correspondence between multiple sources of information, resulting in inaccurate anatomical structure localization in the fused 3D model and affecting the accuracy of the surgical plan. The overall approach is as follows: First, motion state assessment is performed based on joint trajectory data from various pose points in multimodal imaging, and reference joint images are dynamically acquired based on the assessment results. Then, image accuracy is assessed based on joint position data contained in each reference joint image, and the joint data is dynamically corrected based on the accuracy assessment results. Finally, the corrected joint data is imported into medical image processing software to construct a three-dimensional joint model. Based on joint assessment parameters such as core and non-core parameters in the three-dimensional joint model, dynamic region division is performed, and the geometric accuracy of the three-dimensional joint model is dynamically adjusted by combining the region division results with the joint category.

[0015] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0016] like Figure 1 The diagram shown is a flowchart of a joint surgery data fusion processing method based on multimodal images provided in this application embodiment. The method includes the following steps: Motion state assessment is performed based on joint trajectory data from various pose points in multimodal imaging, and reference joint images are dynamically acquired based on the motion state assessment results. Image accuracy is assessed based on joint position data in each reference joint image, and joint data is dynamically corrected based on the image accuracy assessment results. The corrected joint data is imported into medical image processing software (such as Mimics, 3D Slicer, etc.) to form a three-dimensional joint model. Dynamic region division is performed based on joint assessment parameters in the three-dimensional joint model, and the geometric accuracy of the three-dimensional joint model is dynamically adjusted based on the region division results and joint category. Joint assessment parameters include core joint parameters and non-core joint parameters.

[0017] In this embodiment, a pose point refers to a single instant or specific pose image frame captured and extracted from a continuous motion sequence of multimodal images. It is the basic data unit for motion state assessment. The joint 3D model is a digital skeletal model that can be arbitrarily rotated and observed in 3D space based on the corrected joint data. It accurately reproduces the skeletal geometry of specific joints of the patient, such as the knee and hip joints, including all the concave and convex details of the bone surface. It provides doctors with direct and visual basis for surgical planning, simulated osteotomy, and implant placement.This invention significantly improves the objectivity, accuracy, and efficiency of sports medicine analysis and diagnosis through a progressive intelligent evaluation and dynamic adjustment mechanism. First, it assesses motion status by fusing joint trajectory data from different posture points in multimodal imaging. Then, it integrates high- and low-priority posture point data with anatomical structure information to generate reference joint images. Next, it fuses joint position data from different reference images for accuracy evaluation. Finally, it dynamically corrects joint data by fusing reliable data from standard images and deviation characteristics from abnormal images. The corrected data is then integrated with the modeling capabilities of medical image processing software to construct a 3D joint model. Furthermore, this system exhibits excellent local deployment feasibility: its software architecture supports storage as a computer program product on common storage media such as disk storage and CD-ROM, is compatible with various computing devices including general-purpose computers, special-purpose computers, and embedded processors, and can import the corrected joint data into Mimics and 3D Slicer. The inclusion of locally installed medical image processing software further ensures the localization of 3D model construction and subsequent functions, adapting to the equipment environment requirements of clinical scenarios. The system also integrates core and non-core joint assessment parameters for dynamic region division, and adjusts the model's geometric accuracy by combining the region division results with joint category characteristics. It also features practical simulated osteotomy and prosthesis installation functions, providing doctors with visual guidance for simulated osteotomy and implant placement. Through precise implant adaptation functions combined with parameters such as medullary cavity diameter and bone defect size, it assists in implant selection and adaptation. Combined with a digital bone model that can be rotated and observed arbitrarily in 3D space, it provides intuitive and accurate support for surgical plan deduction. The entire process achieves deep integration of multimodal image data, anatomical data, key surgical parameters, and software tool capabilities. This fusion approach breaks through the limitations of single data or single tools. This integrated processing mode not only improves the accuracy, completeness, and reliability of joint data through complementary verification of multi-source data, avoiding the bias or information loss problems of single-modality images, but also ensures high-precision modeling of the core surgical focus areas while rationally optimizing the allocation of computing resources, balancing model quality and processing efficiency through dynamic correction and precise regional accuracy adjustment. Furthermore, the adaptability of local deployment and the implementation of simulated osteotomy and prosthesis installation functions allow the system to directly meet the actual needs of clinical surgical planning. Ultimately, it provides more comprehensive, accurate, and clinically relevant support for joint surgery planning, helping to reduce surgical errors, improve surgical safety and outcomes, and fully demonstrating the core advantages of data fusion in joint surgery data processing.

[0018] Furthermore, the steps for motion state assessment based on joint trajectory data from pose points in multimodal imagery include: Joint trajectory data includes the deviation positions and angles of each joint landmark. The deviation position is the average deviation between the position of each joint landmark in the image and the standard joint landmark position. Spatial coordinates of the joints are extracted from the image using a pose estimation algorithm. The Euclidean distance between the joints and the standard position at each time point is calculated, and the average distance over the entire sequence is obtained, reflecting the overall degree of spatial deviation of the joint from the standard trajectory. The deviation angle is the average deviation between the angle of each joint landmark in the image and the standard joint landmark angle. Limb vectors are constructed by connecting the joints, and the angle between the vectors is calculated to obtain the actual joint angle. The absolute difference between the actual joint angle and the standard angle at each time point is calculated, and finally, the average difference over the entire sequence is obtained, reflecting the overall degree of spatial deviation of the joint from the standard trajectory. The degree to which the range of motion deviates from the standard angle is determined. The joint trajectory data of each pose point image in the multimodal imagery are compared and coupled with preset joint trajectory reference data. Then, the comparison results are weighted and coupled using preset joint trajectory scaling coefficients to obtain the joint trajectory deviation index for each pose point image. The joint trajectory reference data includes deviation position reference values ​​and deviation angle reference values, and the joint trajectory scaling coefficients include deviation position scaling coefficients and deviation angle scaling coefficients. The joint trajectory deviation index of each pose point image is compared with a preset joint trajectory deviation threshold. If the joint trajectory deviation index of any pose point image does not reach the joint trajectory deviation threshold, the corresponding pose point image is marked as a normal pose point image; if the joint trajectory deviation index of any pose point image reaches the joint trajectory deviation threshold, the corresponding pose point image is marked as an abnormal pose point image.

[0019] The joint trajectory deviation index for each pose point image is obtained as follows: ; In the formula, This represents the joint trajectory deviation index of the image at the i-th pose point. and These represent the deviation position proportionality coefficient and the deviation angle proportionality coefficient, respectively. and These represent the deviation position and deviation angle of the j-th joint landmark in the image of the i-th pose point, respectively. and These represent the reference values ​​for the deviation position and the deviation angle, respectively. Here, i is the image number of each posture point, i=1,2,3,...,N,N is the total number of posture point images, and j is the image number of each joint landmark, j=1,2,3,...,M,M is the total number of joint landmarks.

[0020] In this embodiment, the spatial coordinates of joint landmarks are extracted using a posture estimation algorithm, and the average Euclidean distance from the standard position is calculated to obtain the deviation position. Simultaneously, the average absolute difference between the actual joint angle and the standard angle is calculated using the limb vector angle to obtain the deviation angle. This process accurately captures the overall degree of joint deviation from the standard trajectory from two core dimensions: spatial position and movement angle, providing objective and quantifiable basic data for motion state assessment. Then, a joint trajectory deviation index is obtained based on a comprehensive evaluation of these two core dimensions, effectively integrating deviation information from both position and angle dimensions to form a unified comprehensive evaluation index. This avoids the limitations of single-dimensional evaluation and achieves comprehensive quantification of joint motion state in images of various posture points. The deviation index is compared with a preset threshold to mark normal or abnormal posture point images, clearly distinguishing the quality of each posture point image, accurately filtering out normal images that meet the standard, and accurately identifying abnormal images that deviate from the standard. This provides a reliable screening basis for subsequent dynamic acquisition of reference joint images, ensuring the accuracy and effectiveness of subsequent data fusion processing from the source, and laying a solid foundation for the precise application of joint surgical data.

[0021] Furthermore, the steps for dynamically acquiring reference joint images from multimodal images based on motion state assessment results include: counting the number of normal posture point images and abnormal posture point images in the multimodal images; Step 1: determining whether the number of normal posture point images in each modality exceeds a preset threshold for the number of normal posture point images; if so, proceeding to Step 2; otherwise, entering the re-shooting process; Step 2: determining whether the number of abnormal posture point images exceeds a preset threshold for the number of abnormal posture point images; if so, entering the re-shooting process; otherwise, marking the difference between the joint trajectory deviation index of each abnormal posture point image and the joint trajectory deviation threshold as the joint trajectory deviation margin; and then... The joint trajectory deviation margin of the pose point image is compared with the preset joint trajectory abnormality threshold. If the joint trajectory deviation margin of any abnormal pose point image does not exceed the joint trajectory abnormality threshold, the corresponding abnormal pose point image is marked as a repairable pose point image. If the joint trajectory deviation margin of any abnormal pose point image exceeds the joint trajectory abnormality threshold, the corresponding abnormal pose point image is marked as an unrepairable pose point image. Step 3: Determine whether the number of unrepairable pose point images exceeds the preset unrepairable pose point image number threshold. If so, proceed to the reshooting process. Otherwise, dynamically acquire the reference joint image of the corresponding modal image based on the pose point priority order.

[0022] In this embodiment, the present invention establishes a basic data dimension for subsequent screening by statistically analyzing the number of normal and abnormal pose point images. By determining whether the number of normal pose point images meets the standard, it ensures from the source that the reference images have a sufficient number of qualified samples, avoiding subsequent processing deviations due to insufficient effective data. If the standard is not met, re-shooting is triggered, directly ensuring the basic quality of the data. Step two further controls the overall proportion of abnormal data by determining whether the number of abnormal pose point images exceeds the standard, preventing too many abnormal images from interfering with the reliability of the reference images. At the same time, by calculating the joint trajectory deviation margin and comparing it with the abnormal threshold, it accurately distinguishes between repairable and unrepairable abnormal images, thus retaining images that can be salvaged through subsequent processing to reduce invalid shooting. Completely unusable images were excluded to avoid unnecessary processing costs. By judging whether the number of unrepairable images exceeded the limit, the data quality baseline was further strictly controlled to ensure that the proportion of severely abnormal images in the final reference image generation was within a controllable range. Finally, reference images were acquired based on the priority order of posture points, prioritizing the inclusion of images of core posture points such as the maximum flexion position of the joint and key angles for clinical diagnosis, while reasonably handling transitional low-priority posture points. This not only ensured the complete coverage of key anatomical and kinematic information by the reference images, but also reduced unnecessary re-shooting processes through dynamic screening, significantly improving the accuracy, completeness, and acquisition efficiency of reference joint images, and laying a high-quality data foundation for the accurate processing of subsequent joint data.

[0023] Furthermore, the step of dynamically acquiring reference joint images for corresponding modal images based on the priority order of pose points includes: marking pose points in the modal images with preset key angles as high-priority pose points, including pose points in the maximum flexion, maximum extension, and clinically diagnostic key angles of the joint; marking pose points in the modal images without preset key angles as low-priority pose points, including transition pose points during joint movement, whose optimization weight is lower than that of high-priority pose points; and using a smooth spline curve interpolation algorithm, based on the joint data of high-priority pose points as constraint points, combined with joint anatomical structures (such as joint axis position and bone length), fitting the motion path between high-priority pose points through interpolation to obtain the initial joint movement. The trajectory and joint data include the three-dimensional spatial coordinates and joint motion angle parameters corresponding to each high-priority pose point. The Euclidean distance between the spatial coordinates of the low-priority pose points and the three-dimensional spatial coordinates of the corresponding points on the initial joint motion trajectory is calculated. The Euclidean distance is used as the core element to construct the objective function. The constraints of the trajectory curve are set as follows: the sum of the squares of the second derivatives of the trajectory curve is less than a preset smoothing threshold, the joint motion angle is within the anatomically permissible range, and the coordination of adjacent joint motions is satisfied. The objective function is minimized by the least squares method to optimize the curve parameters of the initial joint motion trajectory. The optimized initial joint motion trajectory is fused with the anatomical structure information of the corresponding modal image to generate a reference joint image that conforms to physiological laws and data reliability.

[0024] In this embodiment, the present invention focuses on core information by clearly prioritizing key information. Posture points containing preset key angles such as maximum joint flexion, maximum extension, and clinically diagnostic key angles are marked as high priority, ensuring that critical motion states crucial to surgical decisions are preferentially retained and utilized. Transitional joint motion posture points are marked as low priority and assigned lower optimization weights, achieving a reasonable allocation of data processing resources. Using the three-dimensional spatial coordinates and joint motion angles of high-priority posture points as constraint points, combined with anatomical structures such as joint axis position and bone length, a smooth spline curve interpolation algorithm is used to fit the motion path between high-priority points, initially constructing an initial joint motion trajectory that conforms to the anatomical basis, providing a benchmark that fits the physiological structure for subsequent optimization. An objective function is constructed by calculating the Euclidean distance between the spatial coordinates of low-priority posture points and the corresponding points of the initial trajectory, and trajectory smoothness is set. The optimized trajectory is fused with the anatomical structure information of the modal image to generate a reference joint image. Prioritization ensures the core status of key posture points, avoiding interference from non-critical information on core data. The interpolation and optimization process, combining anatomical structures and physiological constraints, ensures that the generated reference joint images strictly adhere to joint anatomy and physiological range of motion, while precise fitting of low-priority points guarantees the integrity and continuity of the motion trajectory. The final generated reference joint images possess accuracy of key information, physiological rationality of the motion trajectory, and complete data coverage, providing high-quality, highly reliable foundational image data for subsequent accuracy assessment and correction of joint data. This effectively supports the precision and scientific rigor of joint surgical data fusion processing.

[0025] like Figure 2The diagram shows a flowchart of joint data correction in a multimodal image-based joint surgery data fusion processing method provided in this application embodiment. The index in the diagram represents the image deviation evaluation index, the threshold represents the image deviation evaluation threshold, and the proportion represents the proportion of highly adapted image pairs. The steps for evaluating image accuracy based on joint position data in each reference joint image include: for any two sets of reference joint images in the multimodal imagery, obtaining joint measurement data based on the joint position data of the two sets of images. The joint measurement data includes the average distance difference of each joint landmark, the maximum distance difference of each joint landmark, and structural similarity. The average distance difference is obtained by calculating the spatial position (Euclidean distance) of the same joint point across all corresponding time frames and taking its arithmetic mean, reflecting the systematic positional deviation of the joint point. The maximum distance difference is taken from the maximum value in the instantaneous distance set and used to capture extreme positional deviations. The structural similarity is calculated by constructing a skeleton diagram of the joint point coordinates in each frame and calculating the inter-frame similarity. The similarity score of the skeleton topology is averaged to evaluate the consistency of the overall posture structure. The average distance difference, maximum distance difference, and structural similarity reference value of each joint marker are compared with the preset average distance reference value, maximum distance reference value, and joint similarity. The comparison results are then weighted and coupled using preset joint measurement scaling coefficients to obtain the image deviation evaluation index between the two sets of images. The joint measurement reference data includes the average distance reference value, joint similarity reference value, and maximum distance reference value. The joint measurement scaling coefficients include the average distance scaling coefficient, joint similarity scaling coefficient, and maximum distance scaling coefficient. The image deviation evaluation index between each pair of images is compared with a preset image deviation evaluation threshold. If the image deviation evaluation index exceeds the image deviation evaluation threshold, it is marked as a low-fit image pair; if the image deviation evaluation index does not exceed the image deviation evaluation threshold, it is marked as a high-fit image pair.

[0026] The image deviation assessment index is obtained as follows: ; In the formula, This represents the image deviation assessment index. , and These represent the average distance scaling factor, the joint similarity scaling factor, and the maximum distance scaling factor, respectively. , and Let represent the average distance difference, the maximum distance difference, and the structural similarity of the j-th joint landmark, respectively. , and These represent the average distance reference value, the joint similarity reference value, and the maximum distance reference value, respectively.

[0027] Furthermore, the steps for dynamically correcting joint data based on image accuracy assessment results include: statistically analyzing the proportion of highly adapted image pairs; if the proportion of highly adapted image pairs exceeds a preset overall fusion threshold, the multimodal images are determined to meet the fusion conditions as a whole, and no additional processing is performed; if the proportion of highly adapted image pairs does not exceed the preset overall fusion threshold, images in low-adaptive image pairs that appear more than a preset threshold number of times are marked as abnormal images, and images in high-adaptive image pairs that appear more than a preset threshold number of times are marked as standard images; the joint data of abnormal images are dynamically corrected based on the average coordinates of each joint marker point in the standard images and the joint measurement data of the abnormal images, until the proportion of highly adapted image pairs exceeds the preset overall fusion threshold.

[0028] In this embodiment, the present invention first predicts the overall fusion feasibility of multimodal images by statistically analyzing the proportion of highly adapted image pairs and comparing it with the overall fusion threshold. If the proportion meets the standard, the fusion condition is directly met, avoiding unnecessary redundant correction operations and significantly improving data processing efficiency. If the proportion does not meet the standard, based on the frequency threshold of images appearing in the adaptation process, standard images with excessively high adaptation frequency and abnormal images with excessively low adaptation frequency are accurately screened. Through the frequency accumulation effect, interference from images with accidental adaptation or mismatch is eliminated, making the standard images more representative and reliable, and the location of abnormal images more targeted, thus ensuring the scientific nature of the correction basis from the source. The accuracy of the correction target is ensured by using the average coordinates of each joint landmark in the standard image as a reliable calibration benchmark, combined with the joint measurement data of the abnormal image itself for dynamic correction. This ensures that the correction process always conforms to the real anatomical position and image characteristics, and can also make up for the deviations and shortcomings of the abnormal image. It avoids new errors caused by blind correction, and ensures that all image data involved in subsequent fusion have high consistency and reliability. It effectively avoids the interference of low-quality images on the fusion results, and finally provides accurate, unified, and high-quality joint data that meets the fusion requirements for the construction of the joint 3D model. This lays a solid foundation for the accuracy of subsequent surgical planning and model analysis from the data level.

[0029] Furthermore, the step of dynamically correcting the joint data of the abnormal image based on the average coordinates of each joint marker in the standard image and the joint measurement data of the abnormal image includes: obtaining the average joint measurement data between the abnormal image and the standard image, the average joint measurement data including the average deviation distance value of each joint marker, the maximum distance difference of each joint marker, and the average structural similarity; and correcting the joint measurement data in the abnormal image that do not meet the preset threshold conditions in descending order of the joint measurement ratio coefficient, specifically including: if the average distance difference of any joint marker in the abnormal image exceeds the preset distance difference threshold, then based on the average position coordinates of the corresponding joint marker in the standard image, correcting the joint data step by step according to the preset coordinate correction unit. The coordinates of the joint markers corresponding to the anomalous image are corrected step by step until the average distance difference of the corresponding joint markers does not exceed a preset distance difference threshold. If the maximum distance difference of any joint marker in the anomalous image exceeds the preset maximum distance difference threshold, the coordinates of the joint markers corresponding to the anomalous image are gradually corrected according to a preset coordinate correction unit based on the average position coordinates of the corresponding joint markers in the standard image until the maximum distance difference of the corresponding joint markers does not exceed the preset maximum distance difference threshold. If the average structural similarity of the anomalous image is lower than the preset structural similarity threshold, structural similarity correction is performed based on the average coordinates of each joint marker in the standard image until the average structural similarity of the anomalous image is not lower than the structural similarity threshold.

[0030] In this embodiment, the present invention acquires average joint measurement data such as the average deviation distance, maximum distance difference, and average structural similarity between abnormal images and standard images. From three core dimensions—systematic positional deviation, extreme positional deviation, and overall posture and structural consistency—it accurately identifies the type and degree of deviation in abnormal images, providing a clear target basis for subsequent correction and avoiding blind correction. By correcting measurement data that do not meet the threshold conditions in descending order of joint measurement ratio coefficients, the present invention prioritizes deviation dimensions with higher weight in image quality, rationally allocates correction resources, ensures that core deviation problems are addressed first, and improves correction efficiency and targeting. Differentiated and precise correction strategies are adopted for different types of deviations: for average and maximum distance differences exceeding the threshold, correction units are adjusted according to preset coordinates based on the average position coordinates of the corresponding joint markers in the standard image. Gradual adjustments ensured the controllability of the correction process while accurately offsetting local positional deviations, avoiding anatomical distortion caused by over-correction. For average structural similarity below a threshold, overall structural similarity correction was performed based on the average coordinates of joint landmarks in the standard image, ensuring that the overall pose of the abnormal image remained consistent with the anatomical logic of the standard image, balancing local deviation correction with overall structural coordination. Continuous optimization continued until all measurement data met the preset threshold, achieving comprehensive compliance of joint data in abnormal images. This resulted in the corrected abnormal images being highly compatible with the standard images in terms of local joint position accuracy, extreme deviation control, and overall structural consistency, effectively eliminating adaptation differences between multimodal images and significantly improving the uniformity and reliability of joint data. This provided high-standard, high-quality basic data support for subsequent multimodal image fusion and accurate construction of joint 3D models.

[0031] Furthermore, the steps for dynamic region division based on joint assessment parameters in the 3D joint model include: core joint parameters including at least one of the following: lower limb force line angle, osteotomy surface angle, key bony anatomical dimensions, medullary cavity diameter, bone defect size, cartilage damage extent, and ligament attachment point displacement; non-core joint parameters including at least one of the following: joint alignment deviation, articular surface coverage, osteophyte size and location, and joint range of motion limit angle; comparing the joint assessment parameters of each region with the preset joint assessment reference range, and counting the number of core joint parameters in each region that exceed the preset joint assessment reference range as the number of core joint abnormalities. The number of non-core joint parameters exceeding the preset joint assessment reference range is counted as the number of non-core joint abnormalities. If the number of core joint abnormalities in any region exceeds the preset threshold for the number of core joint abnormalities, the corresponding region is marked as the first region. If the number of core joint abnormalities in any region exceeds 0 but does not exceed the preset threshold for the number of core joint abnormalities, the corresponding region is marked as the second region. If the number of core joint abnormalities in any region is 0, but the number of non-core joint abnormalities exceeds 0, the corresponding region is marked as the third region. If both the number of core joint abnormalities and the number of non-core joint abnormalities in any region are 0, the corresponding region is marked as the fourth region.

[0032] In this embodiment, the present invention defines key indicators that directly affect the surgical planning and execution effect, such as the lower limb force line angle and osteotomy surface angle, as core joint parameters, and indicators that have a relatively minor impact on the surgery, such as joint alignment deviation and osteophyte size, as non-core joint parameters. Through clear parameter priority division, the core focus dimension of the surgery is ensured to become the core basis for region division, avoiding non-critical factors from interfering with core judgment. By comparing the core and non-core parameters of each region with preset reference ranges, the number of core abnormalities and non-core abnormalities in each region are statistically analyzed and quantified. Objective data replaces subjective judgment, accurately capturing the degree and type of abnormality in each region, avoiding the ambiguity and randomness of region division. Based on the combination of the number of core abnormalities and the number of non-core abnormalities, the model is divided into a first region with severe core abnormalities, a second region with mild core abnormalities, a third region with only non-core abnormalities, and a fourth region with no abnormalities, forming a gradient region classification system. This system can clearly and intuitively present the quality status and problem level of each region of the model, highlighting the priority of core abnormal regions while not ignoring the potential impact of non-core abnormal regions. This invention not only achieves accurate profiling of each region in a 3D joint model and clarifies the severity of problems and key concerns in different regions, but also provides precise and quantitative decision-making basis for dynamic adjustment of the model's geometric accuracy based on the region division results. This ensures that subsequent accuracy adjustments can target problem areas and allocate computational resources reasonably, guaranteeing high accuracy in core abnormal regions to meet surgical needs while avoiding excessive precision calculations in non-abnormal regions that would waste resources. Thus, it provides accurate and efficient model support for joint surgery.

[0033] Furthermore, the steps for dynamically adjusting the geometric accuracy of the joint 3D model based on the region division results and joint categories include: model geometric accuracy parameters include sampling rate and mesh density; if the region division result is the first region, the sampling rate in the corresponding region of the joint 3D model is directly adjusted to the preset maximum sampling rate; if the region division result is the second region, the number of joint core anomalies is matched with a preset first sampling rate adjustment value mapping table to obtain the corresponding first sampling rate adjustment value, and the current sampling rate in the corresponding region of the joint 3D model is adjusted based on the first sampling rate adjustment value; if the region division result is the third region, the number of joint non-core anomalies is matched with a preset second sampling rate adjustment value mapping table to obtain the corresponding first sampling rate adjustment value. The sampling rate adjustment value mapping table is matched to obtain the corresponding second sampling rate adjustment value. Based on the second sampling rate adjustment value, the current sampling rate in the corresponding region of the joint 3D model is adjusted. If the region division result is the fourth region, the sampling rate in the corresponding region of the joint 3D model is directly adjusted to the preset minimum sampling rate value. The joint category is matched with the preset mesh density base value mapping table to obtain the corresponding mesh density base value. The number of the first region is counted and matched with the preset mesh density adjustment value mapping table to obtain the corresponding mesh density adjustment value. Based on the mesh density adjustment value, the mesh density base value is corrected to obtain the mesh density of the joint 3D model.

[0034] In this embodiment, the mapping table is a lookup table that maps specific input parameters (such as the number of core joint abnormalities, the number of non-core joint abnormalities, joint categories, or the number of first regions) to corresponding output adjustment values ​​(such as sampling rate adjustment values ​​or basic grid density values). By establishing a quantitative relationship between key indicators and model accuracy parameters, it provides data basis for achieving dynamic and differentiated adjustment of model geometric accuracy. The geometric accuracy parameters of this invention focus on two core parameters: sampling rate and grid density. Differentiated sampling rate adjustment strategies are adopted for different region division results: the maximum sampling rate is directly used for the first region with severe core abnormalities to ensure that the joint anatomical details and abnormal features in this region are accurately captured, meeting the high-precision requirements of core problem areas in surgical planning; for the second region with slight core abnormalities, the sampling rate is accurately adapted by matching the number of core abnormalities with the preset mapping table adjustment value, ensuring that abnormal details are not missed while avoiding oversampling and resource waste; for the third region with only non-core abnormalities, the sampling rate is adjusted according to the number of non-core abnormalities, reasonably reducing the computational load while controlling the accuracy to meet the standard; the minimum sampling rate is used for the fourth region with no abnormalities to maximize the saving of computational resources. In terms of mesh density adjustment, the base mesh density value is first matched according to the joint type to ensure that the density adapts to the differences in anatomical structure of different joints (such as the knee joint and hip joint). Then, the base value is corrected by matching the adjustment value through the statistical number of the first region. The more severe core abnormal regions there are, the higher the mesh density is, so that the overall density of the model not only fits the structural characteristics of the joint itself, but also supports the accuracy requirements of the core abnormal regions. This achieves a precise match between geometric accuracy and actual needs. It not only ensures high accuracy of core abnormal regions and key joints through targeted adjustment, providing accurate data support for surgical planning, but also avoids meaningless accuracy redundancy through differentiated resource allocation, significantly improving the model processing efficiency. At the same time, it takes into account the overall coordination and practicality of the model, laying a high-quality model foundation for the precise implementation of joint surgery.

[0035] like Figure 3 The diagram shown is a structural schematic of a joint surgery data fusion processing system based on multimodal images provided in this application embodiment. The joint surgery data fusion processing system based on multimodal images provided in this application embodiment includes: Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0036] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0037] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0038] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0039] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0040] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for fusion processing of joint surgical data based on multimodal images, characterized in that, Includes the following steps: The joint trajectory data of each pose point image in the multimodal image is acquired, the motion state is evaluated based on the joint trajectory data of each pose point image, and a reference joint image of the multimodal image is obtained based on the motion state evaluation result. The reference joint image is used to quantify the joint motion state and functional abnormality of the corresponding modal image, and provides accurate data support for the construction of the joint three-dimensional model. The joint position data in each reference joint image is acquired. The image accuracy is evaluated based on the joint position data in each reference joint image. Based on the image accuracy evaluation results, it is determined whether to dynamically correct the joint data. If so, the joint data is fused to form a three-dimensional joint model after dynamic correction. If not, the joint data is directly fused to form a three-dimensional joint model. Joint evaluation parameters are obtained from the 3D model of the joint. Based on the joint evaluation parameters in the 3D model, dynamic region division is performed, and the model accuracy is dynamically adjusted based on the region division results and joint category. The dynamic adjustment of model accuracy includes dynamic adjustment of sampling rate and dynamic adjustment of mesh density. The joint evaluation parameters include core joint parameters and non-core joint parameters. The core joint parameters are key indicators that directly determine the design of joint surgical plans, assessment of core anatomical functions, and selection of implants. The non-core joint parameters are supplementary indicators that assist in assessing the overall condition of the joint, affecting the optimization of surgical details, and prediction of postoperative recovery.

2. The method for fusion processing of joint surgical data based on multimodal imaging as described in claim 1, characterized in that, The steps for evaluating motion state based on joint trajectory data from each pose point image include: The joint trajectory data includes the deviation position and deviation angle of each joint marker point; The joint trajectory data of each pose point image in the multimodal image are compared and coupled with preset joint trajectory reference data. Then, the comparison results are weighted and coupled using preset joint trajectory scaling coefficients to obtain the joint trajectory deviation index of each pose point image. The joint trajectory reference data includes deviation position reference value and deviation angle reference value. The joint trajectory scaling coefficients include deviation position scaling coefficient and deviation angle scaling coefficient. The joint trajectory deviation index of each pose point image is compared with the preset joint trajectory deviation threshold. If the joint trajectory deviation index of any pose point image does not reach the joint trajectory deviation threshold, the corresponding pose point image is marked as a normal pose point image. If the joint trajectory deviation index of any pose point image reaches the joint trajectory deviation threshold, the corresponding pose point image will be classified as an abnormal pose point image.

3. The method for fusion processing of joint surgical data based on multimodal imaging as described in claim 2, characterized in that, The steps for obtaining reference joint images for multimodal imaging based on motion state assessment results include: Count the number of images with normal pose points and the number of images with abnormal pose points in multimodal images; Step 1: Determine whether the number of normal pose point images in each modal image exceeds the preset threshold for the number of normal pose point images. If so, proceed to Step 2; otherwise, proceed to the reshooting process. Step 2: Determine whether the number of abnormal posture point images exceeds the preset threshold for the number of abnormal posture point images. If so, proceed to the reshooting process; otherwise, mark the difference between the joint trajectory deviation index of each abnormal posture point image and the joint trajectory deviation threshold as the joint trajectory deviation margin. The joint trajectory deviation margin of each abnormal posture point image is compared with the preset joint trajectory abnormality threshold. If the joint trajectory deviation margin of any abnormal posture point image does not exceed the joint trajectory abnormality threshold, the corresponding abnormal posture point image is marked as a repairable posture point image. If the joint trajectory deviation margin of any abnormal pose point image exceeds the joint trajectory abnormality threshold, the corresponding abnormal pose point image will be marked as an unrepairable pose point image. Step 3: Determine whether the number of irreparable pose point images exceeds the preset threshold for the number of irreparable pose point images. If so, proceed to the reshooting process; otherwise, dynamically acquire the reference joint image of the corresponding modal image based on the pose point priority order.

4. The method for fusion processing of joint surgical data based on multimodal imaging as described in claim 3, characterized in that, The step of dynamically acquiring the reference joint image of the corresponding modal image based on the priority order of pose points includes: The pose points in the modal image that have preset key angles are marked as high-priority pose points; Pose points in the modal image that do not have a preset key angle are marked as low-priority pose points; Using joint data from high-priority pose points as constraint points, the initial joint motion trajectory is obtained by interpolating and fitting the motion paths between high-priority pose points. Calculate the Euclidean distance between the spatial coordinates of the low-priority pose point and the three-dimensional spatial coordinates of the corresponding position point on the initial joint motion trajectory. Use the Euclidean distance as the core element to construct the objective function. Set the constraint condition of the trajectory curve as the sum of the squares of the second derivatives of the trajectory curve being less than a preset smoothing threshold. By minimizing the objective function to optimize the curve parameters of the initial joint motion trajectory, the optimized initial joint motion trajectory is fused with the anatomical information of the corresponding modal image to generate a reference joint image.

5. The method for fusion processing of joint surgical data based on multimodal imaging as described in claim 1, characterized in that, The steps for evaluating image accuracy based on joint position data in each reference joint image include: For any two sets of reference joint images in multimodal imaging, joint measurement data are obtained based on the joint position data of the two sets of images. The joint measurement data includes the average distance difference of each joint landmark, the maximum distance difference of each joint landmark, and the structural similarity. The average distance difference, maximum distance difference, and structural similarity reference value of each joint landmark are compared with the preset average distance reference value, maximum distance reference value, and joint similarity. Then, the comparison results are weighted and coupled using preset joint measurement scaling coefficients to obtain the image deviation evaluation index between the two sets of images. The joint measurement reference data includes the average distance reference value, joint similarity reference value, and maximum distance reference value. The joint measurement scaling coefficients include the average distance scaling coefficient, joint similarity scaling coefficient, and maximum distance scaling coefficient. The image deviation evaluation index between each pair of images is compared with the preset image deviation evaluation threshold. If the image deviation evaluation index exceeds the image deviation evaluation threshold, it is marked as a low-fit image pair. If the image deviation assessment index does not exceed the image deviation assessment threshold, it is marked as a highly adapted image pair.

6. The method for fusion processing of joint surgical data based on multimodal imaging as described in claim 5, characterized in that, The step of determining whether to dynamically correct joint data based on the image accuracy assessment results includes: The proportion of highly adapted image pairs is counted. If the proportion of highly adapted image pairs exceeds the preset overall fusion threshold, no additional processing is performed. If the proportion of highly adapted image pairs does not exceed the preset overall fusion threshold, images in low-adapted image pairs that appear more than the preset threshold number of times are marked as abnormal images, and images in highly adapted image pairs that appear more than the preset threshold number of times are marked as standard images. The joint data of abnormal images are dynamically corrected based on the average coordinates of each joint marker point in the standard images and the joint measurement data of the abnormal images until the proportion of highly adapted image pairs exceeds the preset overall fusion threshold.

7. The method for fusion processing of joint surgical data based on multimodal imaging as described in claim 6, characterized in that, The steps for dynamically correcting the joint data of abnormal images based on the average coordinates of each joint marker point in the standard image and the joint measurement data of the abnormal image include: The average joint measurement data between the abnormal image and the standard image is obtained. The average joint measurement data includes the average deviation distance value of each joint landmark, the maximum distance difference of each joint landmark, and the average structural similarity. According to the order of joint measurement ratio coefficient from largest to smallest, joint measurement data in abnormal images that do not meet the preset threshold conditions are corrected sequentially, specifically including: If the average distance difference of any joint marker in the abnormal image exceeds the preset distance difference threshold, the coordinates of the corresponding joint marker in the abnormal image are gradually corrected according to the preset coordinate correction unit based on the average position coordinates of the corresponding joint marker in the standard image until the average distance difference of the corresponding joint marker does not exceed the preset distance difference threshold. If the maximum distance difference of any joint marker in the abnormal image exceeds the preset maximum distance difference threshold, the coordinates of the corresponding joint marker in the abnormal image are gradually corrected according to the preset coordinate correction unit based on the average position coordinates of the corresponding joint marker in the standard image until the maximum distance difference of the corresponding joint marker does not exceed the preset maximum distance difference threshold. If the average structural similarity of the abnormal image is lower than the preset structural similarity threshold, structural similarity correction is performed based on the average coordinates of each joint marker in the standard image until the average structural similarity of the abnormal image is not lower than the structural similarity threshold.

8. The method for fusion processing of joint surgical data based on multimodal imaging as described in claim 1, characterized in that, The steps for dynamic region division based on joint evaluation parameters in the 3D joint model include: The core joint parameters include at least one of the following: lower limb force line angle, osteotomy surface angle, key bony anatomical dimensions, medullary cavity diameter, bone defect size, cartilage damage extent, and ligament attachment point displacement. The non-core parameters of the joint include at least one of the following: joint alignment deviation, articular surface coverage, osteophyte size and location, and joint range of motion limit angle. The joint assessment parameters of each region are compared with the preset joint assessment reference range. The number of core joint parameters in each region that exceed the preset joint assessment reference range is counted as the number of core joint abnormalities. The number of non-core joint parameters that exceed the preset joint assessment reference range is counted as the number of non-core joint abnormalities. If the number of joint core abnormalities in any region exceeds the preset threshold for the number of core abnormalities, the corresponding region will be marked as the first region. If the number of joint core abnormalities in any region exceeds 0 but does not exceed the preset threshold for the number of core abnormalities, then the corresponding region will be marked as the second region. If the number of joint core abnormalities in any region is 0, but the number of joint non-core abnormalities exceeds 0, then the corresponding region is marked as the third region. If the number of joint core abnormalities and the number of joint non-core abnormalities in any region are both 0, then the corresponding region is marked as the fourth region.

9. The method for fusion processing of joint surgical data based on multimodal imaging as described in claim 8, characterized in that, The steps for dynamically adjusting model accuracy based on region segmentation results and joint categories include: If the region division result is the first region, then directly adjust the sampling rate of the corresponding region of the joint 3D model to the preset maximum sampling rate. If the region division result is the second region, the number of joint core abnormalities is matched with the preset first sampling rate adjustment value mapping table to obtain the corresponding first sampling rate adjustment value, and the current sampling rate in the corresponding region of the joint three-dimensional model is adjusted based on the first sampling rate adjustment value. If the region division result is the third region, the number of non-core abnormalities of the joint is matched with the preset second sampling rate adjustment value mapping table to obtain the corresponding second sampling rate adjustment value. Based on the second sampling rate adjustment value, the current sampling rate in the corresponding region of the joint 3D model is adjusted. If the region division result is the fourth region, then directly adjust the sampling rate of the corresponding region of the joint 3D model to the preset minimum sampling rate. Match the joint category with the preset mesh density base value mapping table to obtain the corresponding mesh density base value; The number of first regions is counted, and the number of first regions is matched with a preset mesh density adjustment value mapping table to obtain the corresponding mesh density adjustment value. Based on the mesh density adjustment value, the basic mesh density value is corrected to obtain the mesh density of the joint 3D model.

10. A joint surgery data fusion processing system based on multimodal images, employing the joint surgery data fusion processing method based on multimodal images as described in any one of claims 1 to 9, characterized in that, It includes an image acquisition module, a precision calibration module, and a model optimization module: The image acquisition module is used to acquire joint trajectory data of each pose point image in the multimodal image, perform motion state evaluation based on the joint trajectory data of each pose point image, and obtain reference joint images of the multimodal image based on the motion state evaluation results. The accuracy calibration module is used to acquire joint position data in each reference joint image, evaluate the image accuracy based on the joint position data in each reference joint image, and determine whether to dynamically correct the joint data based on the image accuracy evaluation results. If yes, the joint data is dynamically corrected and then fused to form a three-dimensional joint model; otherwise, the joint data is directly fused to form a three-dimensional joint model. The model optimization module is used to obtain joint evaluation parameters in the three-dimensional joint model, perform dynamic region division based on the joint evaluation parameters in the three-dimensional joint model, and dynamically adjust the model accuracy based on the region division results and joint category.

Images