Caput femoris osteotomy operation planning method and device based on multi-modal image registration

Abstract

The invention provides a femoral head osteotomy operation planning method and device based on multi-modal image registration, and the method comprises the steps: obtaining a multi-modal medical image of any object, and the multi-modal medical image at least comprises an X-ray image, a CT image and an MRI image; fusing the X-ray image, the CT image and the MRI image to obtain a fused image; and based on the fused image, planning the femoral neck basal rotation osteotomy. According to the method, image data of different modals are integrated by utilizing a multi-modal fusion technology, more comprehensive and accurate information of the femoral head and the necrotic area is provided, and accurate surgical planning of femoral head osteotomy is realized on the basis, so that the surgical accuracy is improved, and postoperative complications are reduced.

Description

Technical Field

[0001] This application relates to the field of medical image processing technologies, and more particularly, to a femoral head osteotomy surgical planning method and apparatus based on multimodal image registration. Background Art

[0002] In surgical treatment, femoral neck basal rotational osteotomy is an important hip-preserving surgical technique. Its purpose is to move the necrotic area out of the weight-bearing area of the femoral head, use normal bone for weight-bearing, reduce interference with the blood supply of the femoral head, and not affect subsequent joint replacement.

[0003] However, due to the complexity of the femoral head necrosis area, traditional osteotomy has certain limitations, including problems such as insufficient surgical precision and the risk of postoperative complications. Therefore, there is an urgent need for a specific solution that can perform precise surgical planning. Summary of the Invention

[0004] The problem solved by this application is the lack of accurate preoperative planning for femoral head osteotomy.

[0005] To solve the above problems, the first aspect of this application provides a femoral head osteotomy surgical planning method based on multimodal image registration, which includes:

[0006] Obtain multimodal medical images of any subject, where the multimodal medical images at least include X-ray images, CT images, and MRI images;

[0007] Fuse the X-ray images, CT images, and MRI images to obtain a fused image;

[0008] Based on the fused image, plan the femoral neck basal rotational osteotomy.

[0009] The second aspect of this application provides a manufacturing system for a femoral head osteotomy surgical planning method based on multimodal image registration, which includes:

[0010] An image acquisition module, which is used to obtain multimodal medical images of any subject, where the multimodal medical images at least include X-ray images, CT images, and MRI images;

[0011] An image fusion module, which is used to fuse the X-ray images, CT images, and MRI images to obtain a fused image;

[0012] An osteotomy planning module, which is used to plan the femoral neck basal rotational osteotomy based on the fused image.

[0013] The third aspect of this application provides an electronic device, including: a memory and a processor; the memory can be configured to store a program, and the processor is coupled to the memory and is used to execute the program in the memory for:

[0014] Obtain multi-modal medical images of any object, where the multi-modal medical images at least include X-ray images, CT images, and MRI images;

[0015] Fuse the X-ray images, CT images, and MRI images to obtain a fused image;

[0016] Based on the fused image, plan femoral neck base rotational osteotomy.

[0017] The fourth aspect of this application provides a computer-readable storage medium with a computer program stored thereon, and the program is executed by a processor to implement the foregoing femoral head osteotomy surgical planning method based on multi-modal image registration.

[0018] In this application, multi-modal fusion technology is used to integrate imaging data of different modalities, providing more comprehensive and accurate information on the femoral head and the necrotic area. On this basis, precise surgical planning for femoral head osteotomy is achieved, thereby improving surgical accuracy and reducing postoperative complications. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of the femoral head osteotomy surgical planning method based on multi-modal image registration according to an embodiment of this application;

[0020] Figure 2 It is a schematic diagram of the femoral head osteotomy surgical planning result based on multi-modal image registration according to an embodiment of this application;

[0021] Figure 3 It is an architecture diagram of the femoral head osteotomy surgical planning device based on multi-modal image registration according to an embodiment of this application;

[0022] Figure 4 It is an architecture diagram of an electronic device according to an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] To make the above objects, features, and advantages of this application more obvious and understandable, the following detailed description of the specific embodiments of this application is provided in conjunction with the accompanying drawings. Although the exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0024] It should be noted that unless otherwise specified, the technical terms or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art to which this application belongs.

[0025] Avascular necrosis of the femoral head is a common orthopedic disease, and its treatment methods are diverse, including non-surgical treatment and surgical treatment. Among them, osteotomy is one of the important surgical methods for treating femoral head lesions, which can avoid or delay the patient from undergoing artificial joint replacement, thus preserving the patient's own femoral head. However, in the past, osteotomy was performed between the femoral trochanters, which was prone to damage the branches of the medial circumflex femoral artery and cause avascular necrosis of the femoral head. Due to the lack of precise positioning of the necrotic area and surgical planning, the postoperative effect of the femoral neck basal rotational osteotomy was severely affected. With the development of medical imaging technologies, such as the application of multi-modal imaging technologies such as CT and MRI, more accurate information has been provided for the judgment and treatment of avascular necrosis of the femoral head. These technologies can provide rich anatomical and pathological information, which helps doctors to perform more accurate preoperative planning and surgical operations.

[0026] In surgical treatment, femoral neck basal rotational osteotomy is an important hip-preserving surgical technique, the purpose of which is to move the necrotic area out of the weight-bearing area of the femoral head, use normal bone for weight-bearing, reduce the interference with the blood supply of the femoral head, and does not affect later joint replacement. However, due to the complexity of the avascular necrosis area of the femoral head, traditional osteotomy has certain limitations, including problems such as insufficient surgical precision and the risk of postoperative complications.

[0027] Therefore, there is an urgent need for a specific solution that can perform precise surgical planning.

[0028] The embodiment of the present application provides the above-mentioned femoral head osteotomy surgical planning method based on multi-modal image registration. The specific solution of this method is Figure 1 - Figure 2 shown. This method can be executed by a femoral head osteotomy surgical planning device based on multi-modal image registration. The femoral head osteotomy surgical planning device based on multi-modal image registration can be integrated in electronic devices such as computers, servers, computers, server clusters, and data centers. Combining Figure 1 shown, wherein, the femoral head osteotomy surgical planning method based on multi-modal image registration includes:

[0029] S101, obtaining multi-modal medical images of any object, where the multi-modal medical images at least include X-ray images, CT images, and MRI images;

[0030] In the present application, X-ray images: fast, low-cost, and provide two-dimensional skeletal structure information. Used to evaluate the rotation of the femoral neck base, skeletal alignment, and deformities.

[0031] In the present application, CT images: provide high-resolution three-dimensional skeletal anatomical information. Used to accurately locate the osteotomy point of the femoral neck base and determine the three-dimensional spatial relationship of the bones.

[0032] In this application, MRI images: High-contrast soft tissue imaging, showing articular cartilage, tendons, ligaments, etc. Evaluate the surrounding soft tissue conditions to prevent damage to important tissues during osteotomy.

[0033] In this application, through X-ray images, anteroposterior and lateral images are obtained to ensure clear visualization of the femoral neck region.

[0034] In this application, through CT images, high-resolution three-dimensional CT scans are performed to obtain the detailed anatomical structure of the femur.

[0035] In this application, through MRI images, detailed information about the femoral head and surrounding soft tissues is obtained to help determine the type of lesion.

[0036] S102, fuse the X-ray image, CT image, and MRI image to obtain a fused image;

[0037] S103, based on the fused image, plan the femoral neck base rotational osteotomy.

[0038] In this application, the multi-modal fusion technology is used to integrate different modal imaging data, providing more comprehensive and accurate information about the femoral head and the necrotic area. On this basis, an accurate surgical plan for femoral head osteotomy is realized, thereby improving the surgical accuracy and reducing postoperative complications.

[0039] In this application, the research background of exploring the multi-modal fusion intelligent planning system combined with femoral neck base rotational osteotomy is that by using advanced medical imaging technology and image processing technology, the surgical accuracy and safety can be improved. The multi-modal fusion technology can integrate different modal imaging data, providing more comprehensive and accurate information about the femoral head and the necrotic area, thus providing more accurate guidance for surgical planning. Through the intelligent planning system, doctors can design a more reasonable osteotomy line, predict the surgical effect, reduce the surgical risk, and improve the surgical success rate. In addition, the multi-modal fusion intelligent planning system can also assist doctors in postoperative evaluation, monitor the surgical effect and the patient's recovery, providing a more comprehensive solution for the treatment of femoral head necrosis. Therefore, the research and development of the application of the multi-modal fusion intelligent planning system in femoral neck base rotational osteotomy has important clinical significance and research value for improving the surgical effect and the patient's quality of life.

[0040] In this application, a multi-modal imaging (X-ray, CT, magnetic resonance) fusion technology is adopted to integrate information from different imaging modalities, providing more comprehensive information for the assessment of the femoral head necrosis area and enabling better assessment of the integrity rate of the lateral column. This method can effectively overcome the limitations brought by a single imaging modality, provide more accurate anatomical structure and pathological information, and support doctors in making more precise preoperative plans. It avoids inaccurate assessment caused by a single image, thereby implementing an incorrect surgical plan (osteotomy for patients who should not have osteotomy, or replacement surgery for patients who still have the opportunity for osteotomy-preserving hip surgery due to incorrect assessment), resulting in unsatisfactory surgical outcomes.

[0041] In one embodiment, the X-ray image includes a frontal X-ray image and a lateral X-ray image.

[0042] In this application, the frontal X-ray image: provides frontal skeletal structure information, suitable for observing bone alignment, symmetry of anatomical landmarks, and angles. It assesses the anatomical relationships in the hip joint or the base area of the femoral neck, such as the position of the femoral head, joint space, and anteversion angle of the femoral neck.

[0043] In this application, the lateral X-ray image: provides information for observing the femoral neck, femoral shaft, and joint structure from the side. It is used to supplement the frontal image, provides the relationship between the femoral neck and the acetabulum from a lateral perspective, and helps measure the base angle of the femoral neck.

[0044] In one embodiment, before fusing the X-ray image, CT image, and MRI image to obtain a fused image, it further includes:

[0045] Preprocessing the X-ray image, CT image, and MRI image respectively to obtain preprocessed X-ray image, CT image, and MRI image.

[0046] In this application, through preprocessing, the quality of the images is improved, and the key features of bones and soft tissues are enhanced; the contrast, brightness, and resolution of different modality images are unified to ensure the fusion effect of multi-modal images.

[0047] In one embodiment, preprocessing the X-ray image, CT image, and MRI image respectively includes:

[0048] Filtering the X-ray image, CT image, and MRI image through a median filtering algorithm;

[0049] Performing image enhancement processing on the X-ray image, CT image, and MRI image through histogram equalization processing;

[0050] Normalizing the X-ray image, CT image, and MRI image after image enhancement.

[0051] In this application, median filtering is a non-linear filtering method used to replace pixel values, based on which noise in the image is removed while preserving the edge information of the image, especially the key features of the bone structure.

[0052] Through median filtering, artifacts or random noise in X-ray images can be reduced, high-frequency noise caused by equipment or reconstruction algorithms in CT images can be eliminated, and motion artifacts or gradient noise in MRI images can be suppressed.

[0053] In this application, histogram equalization stretches the gray value distribution of the image, redistributes the pixel gray values over the entire dynamic range, and enhances the contrast. Based on this, the contrast of the image can be enhanced, making the features of bones and soft tissues clearer.

[0054] In this application, through histogram equalization, the contrast between bone edges and soft tissues in X-ray images can be enhanced, the contrast between different bone regions (such as the femoral neck and base) in CT images can be enhanced, and the features of soft tissues (such as tendons and ligaments) in MRI images can be highlighted.

[0055] In this application, different modalities of image data are standardized to a unified gray scale range for multi-modal fusion.

[0056] In this application, Gaussian filtering or median filtering algorithms are used to remove random noise in the images and improve the image quality.

[0057] In this application, histogram equalization technology is applied to enhance the contrast of the images and make the key structures more obvious.

[0058] In this application, different modalities of images are standardized to eliminate image differences caused by different imaging conditions.

[0059] In one implementation, the X-ray image, CT image, and MRI image are fused to obtain a fused image, including:

[0060] Edge detection is performed on the X-ray image, CT image, and MRI image respectively to obtain the corresponding edge information;

[0061] Feature point extraction is performed on the X-ray image, CT image, and MRI image respectively to obtain the corresponding descriptors;

[0062] The X-ray image, CT image, and MRI image with edge information and descriptors are fused to obtain a fused image.

[0063] In this application, through edge detection, the significant structural boundaries in the image are extracted, especially the edge features of bones and soft tissues, providing a geometric and morphological basis for image fusion.

[0064] In this application, through feature point extraction, key feature points and regions are extracted from the image to represent and match the geometric structures of different modality images.

[0065] In this application, the Canny algorithm is used to extract the edge information of the image to help clarify the contour of the bone structure.

[0066] In this application, the SIFT or ORB algorithm is adopted to extract the key feature points in the image and calculate their descriptors for subsequent registration use.

[0067] In one implementation, the X-ray image, CT image, and MRI image with edge information and descriptors are fused to obtain a fused image, including:

[0068] Register the X-ray image and CT image with edge information and descriptors to obtain the mapping relationship between the X-ray image and CT image;

[0069] Register the CT image and MRI image with edge information and descriptors to obtain the mapping relationship between the CT image and MRI image;

[0070] Based on the mapping relationship between the X-ray image and CT image and the mapping relationship between the CT image and MRI image, the X-ray image, CT image, and MRI image are fused to obtain a fused image.

[0071] In this application, the mapping relationship between multi-modal images (X-ray image, CT image, and MRI image) is determined through registration, and these relationships are used to integrate multi-modal information into a single fused image. The specific process is step-by-step registration and fusion based on known relationships.

[0072] In this application, the CT image contains three-dimensional bone details. Based on these three-dimensional bone details, the CT image can be accurately registered with the MRI image; and can also be accurately registered with the bone contour of the X-ray image.

[0073] In this application, taking the CT image as the intermediate image, it is registered with the X-ray image and MRI image respectively, so that on the basis of achieving accurate registration, the registration efficiency can also be increased.

[0074] In this application, based on the registered multi-modal images, it can be a pixel-level fused image obtained by weighted superposition; or a feature-level fused image generated by integrating the aligned feature points and edge information to generate a high-dimensional feature representation.

[0075] In this application, the fused image contains comprehensive information of multiple modalities, such as the bone contour of the X-ray, the three-dimensional structural details of the CT, and the soft tissue features of the MRI.

[0076] In one embodiment, when fusing the X-ray image, CT image, and MRI image with edge information and descriptors to obtain a fused image, the registration between any two images may include:

[0077] Feature matching: Finding corresponding feature points in multimodal images to achieve alignment:

[0078] Brute-force matching: For each feature point descriptor, calculating its Euclidean distance from all descriptors in the target image to find the nearest neighbor match; FLANN (Fast Library for Approximate Nearest Neighbors): Using approximate nearest neighbor search to accelerate the matching process; Distance ratio test: Retaining the matching points where the distance ratio between the nearest neighbor and the second nearest neighbor is less than a certain threshold; RANSAC optimization: Using the RANSAC (Random Sample Consensus) algorithm to eliminate incorrect matching points, fitting a model by the least squares method, and eliminating the points that do not conform to the model. Based on this, effective matching point pairs between multimodal images are obtained.

[0079] Geometric transformation: Calculating registration parameters based on the matching point pairs for image spatial alignment:

[0080] Calculating registration parameters: Using the matching point pairs to calculate affine transformation or perspective transformation parameters by the least squares method; Optimizing registration: Using RANSAC to eliminate incorrect matching points to ensure the robustness of the registration parameters; Transforming the image: Using the calculated registration parameters to transform the X-ray, CT, and MRI images into a unified coordinate system. Based on this, aligned multimodal images are obtained.

[0081] In this application, during the registration process of any two images, feature point matching is performed using brute-force matching or the FLANN algorithm to determine the corresponding relationship between different modality images; calculating the Euclidean distance of feature points to screen effective matching points to improve the accuracy of matching; applying the least squares method to calculate registration parameters and performing spatial alignment of the images using affine transformation or perspective transformation; using the RANSAC algorithm to eliminate incorrect matches and optimize the registration result.

[0082] It should be noted that the registration in this application is pairwise registration, but the specific fusion is the simultaneous fusion of three images.

[0083] In one embodiment, based on the fused image, planning for femoral neck basal rotational osteotomy includes:

[0084] Based on the fused image, confirming the size and location of the femoral head necrosis area;

[0085] Based on the femoral head necrosis area, planning the osteotomy location of femoral neck basal rotational osteotomy;

[0086] Displaying the planned osteotomy location in a three-dimensional bone model.

[0087] In this application, based on the registered images, the position and size of the osteonecrosis area of the femoral head are evaluated.

[0088] In this application, the position and size of the osteonecrosis area of the femoral head can be obtained by processing through a trained deep convolutional model.

[0089] In this application, combined with the doctor's clinical experience and the patient's individual differences, a reasonable femoral neck base rotational osteotomy plan is designed, and the judgment basis is: the intact rate of the lateral column of the femoral head after rotation is not less than 34% of the entire eyebrow arch.

[0090] In this application, the three-dimensional visualization technology is used to intuitively display the preoperative planning results for doctors' reference.

[0091] In this application, the preoperative planning results can be Figure 2 as shown in the schematic diagram of the femoral head osteotomy surgical planning results.

[0092] In this application, through the registration of multi-modal image data and the accurate identification of the osteonecrosis area of the femoral head, comprehensive image information is provided, thereby improving the accuracy of the preoperative planning of femoral neck base rotational osteotomy and providing a reliable and accurate surgical plan for surgeons to perform femoral neck base rotational osteotomy.

[0093] In this application, a reliable and accurate surgical plan is provided for surgeons to perform femoral neck base rotational osteotomy, improving the doctor's decision-making efficiency and promoting the implementation of personalized medicine. This technical method improves the accuracy of the preoperative planning of femoral neck base rotational osteotomy, better improves the intact rate of the lateral column of the femoral head, reduces the risk of postoperative femoral head collapse, improves the success rate and surgical effect of osteotomy treatment for femoral head necrosis, and reduces the risk of surgical complications and secondary surgery.

[0094] In this application, a multi-modal image data (X-ray, CT scan, and magnetic resonance imaging (MRI)) integration system is adopted. Through algorithms, multiple image modalities are effectively integrated to achieve more comprehensive information acquisition of the femoral head and the necrosis area, and accurate positioning of the osteonecrosis area of the femoral head is realized. The intact rate of the lateral column of the femoral head can be better evaluated. This method can effectively overcome the limitations brought by a single image modality, provide more accurate anatomical structure and pathological information, and support doctors to perform more accurate preoperative planning.

[0095] In this application, the image preprocessing steps are aimed at improving the quality and comparability of different modality images. By applying denoising algorithms (such as Gaussian filtering, median filtering, etc.) to remove random noise in the images, enhancement techniques such as histogram equalization are used to improve the image contrast, and different modality images are standardized to eliminate imaging condition differences. This series of preprocessing steps lays a good foundation for subsequent feature extraction and registration, ensuring that the final image quality meets the clinical application standards.

[0096] In this application, an intelligent preoperative planning system is constructed, which can generate a personalized femoral neck base rotational osteotomy plan according to multimodal image data and registration results. In this application, by comprehensively analyzing the patient's image data, the relationship between the weight-bearing area and the necrotic area of the femoral head is evaluated, and the osteotomy line design is optimized to ensure the rationality and effectiveness of the surgical plan.

[0097] In this application, advanced algorithms are combined with clinical knowledge to provide scientific decision-making support for surgeons and promote the implementation of personalized medicine.

[0098] In this application, combined with the doctor's clinical experience and the patient's individual differences, a reasonable femoral neck base rotational osteotomy plan is designed to ensure that the intact rate of the lateral column of the femoral head after rotation is not less than 34% of the entire eyebrow arch. In this way, the biomechanical strength of the lateral column and the surgical success rate are greatly improved.

[0099] In this application, a more reasonable femoral neck base rotational osteotomy plan is designed to ensure that the necrotic area of the surgical patient can be effectively removed from the weight-bearing area, and at the same time, the weight-bearing area can obtain sufficient mechanical support, which can greatly improve the surgical success rate and scientific predictability. Correspondingly, this optimized design reduces the interference with the blood supply of the femoral head and improves the surgical success rate.

[0100] An embodiment of this application provides a femoral head osteotomy surgical planning device based on multimodal image registration, which is used to execute the femoral head osteotomy surgical planning method based on multimodal image registration described above. The femoral head osteotomy surgical planning device based on multimodal image registration will be described in detail below.

[0101] As Figure 3 shown, the femoral head osteotomy surgical planning device based on multimodal image registration includes:

[0102] An image acquisition module 101, which is used to acquire multimodal medical images of any object, and the multimodal medical images at least include X-ray images, CT images, and MRI images;

[0103] An image fusion module 102, which is used to fuse the X-ray images, CT images, and MRI images to obtain a fused image;

[0104] An osteotomy planning module 103, which is used to plan the femoral neck base rotational osteotomy based on the fused image.

[0105] In one embodiment, the X-ray images include X-ray anteroposterior images and X-ray lateral images.

[0106] In one embodiment, the image fusion module 102 is further used for:

[0107] Preprocess the X-ray image, CT image, and MRI image respectively to obtain the preprocessed X-ray image, CT image, and MRI image.

[0108] In one embodiment, the image fusion module 102 is further configured to:

[0109] Filter the X-ray image, CT image, and MRI image through a median filtering algorithm; perform image enhancement processing on the X-ray image, CT image, and MRI image through histogram equalization processing; and standardize the X-ray image, CT image, and MRI image after image enhancement.

[0110] In one embodiment, the image fusion module 102 is further configured to:

[0111] Perform edge detection on the X-ray image, CT image, and MRI image respectively to obtain corresponding edge information; extract feature points from the X-ray image, CT image, and MRI image respectively to obtain corresponding descriptors; and fuse the X-ray image, CT image, and MRI image with edge information and descriptors to obtain a fused image.

[0112] In one embodiment, the image fusion module 102 is further configured to:

[0113] Register the X-ray image and CT image with edge information and descriptors to obtain the mapping relationship between the X-ray image and CT image; register the CT image and MRI image with edge information and descriptors to obtain the mapping relationship between the CT image and MRI image; and fuse the X-ray image, CT image, and MRI image based on the mapping relationship between the X-ray image and CT image and the mapping relationship between the CT image and MRI image to obtain a fused image.

[0114] In one embodiment, the osteotomy planning module 103 is further configured to:

[0115] Based on the fused image, confirm the size and location of the femoral head necrosis area; plan the osteotomy position of the femoral neck base rotational osteotomy based on the femoral head necrosis area; and display the planned osteotomy position in the three-dimensional bone model.

[0116] The femoral head osteotomy surgery planning device based on multi-modal image registration provided in the above embodiments of the present application has a corresponding relationship with the femoral head osteotomy surgery planning method based on multi-modal image registration provided in the embodiments of the present application. Therefore, the specific content in this system has a corresponding relationship with the femoral head osteotomy surgery planning method based on multi-modal image registration. The specific content can be referred to the records in the femoral head osteotomy surgery planning method based on multi-modal image registration, and will not be elaborated herein.

[0117] The femoral head osteotomy surgical planning device provided by the above embodiments of the present application and the femoral head osteotomy surgical planning method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0118] The internal functions and structures of the femoral head osteotomy surgical planning device based on multimodal image registration are described above. As Figure 4 shown, in practice, the femoral head osteotomy surgical planning device based on multimodal image registration can be implemented as an electronic device, including: a memory 301 and a processor 303.

[0119] The memory 301 can be configured to store programs.

[0120] In addition, the memory 301 can also be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any application program or method for operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0121] The memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk or an optical disc. The processor 303, coupled to the memory 301, is used to execute the programs in the memory 301 for:

[0122] Obtaining multimodal medical images of any object, where the multimodal medical images at least include X-ray images, CT images, and MRI images;

[0123] Fusing the X-ray images, CT images, and MRI images to obtain a fused image;

[0124] Based on the fused image, planning femoral neck base rotational osteotomy.

[0125] In one implementation, the processor 303 is further used for:

[0126] Preprocessing the X-ray images, CT images, and MRI images respectively to obtain preprocessed X-ray images, CT images, and MRI images.

[0127] In one implementation, the processor 303 is further used for:

[0128] Filter the X-ray images, CT images, and MRI images through a median filtering algorithm; perform image enhancement processing on the X-ray images, CT images, and MRI images through histogram equalization processing; standardize the X-ray images, CT images, and MRI images after image enhancement.

[0129] In one embodiment, the processor 303 is further configured to:

[0130] Perform edge detection on the X-ray images, CT images, and MRI images respectively to obtain corresponding edge information; perform feature point extraction on the X-ray images, CT images, and MRI images respectively to obtain corresponding descriptors; fuse the X-ray images, CT images, and MRI images with edge information and descriptors to obtain a fused image.

[0131] In one embodiment, the processor 303 is further configured to:

[0132] Register the X-ray images and CT images with edge information and descriptors to obtain the mapping relationship between the X-ray images and CT images; register the CT images and MRI images with edge information and descriptors to obtain the mapping relationship between the CT images and MRI images; fuse the X-ray images, CT images, and MRI images based on the mapping relationship between the X-ray images and CT images and the mapping relationship between the CT images and MRI images to obtain a fused image.

[0133] In one embodiment, the processor 303 is further configured to:

[0134] Based on the fused image, confirm the size and location of the femoral head necrosis area; plan the osteotomy position of the femoral neck base rotation osteotomy based on the femoral head necrosis area; display the planned osteotomy position in the three-dimensional bone model.

[0135] In this application, Figure 4 only some components are schematically shown, which does not mean that the electronic device only includes Figure 4 the components shown.

[0136] The electronic device provided in this embodiment and the femoral head osteotomy surgery planning method based on multi-modal image registration provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by the application programs stored therein.

[0137] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce a means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction means that implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0140] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory. The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (Flash RAM). The memory is an example of computer-readable media.

[0141] The present application also provides a computer-readable storage medium corresponding to the femoral head osteotomy surgery planning method based on multimodal image registration provided in the aforementioned embodiment, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the interactive image analysis auxiliary method for 3D aerial imaging provided in any of the aforementioned embodiments.

[0142] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0143] The computer-readable storage medium provided in the above-mentioned embodiment of the present application and the interactive image analysis auxiliary method for 3D aerial imaging provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0144] It should be noted that in the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0145] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0146] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A femoral head osteotomy surgery planning method based on multimodal image registration, characterized in that: include: Acquire a multimodal medical image of any object, wherein the multimodal medical image includes at least an X-ray image, a CT image, and an MRI image; fusing the X-ray image, the CT image and the MRI image to obtain a fused image; Based on the fused images, the femoral neck base rotation osteotomy was planned.

2. The femoral head osteotomy surgery planning method based on multimodal image registration according to claim 1, characterized in that: The X-ray images include X-ray frontal images and X-ray lateral images.

3. The femoral head osteotomy surgery planning method based on multimodal image registration according to claim 1, characterized in that: Before fusing the X-ray image, the CT image and the MRI image to obtain a fused image, the method further includes: The X-ray image, CT image and MRI image are preprocessed respectively to obtain the preprocessed X-ray image, CT image and MRI image.

4. The femoral head osteotomy surgery planning method based on multimodal image registration according to any one of claims 1 to 3, characterized in that: The X-ray image, CT image and MRI image are preprocessed respectively, including: Performing filtering processing on the X-ray image, CT image and MRI image by using a median filtering algorithm; Performing image enhancement processing on the X-ray image, CT image and MRI image through histogram equalization processing; The X-ray image, CT image and MRI image after image enhancement are standardized.

5. The femoral head osteotomy surgery planning method based on multimodal image registration according to any one of claims 1 to 3, characterized in that: Fusing the X-ray image, the CT image and the MRI image to obtain a fused image includes: Performing edge detection on the X-ray image, CT image and MRI image respectively to obtain corresponding edge information; Extracting feature points from the X-ray image, the CT image, and the MRI image to obtain corresponding descriptors; The X-ray image, CT image and MRI image with edge information and descriptors are fused to obtain a fused image.

6. The femoral head osteotomy surgery planning method based on multimodal image registration according to claim 5, characterized in that: The X-ray image, the CT image and the MRI image having edge information and descriptors are fused to obtain a fused image, including: Registering the X-ray image and the CT image with edge information and descriptors to obtain a mapping relationship between the X-ray image and the CT image; Registering the CT image and the MRI image having edge information and descriptors to obtain a mapping relationship between the CT image and the MRI image; Based on the mapping relationship between the X-ray image and the CT image, and the mapping relationship between the CT image and the MRI image, the X-ray image, the CT image and the MRI image are fused to obtain a fused image.

7. The femoral head osteotomy surgery planning method based on multimodal image registration according to any one of claims 1 to 3, characterized in that: Based on the fusion images, the femoral neck base rotation osteotomy is planned, including: Based on the fused images, the size and location of the femoral head necrosis area were confirmed; Based on the area of ​​femoral head necrosis, the osteotomy position of the femoral neck base rotation osteotomy was planned; The planned osteotomy position is displayed in the 3D bone model.

8. A femoral head osteotomy surgery planning device based on multimodal image registration, characterized in that: include: An image acquisition module, which is used to acquire a multimodal medical image of any object, wherein the multimodal medical image at least includes an X-ray image, a CT image, and an MRI image; An image fusion module, which is used to fuse the X-ray image, the CT image and the MRI image to obtain a fused image; The osteotomy planning module is used to plan the femoral neck base rotation osteotomy based on the fused image.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program to: Acquire a multimodal medical image of any object, wherein the multimodal medical image includes at least an X-ray image, a CT image, and an MRI image; fusing the X-ray image, the CT image and the MRI image to obtain a fused image; Based on the fused images, the femoral neck base rotation osteotomy was planned.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the femoral head osteotomy surgery planning method based on multimodal image registration as described in any one of claims 1-6.

Images