Planning method for femoral side footprint area of inner patellofemoral ligament complex based on deep learning
Through deep learning-based methods, the knee joint CT images are segmented and thermogram recognition are performed to accurately determine the femoral lateral footprint area, solving the problem of inaccurate planning in the prior art and improving the accuracy and effectiveness of the surgery.
Patent Information
- Application Number
- CN202510124832.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-26
- Publication Date
- 2025-05-13
AI Technical Summary
The prior art is difficult to quickly and accurately perform preoperative planning of the femoral lateral footprint area, which affects the reconstruction surgical effect of the medial patellofemoral ligament complex.
Using a deep learning-based method, by acquiring knee joint CT images, using a segmentation model for coarse segmentation and sperm segmentation, combining thermograms to identify the adductal muscle node node and the medial epicondyle point, and determining the femoral lateral footprint area.
The preoperative planning of the femoral lateral footprint area is achieved quickly and accurately, and the accuracy and effectiveness of the medial patellofemoral ligament complex reconstruction surgery is improved.
Smart Images
Figure CN119991631A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of planning of the femoral footprint area, and in particular, relates to a planning method, system, device and computer-readable storage medium for the femoral footprint area of the medial patellofemoral ligament complex based on deep learning. Background Art
[0002] Patellar dislocation is one of the important diseases that seriously affects the human body's motor function. According to the dislocation mode, it is divided into recurrent patellar dislocation and habitual patellar dislocation.
[0003] For patients with mild symptoms, medial patellofemoral ligament reconstruction (MPFLR) is an important surgical method to stabilize the patella and treat dislocation.
[0004] The range of the medial patellar support structure is actually wider than the range of the patellar ligament. The patellar side involves the medial fibers of the quadriceps tendon. This structure, combined with the medial patellofemoral ligament, forms fan-shaped fibers that end in a linear footprint area including the femoral insertion point of the medial patellofemoral ligament. This stable structure is called the "medial patellofemoral ligament complex" (MPFC). Only when the medial patellofemoral ligament complex structure is completely reconstructed surgically can the comprehensive anatomical reconstruction of the medial patellar stable structure be completed.
[0005] Whether the location and range of the femoral footprint area can be accurately marked by imaging methods before surgery is an important prerequisite for completing medial patellofemoral ligament complex reconstruction surgery.
[0006] At present, the related technology is to manually determine the footprint area and then perform morphological analysis on the footprint area.
[0007] Therefore, how to quickly and accurately perform preoperative planning of the femoral footprint area is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention
[0008] The embodiments of the present application provide a deep learning-based planning method, system, device, and computer-readable storage medium for the femoral footprint area of the medial patellofemoral ligament complex, which can quickly and accurately perform preoperative planning of the femoral footprint area.
[0009] In a first aspect, an embodiment of the present application provides a method for planning a femoral footprint area of a medial patellofemoral ligament complex based on deep learning, comprising:
[0010] Obtain CT images of the knee joint;
[0011] The segmentation model is used to perform rough segmentation and fine segmentation on the knee joint CT image to obtain the segmentation results;
[0012] Based on the segmentation results, the adductor node and medial epicondyle point were identified using heat maps;
[0013] The femoral footprint area was determined based on the adductor tuberosity and medial epicondyle points.
[0014] Optionally, the segmentation model is used to roughly segment the knee joint CT image to obtain a segmentation result, including:
[0015] Using the UNet network structure, the tibia-fibula part is roughly segmented to obtain the first rough segmentation result;
[0016] The femur-patella part is roughly segmented using the DenseBlock network structure to obtain the second rough segmentation result.
[0017] Optionally, the segmentation model is used to perform fine segmentation on the knee joint CT image to obtain a segmentation result, including:
[0018] Using the PointRend network structure, the edge area pixels in the first rough segmentation result are corrected and edge calibrated to obtain the refined result of the tibia-fibula.
[0019] The second rough segmentation result and the refined result of tibia-fibula are spliced and fused to obtain the rough segmentation result of femur-patella;
[0020] The LSTM network structure is used to correct the edge contour points of the rough segmentation result of the femur-patella, and finally the accurate segmentation result of the femur-patella is obtained.
[0021] Optionally, based on the segmentation results, a heat map is used to identify the adductor node and the medial epicondyle point, including:
[0022] Using the Unet network structure and heatmap point recognition technology, the adductor node and medial epicondyle point are identified;
[0023] Among them, the Unet network structure includes three parts: encoding, decoding and skip connection;
[0024] The encoding part gradually extracts low-resolution high-level features through a series of convolution and pooling operations;
[0025] The decoding part gradually restores the spatial resolution through deconvolution;
[0026] The jump connection part is to concatenate the features in the encoder with the features in the decoder to enhance the feature expression capability of the decoder.
[0027] Optional, heatmap point recognition technology, including:
[0028] Read input images or data and generate heat maps;
[0029] Find local maxima in the heat map;
[0030] Find the center of the hotspot and get a more precise location by calculating the centroid or Gaussian weighting;
[0031] Convert the hotspot positions on the heat map back to coordinates in the original image or real space.
[0032] Optionally, the femoral footprint is determined based on the adductor tuberosity and medial epicondyle points and includes:
[0033] Based on the adductor node, the proximal and distal points of the footprint area of the medial patellofemoral ligament complex were determined respectively;
[0034] The femoral footprint area is determined by the angle between the line connecting the proximal point of the footprint area and the distal point of the footprint area and the sagittal plane.
[0035] Optionally, based on the adductor tubercle, the proximal and distal points of the footprint of the medial patellofemoral ligament complex are determined, respectively, including:
[0036] 4.9 mm anterior to the adductor tubercle and 12.7 mm distal to it, mark the distal point of the footprint of the medial patellofemoral ligament complex;
[0037] From the distal point of the footprint area, 10.9 mm proximal and 2.6 mm anterior, mark the proximal point of the footprint area of the medial patellofemoral ligament complex.
[0038] In a second aspect, an embodiment of the present application provides a planning system for the femoral footprint area of the medial patellofemoral ligament complex based on deep learning, comprising:
[0039] An image acquisition module, used for acquiring a knee joint CT image;
[0040] An image segmentation module is used to perform rough segmentation and fine segmentation on the knee joint CT image using a segmentation model to obtain a segmentation result;
[0041] A point recognition module is used to identify the adductor node and the medial epicondyle point based on the segmentation results using a heat map;
[0042] The footprint area determination module is used to determine the femoral side footprint area based on the adductor tuberosity point and the medial epicondyle point.
[0043] In a third aspect, an embodiment of the present application provides an electronic device, characterized in that the electronic device includes: a processor and a memory storing computer program instructions;
[0044] When the processor executes the computer program instructions, it implements a deep learning-based planning method for the femoral footprint area of the medial patellofemoral ligament complex.
[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implements a deep learning-based planning method for the femoral footprint area of the medial patellofemoral ligament complex.
[0046] The deep learning-based planning method, system, device and computer-readable storage medium for the femoral footprint area of the medial patellofemoral ligament complex in the embodiments of the present application can quickly and accurately perform preoperative planning of the femoral footprint area.
[0047] The deep learning-based planning method for the femoral footprint of the medial patellofemoral ligament complex includes:
[0048] Obtain CT images of the knee joint;
[0049] The segmentation model is used to perform rough segmentation and fine segmentation on the knee joint CT image to obtain the segmentation results;
[0050] Based on the segmentation results, the adductor node and medial epicondyle point were identified using heat maps;
[0051] The femoral footprint area was determined based on the adductor tuberosity and medial epicondyle points. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0053] Figure 1 It is a flowchart of a method for planning the femoral footprint area of the medial patellofemoral ligament complex based on deep learning provided by an embodiment of the present application;
[0054] Figure 2 This is a schematic diagram of planning results of the femoral footprint area of the medial patellofemoral ligament complex based on deep learning provided by an embodiment of the present application;
[0055] Figure 3 is a schematic diagram of a rough segmentation process provided by an embodiment of the present application;
[0056] Figure 4 It is a schematic diagram of a process of fine segmentation provided by an embodiment of the present application;
[0057] Figure 5 It is a structural schematic diagram of a planning system for the femoral footprint area of the medial patellofemoral ligament complex based on deep learning provided by an embodiment of the present application;
[0058] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0059] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0060] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0061] In order to solve the problems of the prior art, the embodiments of the present application provide a method, system, device and computer-readable storage medium for planning the femoral footprint area of the medial patellofemoral ligament complex based on deep learning. The following first introduces the planning method for the femoral footprint area of the medial patellofemoral ligament complex based on deep learning provided in the embodiments of the present application.
[0062] Figure 1 FIG. 1 is a flow chart of a method for planning the femoral footprint of the medial patellofemoral ligament complex based on deep learning according to an embodiment of the present application. Figure 1 As shown, the deep learning-based planning method for the femoral footprint area of the medial patellofemoral ligament complex includes:
[0063] S101, obtaining a CT image of a knee joint;
[0064] S102, using the segmentation model to perform coarse segmentation and fine segmentation on the knee joint CT image to obtain a segmentation result;
[0065] S103, based on the segmentation result, using a heat map to identify the adductor node and the medial epicondyle point;
[0066] S104. Determine the femoral footprint area based on the adductor tuberosity point and the medial epicondyle point.
[0067] Figure 2 This is a schematic diagram of the planning results of the femoral footprint area of the medial patellofemoral ligament complex based on deep learning provided by an embodiment of the present application.
[0068] Figure 3 is a schematic diagram of a rough segmentation process provided by an embodiment of the present application;
[0069] In one embodiment, the segmentation model is used to roughly segment the knee joint CT image to obtain a segmentation result, including:
[0070] Using the UNet network structure, the tibia-fibula part is roughly segmented to obtain the first rough segmentation result;
[0071] The femur-patella part is roughly segmented using the DenseBlock network structure to obtain the second rough segmentation result.
[0072] Figure 4 It is a schematic diagram of a process of fine segmentation provided by an embodiment of the present application;
[0073] In one embodiment, the segmentation model is used to perform fine segmentation on the knee joint CT image to obtain a segmentation result, including:
[0074] Using the PointRend network structure, the edge area pixels in the first rough segmentation result are corrected and edge calibrated to obtain the refined result of the tibia-fibula.
[0075] The second rough segmentation result and the refined result of tibia-fibula are spliced and fused to obtain the rough segmentation result of femur-patella;
[0076] The LSTM network structure is used to correct the edge contour points of the rough segmentation result of the femur-patella, and finally the accurate segmentation result of the femur-patella is obtained.
[0077] In one embodiment, based on the segmentation results, the adductor node and the medial epicondyle point are identified using a heat map, including:
[0078] Using the Unet network structure and heatmap point recognition technology, the adductor node and medial epicondyle point are identified;
[0079] Among them, the Unet network structure includes three parts: encoding, decoding and skip connection;
[0080] The encoding part gradually extracts low-resolution high-level features through a series of convolution and pooling operations;
[0081] The decoding part gradually restores the spatial resolution through deconvolution;
[0082] The jump connection part is to concatenate the features in the encoder with the features in the decoder to enhance the feature expression capability of the decoder.
[0083] In one embodiment, the heatmap point recognition technology includes:
[0084] Read input images or data and generate heat maps;
[0085] Find local maxima in the heat map;
[0086] Find the center of the hotspot and get a more precise location by calculating the centroid or Gaussian weighting;
[0087] Convert the hotspot positions on the heat map back to coordinates in the original image or real space.
[0088] In one embodiment, the femoral footprint area is determined based on the adductor tuberosity point and the medial epicondyle point, including:
[0089] Based on the adductor node, the proximal and distal points of the footprint area of the medial patellofemoral ligament complex were determined respectively;
[0090] The femoral footprint area is determined by the angle between the line connecting the proximal point of the footprint area and the distal point of the footprint area and the sagittal plane.
[0091] In one embodiment, based on the adductor node, respectively determining the proximal end point and the distal end point of the footprint area of the medial patellofemoral ligament complex comprises:
[0092] 4.9 mm anterior to the adductor tubercle and 12.7 mm distal to it, mark the distal point of the footprint of the medial patellofemoral ligament complex;
[0093] From the distal point of the footprint area, 10.9 mm proximal and 2.6 mm anterior, mark the proximal point of the footprint area of the medial patellofemoral ligament complex.
[0094] Figure 5 It is a structural schematic diagram of a planning system for the femoral footprint area of the medial patellofemoral ligament complex based on deep learning provided by an embodiment of the present application;
[0095] The deep learning-based planning system for the femoral footprint of the medial patellofemoral ligament complex includes:
[0096] An image acquisition module 501 is used to acquire a CT image of a knee joint;
[0097] An image segmentation module 502 is used to perform rough segmentation and fine segmentation on the knee joint CT image using a segmentation model to obtain a segmentation result;
[0098] A point identification module 503 is used to identify the adductor node and the medial epicondyle point using a heat map based on the segmentation result;
[0099] The footprint area determination module 504 is used to determine the femoral side footprint area based on the adductor tuberosity point and the medial epicondyle point.
[0100] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.
[0101] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.
[0102] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0103] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the electronic device. In a particular embodiment, the memory 602 may be a non-volatile solid-state memory.
[0104] In one embodiment, the memory 602 may be a read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.
[0105] The processor 601 reads and executes the computer program instructions stored in the memory 602 to implement any one of the deep learning-based planning methods for the femoral footprint area of the medial patellofemoral ligament complex in the above-mentioned embodiments.
[0106] In one example, the electronic device may further include a communication interface 603 and a bus 610. Figure 6 As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 610 and communicate with each other.
[0107] The communication interface 603 is mainly used to implement communication between various modules, systems, units and / or devices in the embodiments of the present application.
[0108] Bus 610 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 610 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.
[0109] In addition, in combination with the deep learning-based planning method for the femoral footprint area of the medial patellofemoral ligament complex in the above embodiments, the present application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the deep learning-based planning methods for the femoral footprint area of the medial patellofemoral ligament complex in the above embodiments is implemented.
[0110] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0111] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0112] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or systems. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0113] The above reference is according to the method of the embodiment of the present application, the flow chart of system (system) and computer program product and / or block diagram and describe various aspects of the present application.It should be understood that each square frame in the flow chart and / or block diagram and the combination of each square frame in the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer, special-purpose computer or other programmable data processing system, to produce a kind of machine, so that these instructions executed by the processor of computer or other programmable data processing system enable the realization of the function / action specified in one or more square frames of flow chart and / or block diagram.Such processor can be but not limited to general-purpose processor, special-purpose processor, special application processor or field programmable logic circuit.It can also be understood that each square frame in the block chart and / or flow chart and the combination of square frames in the block chart and / or flow chart can also be realized by the special-purpose hardware that performs the specified function or action, or can be realized by the combination of special-purpose hardware and computer instructions.
[0114] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A deep learning-based planning method for the femoral footprint of the medial patellofemoral ligament complex, characterized in that: include: Obtain CT images of the knee joint; The segmentation model is used to perform rough segmentation and fine segmentation on the knee joint CT image to obtain the segmentation results; Based on the segmentation results, the adductor node and medial epicondyle point were identified using heat maps; The femoral footprint area was determined based on the adductor tuberosity and medial epicondyle points.
2. The method for planning the femoral footprint of the medial patellofemoral ligament complex based on deep learning according to claim 1, characterized in that: The segmentation model is used to roughly segment the knee joint CT image to obtain the segmentation results, including: Using the UNet network structure, the tibia-fibula part is roughly segmented to obtain the first rough segmentation result; The femur-patella part is roughly segmented using the DenseBlock network structure to obtain the second rough segmentation result.
3. The method for planning the femoral footprint of the medial patellofemoral ligament complex based on deep learning according to claim 2, characterized in that: The segmentation model is used to perform precise segmentation on the knee joint CT image to obtain the segmentation results, including: Using the PointRend network structure, the edge area pixels in the first rough segmentation result are corrected and edge calibrated to obtain the refined result of the tibia-fibula. The second rough segmentation result and the refined result of tibia-fibula are spliced and fused to obtain the rough segmentation result of femur-patella; The LSTM network structure is used to correct the edge contour points of the rough segmentation result of the femur-patella, and finally the accurate segmentation result of the femur-patella is obtained.
4. The method for planning the femoral footprint of the medial patellofemoral ligament complex based on deep learning according to claim 3, characterized in that: Based on the segmentation results, the adductor node and medial epicondyle point are identified using heat maps, including: Using the Unet network structure and heatmap point recognition technology, the adductor node and medial epicondyle point are identified; Among them, the Unet network structure includes three parts: encoding, decoding and skip connection; The encoding part gradually extracts low-resolution high-level features through a series of convolution and pooling operations; The decoding part gradually restores the spatial resolution through deconvolution; The jump connection part is to concatenate the features in the encoder with the features in the decoder to enhance the feature expression capability of the decoder.
5. The method for planning the femoral footprint of the medial patellofemoral ligament complex based on deep learning according to claim 4, characterized in that: Heatmap thermal map point recognition technology, including: Read input images or data and generate heat maps; Find local maxima in the heat map; Find the center of the hotspot and get a more precise location by calculating the centroid or Gaussian weighting; Convert the hotspot positions on the heat map back to coordinates in the original image or real space.
6. The method for planning the femoral footprint of the medial patellofemoral ligament complex based on deep learning according to claim 1, characterized in that: Based on the adductor tuberosity and medial epicondyle point, the femoral footprint area is determined, including: Based on the adductor node, the proximal and distal points of the footprint area of the medial patellofemoral ligament complex were determined respectively; The femoral footprint area is determined by the angle between the line connecting the proximal point of the footprint area and the distal point of the footprint area and the sagittal plane.
7. The method for planning the femoral footprint of the medial patellofemoral ligament complex based on deep learning according to claim 6, characterized in that: Based on the adductor node, the proximal and distal points of the footprint area of the medial patellofemoral ligament complex were determined, including: 4.9 mm anterior to the adductor tubercle and 12.7 mm distal to it, mark the distal point of the footprint of the medial patellofemoral ligament complex; From the distal point of the footprint area, 10.9 mm proximal and 2.6 mm anterior, mark the proximal point of the footprint area of the medial patellofemoral ligament complex.
8. A deep learning-based planning system for the femoral footprint of the medial patellofemoral ligament complex, characterized in that: The system comprises: An image acquisition module, used for acquiring a knee joint CT image; An image segmentation module is used to perform rough segmentation and fine segmentation on the knee joint CT image using a segmentation model to obtain a segmentation result; A point recognition module is used to identify the adductor node and the medial epicondyle point based on the segmentation results using a heat map; The footprint area determination module is used to determine the femoral side footprint area based on the adductor tuberosity point and the medial epicondyle point.
9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the deep learning-based planning method for the femoral footprint area of the medial patellofemoral ligament complex as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement a deep learning-based planning method for the femoral footprint area of the medial patellofemoral ligament complex as described in any one of claims 1-7.