Method for diagnosing failure mode after knee joint replacement based on multi-modal data
Through multimodal data fusion and deep learning technology, abnormal features in knee CT and X-ray images were extracted, and the problems of insufficient sensitivity and inefficiency of failure diagnosis after knee arthroplasty were solved, achieving accurate, rapid and non-invasive diagnostic effects.
Patent Information
- Application Number
- CN202411960795.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-06-06
AI Technical Summary
The prior art has problems such as insufficient sensitivity, inefficiency, and insufficient multi-factor comprehensive judgment ability in the diagnosis of failure after knee arthroplasty, making it difficult to accurately, quickly and non-invasively identify the early characteristics and patterns of failure after surgery.
Using a diagnostic method based on multimodal data, the knee joint CT images and X-ray images are acquired and preprocessed, abnormal features are extracted using convolutional neural network (CNN), and multimodal feature fusion is achieved through attention mechanism to output the diagnostic results of postoperative failure mode.
Accurate, rapid and non-invasive diagnosis of failure mode after knee arthroplasty surgery is achieved, early recognition ability is improved, metal artifact interference is overcome, the subjectivity of the diagnosis is reduced, and the level of diagnostic standardization is improved.
Smart Images
Figure CN120108684A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of diagnosis of postoperative failure modes, and in particular, relates to a diagnosis method, device, equipment and computer-readable storage medium for postoperative failure modes of knee replacement based on multimodal data. Background Art
[0002] TKA is an effective method for treating severe knee diseases (such as osteoarthritis, rheumatoid arthritis, etc.). However, postoperative failure is not uncommon, and the main reasons include infection, aseptic loosening, osteolysis, and poor prosthesis alignment. These problems can significantly affect the patient's quality of life and increase the risk of reoperation (revision) and medical costs. At present, clinical experience and X-rays are mainly used to determine whether knee replacement surgery has failed.
[0003] Currently, the diagnosis of knee replacement failure in clinical practice relies on the following methods:
[0004] 1. Clinical symptom assessment: Symptoms are highly subjective and cannot clearly point to a specific cause of failure.
[0005] 2. Imaging examination:
[0006] (1) X-ray: Early loosening and osteolysis characteristics are difficult to detect.
[0007] (2) CT scan: Metal artifacts interfere significantly and the ability to assess soft tissue is limited.
[0008] (3) MRI and radionuclide scanning: high cost and lack of specificity limit their widespread application.
[0009] 3. Laboratory examination:
[0010] Nonspecific inflammatory markers (CRP, ESR) have limited sensitivity and specificity for infectious failure.
[0011] 4. Pathological examination:
[0012] Invasive procedures are required, increasing patient pain and infection risks.
[0013] Comprehensive problem: Existing methods have problems such as insufficient diagnostic sensitivity, low efficiency, and insufficient ability to make comprehensive judgments based on multiple factors. Therefore, there is an urgent need for an accurate, rapid, and non-invasive diagnostic method to identify the early characteristics and patterns of postoperative failure. Summary of the invention
[0014] The embodiments of the present application provide a method, apparatus, device and computer-readable storage medium for diagnosing failure modes after knee replacement surgery based on multimodal data, which can accurately, quickly and non-invasively diagnose failure modes after knee replacement surgery.
[0015] In a first aspect, an embodiment of the present application provides a method for diagnosing a failure mode after knee replacement surgery based on multimodal data, comprising:
[0016] Obtain knee CT images and knee X-ray images of patients after knee replacement surgery;
[0017] Perform image preprocessing on knee joint CT images and knee joint X-ray images;
[0018] Input the knee joint CT images and knee joint X-ray images after image preprocessing into a preset postoperative failure mode diagnosis model, and output the postoperative failure mode diagnosis results;
[0019] Among them, the convolutional neural network CNN in the postoperative failure mode diagnosis model extracts abnormal features from knee CT images and knee X-ray images respectively, realizes multimodal feature fusion through the attention mechanism, and highlights the key area features.
[0020] Optional, postoperative failure mode diagnostic results include:
[0021] Infection, aseptic loosening, osteolysis, prosthesis malalignment, periprosthetic fracture, prosthesis breakage, liner wear, instability, others.
[0022] Optionally, perform image preprocessing on the knee CT images, including:
[0023] Artifact suppression algorithm is used to reduce the interference of metal artifacts on knee joint CT images;
[0024] Three-dimensional convolutional neural network (3D-CNN) was used to extract features of osteolysis and prosthesis loosening.
[0025] Optionally, perform image preprocessing on the knee X-ray images, including:
[0026] The alignment of the prosthesis and the changes in the surrounding bone are detected using a two-dimensional convolutional neural network (2D-CNN).
[0027] Optionally, after outputting the postoperative failure mode diagnosis result, it also includes:
[0028] Generate a quantitative score for the risk of failure of postoperative failure patterns;
[0029] Grad-CAM technology is used to generate intuitive visual diagnostic prompts to highlight abnormal areas in the image.
[0030] Optionally, also include:
[0031] Integrate with hospital imaging system PACS and electronic medical record system EMR to provide automatic diagnosis and risk assessment reports.
[0032] Optionally, the postoperative failure mode diagnosis model, before model training, also includes:
[0033] Postoperative failure cases from multiple hospitals were collected, including knee CT images and knee X-ray images;
[0034] Mark key image areas and categorize cases.
[0035] In a second aspect, an embodiment of the present application provides a device for diagnosing failure modes after knee replacement surgery based on multimodal data, comprising:
[0036] An image acquisition module is used to obtain CT images and X-ray images of the knee joints of patients after knee replacement surgery;
[0037] An image preprocessing module, used for performing image preprocessing on knee joint CT images and knee joint X-ray images;
[0038] A failure mode diagnosis module is used to input the pre-processed knee joint CT images and knee joint X-ray images into a preset postoperative failure mode diagnosis model and output a postoperative failure mode diagnosis result;
[0039] Among them, the convolutional neural network CNN in the postoperative failure mode diagnosis model extracts abnormal features from knee CT images and knee X-ray images respectively, realizes multimodal feature fusion through the attention mechanism, and highlights the key area features.
[0040] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;
[0041] When the processor executes the computer program instructions, a method for diagnosing failure modes after knee replacement surgery based on multimodal data is implemented.
[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implements a method for diagnosing a failure mode after knee replacement surgery based on multimodal data.
[0043] The diagnosis method, device, equipment and computer-readable storage medium for failure modes after knee replacement surgery based on multimodal data in the embodiments of the present application can accurately, quickly and non-invasively diagnose failure modes after knee replacement surgery.
[0044] The diagnosis method of failure mode after knee replacement surgery based on multimodal data includes:
[0045] Obtain knee CT images and knee X-ray images of patients after knee replacement surgery;
[0046] Perform image preprocessing on knee joint CT images and knee joint X-ray images;
[0047] Input the knee joint CT images and knee joint X-ray images after image preprocessing into a preset postoperative failure mode diagnosis model, and output the postoperative failure mode diagnosis results;
[0048] Among them, the convolutional neural network CNN in the postoperative failure mode diagnosis model extracts abnormal features from knee CT images and knee X-ray images respectively, realizes multimodal feature fusion through the attention mechanism, and highlights the key area features. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0050] Figure 1 It is a flowchart of a method for diagnosing failure modes after knee replacement surgery based on multimodal data provided by an embodiment of the present application;
[0051] Figure 2 It is a schematic structural diagram of a device for diagnosing failure modes after knee replacement surgery based on multimodal data provided by an embodiment of the present application;
[0052] Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0053] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0054] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0055] In order to solve the problems of the prior art, the embodiments of the present application provide a method, device, equipment and computer-readable storage medium for diagnosing failure modes after knee replacement surgery based on multimodal data. The following first introduces the method for diagnosing failure modes after knee replacement surgery based on multimodal data provided by the embodiments of the present application.
[0056] Figure 1 FIG. 1 is a flow chart of a method for diagnosing a failure mode after knee replacement surgery based on multimodal data provided by an embodiment of the present application. Figure 1 As shown, the diagnosis method of failure mode after knee replacement surgery based on multimodal data includes:
[0057] S101. Obtain CT images and X-ray images of the knee joint of the patient after knee replacement surgery;
[0058] S102, performing image preprocessing on knee joint CT images and knee joint X-ray images;
[0059] S103, inputting the knee joint CT image and knee joint X-ray image after image preprocessing into a preset postoperative failure mode diagnosis model, and outputting the postoperative failure mode diagnosis result;
[0060] Among them, the convolutional neural network CNN in the postoperative failure mode diagnosis model extracts abnormal features from knee CT images and knee X-ray images respectively, realizes multimodal feature fusion through the attention mechanism, and highlights the key area features.
[0061] This application can improve the ability to identify failure modes after knee replacement surgery at an early stage, especially when the imaging features are mild and atypical; it can overcome the interference of metal artifacts on image analysis and achieve efficient integration of CT and X-ray data; it can reduce the subjectivity of diagnosis, improve the level of diagnostic standardization, and provide auxiliary support for clinical decision-making.
[0062] In one embodiment, the post-operative failure mode diagnosis results include:
[0063] Infection, aseptic loosening, osteolysis, prosthesis malalignment, periprosthetic fracture, prosthesis breakage, liner wear, instability, others.
[0064] In one embodiment, image preprocessing is performed on a knee joint CT image, including:
[0065] Artifact suppression algorithm is used to reduce the interference of metal artifacts on knee joint CT images;
[0066] Three-dimensional convolutional neural network (3D-CNN) was used to extract features of osteolysis and prosthesis loosening.
[0067] In this embodiment, artifact suppression processing is performed on the CT image, and a three-dimensional region of interest (ROI) around the prosthesis is extracted.
[0068] In one embodiment, performing image preprocessing on a knee joint X-ray image includes:
[0069] The alignment of the prosthesis and the changes in the surrounding bone are detected using a two-dimensional convolutional neural network (2D-CNN).
[0070] In this embodiment, bone density and alignment feature extraction are performed on the X-ray image.
[0071] Deep learning feature extraction module:
[0072] Abnormal features in CT and X-ray images were extracted based on convolutional neural network (CNN).
[0073] Multimodal feature fusion is achieved through the attention mechanism to highlight the features of key areas.
[0074] In the process of feature fusion and pattern recognition, a fusion model is constructed, and the attention mechanism is used to integrate CT and X-ray features to focus on the key areas of the failure mode.
[0075] In one embodiment, after outputting the postoperative failure mode diagnosis result, the method further includes:
[0076] Generate a quantitative score for the risk of failure of postoperative failure patterns;
[0077] Grad-CAM technology is used to generate intuitive visual diagnostic prompts to highlight abnormal areas in the image.
[0078] In this embodiment, the classification and risk assessment module:
[0079] Combining multimodal imaging features, the model classifies the failure mode (infection, loosening, osteolysis, malalignment).
[0080] Output the quantitative score of failure risk and generate intuitive visual diagnostic prompt map (such as heat map).
[0081] During the diagnostic classification and result visualization process, based on the fusion features, the model automatically determines the failure mode (such as infection, loosening, etc.) and generates a risk score.
[0082] Grad-CAM technology is used to generate image prompt maps to highlight abnormal areas in the image.
[0083] In one embodiment, it further includes:
[0084] Integrate with hospital imaging system PACS and electronic medical record system EMR to provide automatic diagnosis and risk assessment reports.
[0085] In this embodiment, the system integration module:
[0086] Integrate with hospital imaging systems (PACS) and electronic medical record systems (EMR) to provide automatic diagnosis and risk assessment reports.
[0087] During the system integration and feedback optimization process, the system is embedded in the hospital PACS system to realize automated analysis and push of diagnosis results. Doctors can verify the AI results, form a feedback mechanism, and continuously optimize the model performance.
[0088] In one embodiment, the postoperative failure mode diagnosis model further includes, before model training:
[0089] Postoperative failure cases from multiple hospitals were collected, including knee CT images and knee X-ray images;
[0090] Mark key image areas and categorize cases.
[0091] In this embodiment, data collection and annotation:
[0092] Collection scope: Postoperative failure cases from multiple hospitals were collected, including CT and X-ray imaging data.
[0093] Annotation details: Experts mark key imaging areas (such as periprosthetic bone tissue and areas of abnormal alignment angles) and classify cases.
[0094] In summary, this application can be applied to the following scenarios:
[0095] Follow-up after knee replacement surgery and early complication screening.
[0096] Hospital imaging center, improving the efficiency of diagnosis of failed knee surgery.
[0097] The beneficial effects of this application are as follows:
[0098] (1) Improved diagnostic accuracy:
[0099] The deep learning model automatically identified the imaging features of early failure with better sensitivity and specificity than traditional methods.
[0100] (2) Improved diagnostic efficiency:
[0101] Automated analysis of imaging data can significantly reduce doctors’ imaging evaluation time, improve early diagnosis capabilities, and reduce missed diagnosis rates.
[0102] (3) Combination of standardization and personalization:
[0103] The diagnostic process is standardized through AI technology, and a personalized failure risk assessment report is generated for each patient.
[0104] (4) Reduce invasiveness:
[0105] Non-invasive imaging analysis replaces some laboratory tests and pathological biopsies.
[0106] Figure 2 1 is a schematic diagram of a structure of a device for diagnosing a failure mode after knee replacement surgery based on multimodal data provided by an embodiment of the present application. The device for diagnosing a failure mode after knee replacement surgery based on multimodal data comprises:
[0107] An image acquisition module 201 is used to acquire a CT image and an X-ray image of a knee joint after a knee replacement operation of a patient;
[0108] An image preprocessing module 202 is used to perform image preprocessing on knee joint CT images and knee joint X-ray images;
[0109] The failure mode diagnosis module 203 is used to input the knee joint CT image and the knee joint X-ray image after image preprocessing into a preset postoperative failure mode diagnosis model, and output the postoperative failure mode diagnosis result;
[0110] Among them, the convolutional neural network CNN in the postoperative failure mode diagnosis model extracts abnormal features from knee CT images and knee X-ray images respectively, realizes multimodal feature fusion through the attention mechanism, and highlights the key area features.
[0111] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.
[0112] The electronic device may include a processor 301 and a memory 302 storing computer program instructions.
[0113] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0114] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 302 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 302 may be inside or outside the electronic device. In a particular embodiment, the memory 302 may be a non-volatile solid-state memory.
[0115] In one embodiment, the memory 302 may be a read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.
[0116] The processor 301 reads and executes the computer program instructions stored in the memory 302 to implement any one of the diagnosis methods for failure modes after knee replacement surgery based on multimodal data in the above embodiments.
[0117] In one example, the electronic device may further include a communication interface 303 and a bus 310. Figure 3 As shown, the processor 301, the memory 302, and the communication interface 303 are connected via a bus 310 and communicate with each other.
[0118] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0119] Bus 310 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 310 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.
[0120] In addition, in combination with the diagnosis method of failure mode after knee replacement based on multimodal data in the above embodiments, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the diagnosis methods of failure mode after knee replacement based on multimodal data in the above embodiments is implemented.
[0121] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0122] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0123] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0124] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.
[0125] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A method for diagnosing failure modes after knee replacement surgery based on multimodal data, characterized in that: include: Obtain CT images and X-ray images of the knee joints of patients after knee replacement surgery; Perform image preprocessing on knee joint CT images and knee joint X-ray images; Input the knee joint CT images and knee joint X-ray images after image preprocessing into a preset postoperative failure mode diagnosis model, and output the postoperative failure mode diagnosis results; Among them, the convolutional neural network CNN in the postoperative failure mode diagnosis model extracts abnormal features from knee CT images and knee X-ray images respectively, realizes multimodal feature fusion through the attention mechanism, and highlights the key area features.
2. The method for diagnosing failure modes after knee replacement surgery based on multimodal data according to claim 1, characterized in that: Postoperative failure mode diagnostic findings include: Infection, aseptic loosening, osteolysis, prosthesis malalignment, periprosthetic fracture, prosthesis breakage, liner wear, instability, others.
3. The method for diagnosing failure modes after knee replacement surgery based on multimodal data according to claim 2, characterized in that: Perform image preprocessing on knee CT images, including: Artifact suppression algorithm is used to reduce the interference of metal artifacts on knee joint CT images; Three-dimensional convolutional neural network (3D-CNN) was used to extract features of osteolysis and prosthesis loosening.
4. The method for diagnosing failure modes after knee replacement surgery based on multimodal data according to claim 3, characterized in that: Perform image preprocessing on knee joint X-ray images, including: The alignment of the prosthesis and the changes in the surrounding bone are detected using a two-dimensional convolutional neural network (2D-CNN).
5. The method for diagnosing failure modes after knee replacement surgery based on multimodal data according to claim 1, characterized in that: After outputting the postoperative failure mode diagnosis results, it also includes: Generate a quantitative score for the risk of failure of postoperative failure patterns; Grad-CAM technology is used to generate intuitive visual diagnostic prompts to highlight abnormal areas in the image.
6. The method for diagnosing failure modes after knee replacement surgery based on multimodal data according to claim 5, characterized in that: Also includes: Integrate with hospital imaging system PACS and electronic medical record system EMR to provide automatic diagnosis and risk assessment reports.
7. The method for diagnosing failure modes after knee replacement surgery based on multimodal data according to claim 1, characterized in that: Before model training, the postoperative failure mode diagnosis model also includes: Postoperative failure cases from multiple hospitals were collected, including knee CT images and knee X-ray images; Mark key image areas and categorize cases.
8. A diagnostic device for failure modes after knee replacement surgery based on multimodal data, characterized in that: The device comprises: An image acquisition module is used to obtain CT images and X-ray images of the knee joints of patients after knee replacement surgery; An image preprocessing module, used for performing image preprocessing on knee joint CT images and knee joint X-ray images; A failure mode diagnosis module is used to input the pre-processed knee joint CT images and knee joint X-ray images into a preset postoperative failure mode diagnosis model and output a postoperative failure mode diagnosis result; Among them, the convolutional neural network CNN in the postoperative failure mode diagnosis model extracts abnormal features from knee CT images and knee X-ray images respectively, realizes multimodal feature fusion through the attention mechanism, and highlights the key area features.
9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the method for diagnosing failure modes after knee replacement surgery based on multimodal data as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for diagnosing failure modes after knee replacement surgery based on multimodal data as described in any one of claims 1 to 7.