Joint replacement positioning and navigation method, system and equipment based on body intelligence
Through embodied intelligent technology, combined with the multimodal information fusion of knee joint images and RGB depth cameras, real-time osteotomy positioning navigation in joint replacement surgery is achieved, solving the problem of inaccurate positioning in traditional surgery and improving the accuracy of the surgery and flexibility.
Patent Information
- Application Number
- CN202411163248.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-12
AI Technical Summary
Traditional joint replacement surgery cannot be adjusted in real time according to actual conditions during the operation, resulting in insufficient accuracy in osteotomy positioning and navigation.
Using a method based on embodied intelligence, the patient's knee joint images are obtained for segmentation and three-dimensional reconstruction, combined with the RGB depth camera to obtain the surrounding environment images in real time, and the robotic arm is controlled for automatic osteotomy, including knee prosthesis. precise positioning and planning of osteotomy paths.
More accurate real-time location navigation for intraoperative osteotomy is achieved, improving the accuracy and flexibility of the surgery.
Smart Images

Figure CN118924426A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of joint replacement, and in particular, relates to a joint replacement positioning and navigation method, system, device and computer-readable storage medium based on embodied intelligence. Background Art
[0002] Joint replacement surgery is a common and complex surgical procedure used to treat joint-related diseases and injuries. Traditional joint replacement surgery involves controlling the robotic arm to perform bone cutting according to the preoperative plan after intraoperative digital twin registration.
[0003] However, it is not possible to make adjustments according to actual conditions during surgery.
[0004] Therefore, how to more accurately perform real-time positioning navigation for intraoperative osteotomy is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention
[0005] The embodiments of the present application provide a joint replacement positioning and navigation method, system, device and computer-readable storage medium based on embodied intelligence, which can more accurately perform real-time positioning and navigation of intraoperative osteotomy.
[0006] In a first aspect, an embodiment of the present application provides a joint replacement positioning and navigation method based on embodied intelligence, comprising:
[0007] Obtain images of the patient's knee joint;
[0008] Segment and 3D reconstruct the knee joint image to obtain a 3D model of the knee joint;
[0009] Perform preoperative planning based on the three-dimensional model of the knee joint to obtain preoperative planning results; the preoperative planning results include the position of the knee joint prosthesis, the prosthesis model and the planned osteotomy path;
[0010] During the operation, an RGB depth camera is used to obtain a real-time image of the surrounding environment, wherein the surrounding environment image includes visual markers;
[0011] Based on the visual marker, determining the coordinates of the position of the visual marker;
[0012] The intelligent agent will perform multimodal information fusion of the planned osteotomy path, surrounding environment images, visual marker position coordinates, and real-time acquired language instructions, and control the robotic arm to complete automatic osteotomy based on the fusion results.
[0013] Optionally, obtain images of the patient's knee, including:
[0014] Using high-resolution CT or MR scanning equipment, obtain the patient's initial knee CT or MR images;
[0015] The initial knee joint CT or MR image is preprocessed to obtain a final knee joint CT or MR image; wherein the preprocessing at least includes noise removal, image enhancement, edge detection and region segmentation.
[0016] Optionally, preoperative planning is performed based on the three-dimensional model of the knee joint to obtain preoperative planning results, including:
[0017] Based on the 3D model of the knee joint, select the appropriate model and specification of the knee prosthesis;
[0018] Based on the three-dimensional model of the knee joint, the prosthesis is accurately positioned, including the rotation angle, medial and lateral position, anterior and posterior position, and implantation depth of the prosthesis.
[0019] Plan the osteotomy path during surgery based on the design of the prosthesis and the patient's bone morphology.
[0020] Optionally, an RGB depth camera is used during the operation to obtain a surrounding environment image in real time; wherein the surrounding environment image includes visual markers, including:
[0021] Before the operation begins, fix the RGB depth camera in a suitable position so that it can cover the surgical area;
[0022] During the operation, the RGB depth camera collects the surrounding environment images in real time; the surrounding environment images include all visual information in the surgical area, especially surgical instruments, patient anatomical structures and surrounding visual markers.
[0023] Optionally, the intelligent agent performs multimodal information fusion of the planned osteotomy path, the surrounding environment image, the position coordinates of the visual markers, and the language instructions acquired in real time, including:
[0024] De-noise and enhance the surrounding environment image;
[0025] Parse and encode language instructions;
[0026] Extract multimodal features from the planned osteotomy path, denoised and enhanced surrounding environment images, visual marker position coordinates, and parsed and encoded language instructions;
[0027] Use deep learning models or multimodal information fusion algorithms to fuse multimodal features.
[0028] Optionally, a deep learning model or a multimodal information fusion algorithm is used to fuse multimodal features, including:
[0029] Determine the weights of multimodal features respectively;
[0030] The Transformer based on the attention mechanism fuses the multimodal features corresponding to each weight.
[0031] Optionally, the robot arm is controlled to complete automatic osteotomy according to the fusion result, including:
[0032] Obtaining a calibrated osteotomy tool coordinate system;
[0033] Based on the osteotomy tool coordinate system, the planned osteotomy path is mapped into an osteotomy path in the robot arm flange coordinate system;
[0034] Based on the kinematic constraint formula, the osteotomy path in the robot arm flange coordinate system is converted into the joint angle trajectory of each joint;
[0035] Interpolate the joint angle trajectory of each joint;
[0036] The feedforward PID controller outputs the joint angular velocity of each joint angle to control the robot arm;
[0037] Acquire the positioning data of the osteotomy tool positioning frame and the positioning data of the marking positioning frame of the bone to be osteotomized in real time;
[0038] The target joint angle trajectory is updated based on the positioning data of the osteotomy tool positioning frame and the positioning data of the marking positioning frame of the bone to be osteotomized, and the interpolation process of the joint angle trajectory of each joint is returned.
[0039] In a second aspect, the present application provides a joint replacement positioning navigation system based on embodied intelligence, including:
[0040] A joint image acquisition module, used for acquiring a patient's knee joint image;
[0041] A three-dimensional reconstruction module is used to segment and three-dimensionally reconstruct the knee joint image to obtain a three-dimensional model of the knee joint;
[0042] A preoperative planning module is used to perform preoperative planning based on a three-dimensional model of the knee joint and obtain preoperative planning results; wherein the preoperative planning results include the position of the knee joint prosthesis, the prosthesis model and the planned osteotomy path;
[0043] A depth image acquisition module is used to acquire a surrounding environment image in real time using an RGB depth camera during surgery; wherein the surrounding environment image includes visual markers;
[0044] A position coordinate determination module, used to determine the position coordinates of the visual marker based on the visual marker;
[0045] The information fusion module is used by the intelligent agent to perform multimodal information fusion of the planned osteotomy path, surrounding environment images, visual marker position coordinates, and real-time acquired language instructions, and control the robotic arm to complete automatic osteotomy based on the fusion results.
[0046] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;
[0047] When the processor executes the computer program instructions, a joint replacement positioning and navigation method based on embodied intelligence is implemented.
[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement a joint replacement positioning and navigation method based on embodied intelligence.
[0049] The joint replacement positioning and navigation method, system, device and computer-readable storage medium based on embodied intelligence in the embodiments of the present application can more accurately perform real-time positioning and navigation of intraoperative osteotomy.
[0050] The joint replacement positioning and navigation method based on embodied intelligence includes:
[0051] Obtain an image of the patient's knee joint;
[0052] Segment and 3D reconstruct the knee joint image to obtain a 3D model of the knee joint;
[0053] Perform preoperative planning based on the three-dimensional model of the knee joint to obtain preoperative planning results; the preoperative planning results include the position of the knee joint prosthesis, the prosthesis model and the planned osteotomy path;
[0054] During the operation, an RGB depth camera is used to obtain a real-time image of the surrounding environment, wherein the surrounding environment image includes visual markers;
[0055] Based on the visual marker, determining the coordinates of the position of the visual marker;
[0056] The intelligent agent will perform multimodal information fusion of the planned osteotomy path, surrounding environment images, visual marker position coordinates, and real-time acquired language instructions, and control the robotic arm to complete automatic osteotomy based on the fusion results. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0058] Figure 1It is a flowchart of a joint replacement positioning and navigation method based on embodied intelligence provided by an embodiment of the present application;
[0059] Figure 2 It is a flow chart of an intelligent agent outputting action decisions provided by an embodiment of the present application;
[0060] Figure 3 is a structural schematic diagram of a joint replacement positioning and navigation system based on embodied intelligence provided by an embodiment of the present application;
[0061] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0062] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0063] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0064] In order to solve the problems of the prior art, the embodiments of the present application provide a joint replacement positioning and navigation method, system, device and computer-readable storage medium based on embodied intelligence. The joint replacement positioning and navigation method based on embodied intelligence provided by the embodiments of the present application is first introduced below.
[0065] Figure 1 FIG. 1 is a flow chart of a joint replacement positioning and navigation method based on embodied intelligence provided by an embodiment of the present application. Figure 1 As shown, the joint replacement positioning and navigation method based on embodied intelligence includes:
[0066] S101, obtaining a knee joint image of a patient;
[0067] S102, segmenting and 3D reconstructing the knee joint image to obtain a 3D model of the knee joint;
[0068] S103, performing preoperative planning based on the three-dimensional model of the knee joint to obtain a preoperative planning result; wherein the preoperative planning result includes a knee joint prosthesis position, a prosthesis model, and a planned osteotomy path;
[0069] S104, during the operation, using an RGB depth camera to obtain a surrounding environment image in real time; wherein the surrounding environment image includes a visual marker;
[0070] S105, determining the position coordinates of the visual marker based on the visual marker;
[0071] S106. The intelligent agent performs multimodal information fusion of the planned osteotomy path, the surrounding environment image, the position coordinates of the visual markers, and the language instructions acquired in real time, and controls the robotic arm to complete automatic osteotomy according to the fusion result.
[0072] In one embodiment, obtaining a knee joint image of a patient includes:
[0073] Using high-resolution CT or MR scanning equipment, obtain the patient's initial knee CT or MR images;
[0074] The initial knee joint CT or MR image is preprocessed to obtain a final knee joint CT or MR image; wherein the preprocessing at least includes noise removal, image enhancement, edge detection and region segmentation.
[0075] In one embodiment, preoperative planning is performed based on the three-dimensional model of the knee joint to obtain preoperative planning results, including:
[0076] Based on the 3D model of the knee joint, select the appropriate model and specification of the knee prosthesis;
[0077] Based on the three-dimensional model of the knee joint, the prosthesis is accurately positioned, including the rotation angle, medial and lateral position, anterior and posterior position, and implantation depth of the prosthesis.
[0078] Plan the osteotomy path during surgery based on the design of the prosthesis and the patient's bone morphology.
[0079] In one embodiment, an RGB depth camera is used to acquire a surrounding environment image in real time during surgery; wherein the surrounding environment image includes visual markers, including:
[0080] Before the operation begins, fix the RGB depth camera in a suitable position so that it can cover the surgical area;
[0081] During the operation, the RGB depth camera collects the surrounding environment images in real time; the surrounding environment images include all visual information in the surgical area, especially surgical instruments, patient anatomical structures and surrounding visual markers.
[0082] Figure 2 FIG. 1 is a flow chart of an intelligent agent outputting action decisions according to an embodiment of the present application. Figure 2 As shown, in one embodiment, the intelligent agent performs multimodal information fusion of the planned osteotomy path, the surrounding environment image, the position coordinates of the visual marker, and the language instructions acquired in real time, including:
[0083] De-noise and enhance the surrounding environment image;
[0084] Parse and encode language instructions;
[0085] Extract multimodal features from the planned osteotomy path, denoised and enhanced surrounding environment images, visual marker position coordinates, and parsed and encoded language instructions;
[0086] Use deep learning models or multimodal information fusion algorithms to fuse multimodal features.
[0087] In one embodiment, a deep learning model or a multimodal information fusion algorithm is used to perform feature fusion on multimodal features, including:
[0088] Determine the weights of multimodal features respectively;
[0089] The Transformer based on the attention mechanism fuses the multimodal features corresponding to each weight.
[0090] In one embodiment, controlling the robotic arm to complete automatic osteotomy according to the fusion result includes:
[0091] Obtaining a calibrated osteotomy tool coordinate system;
[0092] Based on the osteotomy tool coordinate system, the planned osteotomy path is mapped into an osteotomy path in the robot arm flange coordinate system;
[0093] Based on the kinematic constraint formula, the osteotomy path in the robot arm flange coordinate system is converted into the joint angle trajectory of each joint;
[0094] Interpolate the joint angle trajectory of each joint;
[0095] The robot arm is controlled by outputting the joint angular velocity of each joint angle through a feedforward PID controller;
[0096] Acquire the positioning data of the osteotomy tool positioning frame and the positioning data of the marking positioning frame of the bone to be osteotomized in real time;
[0097] The target joint angle trajectory is updated based on the positioning data of the osteotomy tool positioning frame and the positioning data of the marking positioning frame of the bone to be osteotomized, and the interpolation process of the joint angle trajectory of each joint is returned.
[0098] Figure 3 is a structural schematic diagram of a joint replacement positioning and navigation system based on embodied intelligence provided by an embodiment of the present application;
[0099] The joint replacement positioning navigation system based on embodied intelligence includes:
[0100] A joint image acquisition module 301 is used to acquire a knee joint image of a patient;
[0101] A three-dimensional reconstruction module 303 is used to segment and three-dimensionally reconstruct the knee joint image to obtain a three-dimensional model of the knee joint;
[0102] The preoperative planning module 303 is used to perform preoperative planning based on the three-dimensional model of the knee joint and obtain preoperative planning results; wherein the preoperative planning results include the position of the knee joint prosthesis, the prosthesis model and the planned osteotomy path;
[0103] The depth image acquisition module 304 is used to acquire the surrounding environment image in real time using the RGB depth camera during the operation; wherein the surrounding environment image includes visual markers;
[0104] A position coordinate determination module 305, for determining the position coordinates of the visual marker based on the visual marker;
[0105] The information fusion module 306 is used for the intelligent agent to perform multimodal information fusion of the planned osteotomy path, the surrounding environment image, the position coordinates of the visual marker, and the language instructions acquired in real time, and control the robotic arm to complete automatic osteotomy according to the fusion result.
[0106] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.
[0107] The electronic device may include a processor 401 and a memory 402 storing computer program instructions.
[0108] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0109] Memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, memory 402 may be inside or outside of an electronic device. In a particular embodiment, memory 402 may be a non-volatile solid-state memory.
[0110] In one embodiment, the memory 402 may be a read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.
[0111] The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement any one of the joint replacement positioning and navigation methods based on embodied intelligence in the above embodiments.
[0112] In one example, the electronic device may further include a communication interface 404 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 404 are connected via a bus 410 and communicate with each other.
[0113] The communication interface 404 is mainly used to implement communication between various modules, systems, units and / or devices in the embodiments of the present application.
[0114] Bus 410 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.
[0115] In addition, in combination with the joint replacement positioning and navigation method based on embodied intelligence in the above embodiments, the present application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the joint replacement positioning and navigation methods based on embodied intelligence in the above embodiments is implemented.
[0116] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0117] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0118] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or systems. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0119] The above reference is according to the method of the embodiment of the present application, the flow chart of system (system) and computer program product and / or block diagram and describe various aspects of the present application.It should be understood that each square frame in the flow chart and / or block diagram and the combination of each square frame in the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer, special-purpose computer or other programmable data processing system, to produce a kind of machine, so that these instructions executed by the processor of computer or other programmable data processing system enable the realization of the function / action specified in one or more square frames of flow chart and / or block diagram.Such processor can be but not limited to general-purpose processor, special-purpose processor, special application processor or field programmable logic circuit.It can also be understood that each square frame in the block chart and / or flow chart and the combination of square frames in the block chart and / or flow chart can also be realized by the special-purpose hardware that performs the specified function or action, or can be realized by the combination of special-purpose hardware and computer instructions.
[0120] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A joint replacement positioning and navigation method based on embodied intelligence, characterized in that: include: Obtain an image of the patient's knee joint; Segment and 3D reconstruct the knee joint image to obtain a 3D model of the knee joint; Perform preoperative planning based on the three-dimensional model of the knee joint to obtain preoperative planning results; the preoperative planning results include the position of the knee joint prosthesis, the prosthesis model and the planned osteotomy path; During the operation, an RGB depth camera is used to obtain a real-time image of the surrounding environment, wherein the surrounding environment image includes visual markers; Based on the visual marker, determining the coordinates of the position of the visual marker; The intelligent agent will perform multimodal information fusion of the planned osteotomy path, surrounding environment images, visual marker position coordinates, and real-time acquired language instructions, and control the robotic arm to complete automatic osteotomy based on the fusion results.
2. The joint replacement positioning and navigation method based on embodied intelligence according to claim 1, characterized in that: Obtain images of the patient's knee, including: Using high-resolution CT or MR scanning equipment, obtain the patient's initial knee CT or MR images; The initial knee joint CT or MR image is preprocessed to obtain a final knee joint CT or MR image; wherein the preprocessing at least includes noise removal, image enhancement, edge detection and region segmentation.
3. The joint replacement positioning and navigation method based on embodied intelligence according to claim 1, characterized in that: Preoperative planning is performed based on the three-dimensional model of the knee joint to obtain preoperative planning results, including: Based on the 3D model of the knee joint, select the appropriate model and specification of the knee prosthesis; Based on the three-dimensional model of the knee joint, the prosthesis is accurately positioned, including the rotation angle, medial and lateral position, anterior and posterior position, and implantation depth of the prosthesis. Plan the osteotomy path during surgery based on the design of the prosthesis and the patient's bone morphology.
4. The joint replacement positioning and navigation method based on embodied intelligence according to claim 1, characterized in that: During the operation, an RGB depth camera is used to obtain real-time images of the surrounding environment; the surrounding environment images include visual markers, including: Before the operation begins, fix the RGB depth camera in a suitable position so that it can cover the surgical area; During the operation, the RGB depth camera collects the surrounding environment images in real time; the surrounding environment images include all visual information in the surgical area, especially surgical instruments, patient anatomical structures and surrounding visual markers.
5. The joint replacement positioning and navigation method based on embodied intelligence according to claim 1, characterized in that: The intelligent agent integrates the planned osteotomy path, the surrounding environment image, the position coordinates of the visual markers, and the language instructions obtained in real time for multimodal information fusion, including: De-noise and enhance the surrounding environment image; Parse and encode language instructions; Extract multimodal features from the planned osteotomy path, denoised and enhanced surrounding environment images, visual marker position coordinates, and parsed and encoded language instructions; Use deep learning models or multimodal information fusion algorithms to fuse multimodal features.
6. The joint replacement positioning and navigation method based on embodied intelligence according to claim 5, characterized in that: Use deep learning models or multimodal information fusion algorithms to fuse multimodal features, including: Determine the weights of the multimodal features respectively; The Transformer based on the attention mechanism fuses the multimodal features corresponding to each weight.
7. The joint replacement positioning and navigation method based on embodied intelligence according to claim 1, characterized in that: According to the fusion results, the robot arm is controlled to complete automatic osteotomy, including: Obtaining a calibrated osteotomy tool coordinate system; Based on the osteotomy tool coordinate system, the planned osteotomy path is mapped into an osteotomy path in the robot arm flange coordinate system; Based on the kinematic constraint formula, the osteotomy path in the robot arm flange coordinate system is converted into the joint angle trajectory of each joint; Interpolate the joint angle trajectory of each joint; The feedforward PID controller outputs the joint angular velocity of each joint angle to control the robot arm; Acquire the positioning data of the osteotomy tool positioning frame and the positioning data of the marking positioning frame of the bone to be osteotomized in real time; The target joint angle trajectory is updated based on the positioning data of the osteotomy tool positioning frame and the positioning data of the marking positioning frame of the bone to be osteotomized, and the interpolation process of the joint angle trajectory of each joint is returned.
8. A joint replacement positioning navigation system based on embodied intelligence, characterized in that: The system comprises: A joint image acquisition module, used for acquiring a patient's knee joint image; A three-dimensional reconstruction module is used to segment and three-dimensionally reconstruct the knee joint image to obtain a three-dimensional model of the knee joint; A preoperative planning module is used to perform preoperative planning based on a three-dimensional model of the knee joint and obtain preoperative planning results; wherein the preoperative planning results include the position of the knee joint prosthesis, the prosthesis model and the planned osteotomy path; A depth image acquisition module is used to acquire a surrounding environment image in real time using an RGB depth camera during surgery; wherein the surrounding environment image includes visual markers; A position coordinate determination module, used to determine the position coordinates of the visual marker based on the visual marker; The information fusion module is used by the intelligent agent to perform multimodal information fusion of the planned osteotomy path, surrounding environment images, visual marker position coordinates, and real-time acquired language instructions, and control the robotic arm to complete automatic osteotomy based on the fusion results.
9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the joint replacement positioning and navigation method based on embodied intelligence as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the joint replacement positioning and navigation method based on embodied intelligence as described in any one of claims 1-7.
Citation Information
Patent Citations
Multi-modal fusion mechanical arm control method based on interactive learning
CN115114975A
Full-automatic space registration system based on multi-modal information fusion
CN115358995A
Robot system for assisting periacetabular osteotomy
CN115381526A
Autonomous joint replacement surgical robot navigation positioning system
CN116983086A
Cited By
Ultrasonic system, method and product based on multi-mode body model
CN119517357A
