In-situ biological 3D printing robot system, printing method, medium, terminal and robot

Abstract

The application provides an in-situ biological 3D printing robot system, a printing method, a medium, a terminal and a robot. First, optimal robot-assisted grinding and repair reconstruction path planning is performed according to the three-dimensional structure space coordinates of a cartilage repair body obtained by constructing the cartilage repair body based on input medical image information, so as to obtain an optimal surgical planning scheme. In the surgical process, human-robot cooperation is performed according to the optimal surgical planning scheme to obtain an optimal robot planning scheme. Then, in-situ biological 3D printing is performed according to the optimal robot planning scheme. The robot body controls the robot end to perform in-situ biological 3D printing, so that the 3D printing precision is high, and the 3D printing efficiency is also high.

Description

TECHNICAL FIELD

[0001] The present application relates to a biological 3D printing robot, in particular to an in-situ biological 3D printing robot system, a printing method, a medium, a terminal and a robot. BACKGROUND

[0002] Knee cartilage-bone injury caused by degenerative or traumatic lesions is the most common knee disease in clinic, which seriously affects the motor function and quality of life of patients, and leads to limb disability. The existing treatment methods include arthroscopic autologous cartilage-bone transplantation repair or joint replacement, with more than 300,000 cases per year in China, the treatment cost reaches 60 billion yuan, and still grows at a rate of 15% per year. The above repair methods have problems such as limited supply area, additional surgical damage, and expensive prostheses. In-situ biological 3D printing technology can better solve the above problems, that is, the patient's cells, biological materials, and biological active factors are directly printed at the site, which can accurately adapt to the target defect site and repair cartilage-bone defects.

[0003] At present, when using biological 3D printing technology to repair knee cartilage-bone injury, a small-sized printing gun is generally used to extrude inkjet and light curing for plastic forming. The small-sized printing gun solves the problem that the biological 3D printer is too large to directly complete in-situ printing. However, since the current printing gun control method is generally a handheld method, this method has many shortcomings: first, the printing gun space pose needs to be adjusted repeatedly in the operation, and accurate positioning of the printing site cannot be achieved; second, the printing gun currently used needs to be adjusted repeatedly to adjust the ink volume, and the volume and printing speed of the printing ink cannot be intelligently controlled during the biological 3D printing process; third, the light curing speed is also difficult to control, and the time of light plastic forming cannot be accurately controlled. The above shortcomings result in the problems of single structure and function, low collaboration efficiency, and insufficient printing precision of the in-situ biological 3D printing gun. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide an in-situ biological 3D printing robot system, a printing method, a medium, a terminal and a robot, which solves the problems of single structure and function, low collaboration efficiency, and insufficient printing precision of the 3D printing gun in the prior art.

[0005] To achieve the above object and other related objects, the first aspect of the present application provides a robot system for in-situ biological 3D printing, comprising: a robot body configured to perform optimal robot-assisted grinding and repair reconstruction path planning based on three-dimensional structure space coordinates of a cartilage repair body obtained by constructing the cartilage repair body based on input medical image information before surgery, so as to obtain an optimal surgery planning scheme, and to perform human-robot cooperation for robot obstacle avoidance trajectory according to the optimal surgery planning scheme during surgery to obtain an optimal robot planning scheme; and a robot end connected to the robot body and configured to perform in-situ biological 3D printing according to the optimal robot planning scheme.

[0006] In some embodiments of the first aspect of the present application, the robot body comprises: an intelligent planning system configured to construct three-dimensional structure space coordinates of a cartilage repair body based on input medical image information before surgery based on an intelligent planning model, and to perform optimal robot-assisted grinding and repair reconstruction path planning based on input surgery region information and the three-dimensional structure space coordinates of the cartilage repair body, so as to obtain an optimal surgery planning scheme; wherein the optimal surgery planning scheme comprises: optimal robot-assisted grinding and repair reconstruction path, volume of 3D printing ink, 3D printing speed, and photocuring speed; and an operation cooperation system connected to the intelligent planning system and configured to perform human-robot cooperation for robot obstacle avoidance trajectory according to the optimal surgery planning scheme during surgery based on a reinforcement learning model to obtain an optimal robot planning scheme; wherein the optimal robot planning scheme comprises: optimal robot-assisted grinding and repair reconstruction path obtained by performing human-robot cooperation for robot obstacle avoidance trajectory, volume of the 3D printing ink, 3D printing speed, and photocuring speed.

[0007] In some embodiments of the first aspect of the present application, the intelligent planning model comprises: a primary planning module configured to plan a plurality of robot-assisted grinding and repair reconstruction paths based on three-dimensional structure space coordinates of a cartilage repair body obtained by constructing the cartilage repair body based on input medical image information before surgery; and an optimization planning module connected to the primary planning module and configured to obtain corresponding registration information generated based on different surgery environments based on input surgery region information, and to confirm an optimal robot-assisted grinding and repair reconstruction path from the plurality of robot-assisted grinding and repair reconstruction paths obtained by the primary planning module based on the registration information, so as to obtain an optimal surgery planning scheme; wherein the optimal surgery planning scheme comprises: optimal robot-assisted grinding and repair reconstruction path, volume of 3D printing ink, 3D printing speed, and photocuring speed.

[0008] In some embodiments of the first aspect of the present application, the method of training the intelligent planning model comprises: constructing a sample set based on medical image data, surgical region information, and surgical planning scheme point cloud data of patients corresponding to a plurality of successful surgical samples; inputting the sample set into a neural network point cloud segmentation algorithm model to obtain the intelligent planning model through training.

[0009] In some embodiments of the first aspect of the present application, the human-robot collaboration based on the reinforcement learning model to obtain the optimal robot planning scheme according to the optimal surgical planning scheme for robot obstacle avoidance trajectory comprises: optimizing the robot-assisted grinding and repair reconstruction path based on position information of each obstacle obtained based on a trajectory path generated by guiding the robot end to avoid each obstacle in the surgical environment to a starting point of the robot-assisted grinding and repair reconstruction path.

[0010] In some embodiments of the first aspect of the present application, the robot end comprises: a mechanical arm and an end biological printing system; wherein the mechanical arm is connected with the end biological printing system, and is used to drive the end biological printing system to perform in-situ biological 3D printing according to the optimal robot planning scheme.

[0011] In some embodiments of the first aspect of the present application, the end biological printing system further comprises: a rapid end effector switching system, a grinding head power system, a puncture system, and a biological 3D printing gun; wherein the rapid end effector switching system, the grinding head power system, the puncture system, and the biological 3D printing gun; wherein the rapid end effector switching system is located on the mechanical arm and is connected with the grinding head power system, the puncture system, and the biological 3D printing gun respectively, and is used to switch the grinding head power system, the puncture system, and the biological 3D printing gun according to different needs in the surgical process.

[0012] In some embodiments of the first aspect of the present application, the ink used by the biological 3D printing gun for 3D printing is a novel bioactive gel ink.

[0013] In some embodiments of the first aspect of the present application, an ultraviolet generator is arranged on the biological 3D printing gun, and is used to perform light curing molding on the ink used for 3D printing.

[0014] To achieve the above object and other related objects, the second aspect of the present application provides an in-situ biological 3D printing method, which comprises: obtaining medical image information of a patient; performing optimal robot-assisted grinding and repair reconstruction path planning on three-dimensional structure space coordinates of a cartilage repair body obtained by constructing the cartilage repair body based on the input medical image information before surgery, so as to obtain an optimal surgery planning scheme, and performing human-machine cooperation of robot obstacle avoidance trajectory according to the optimal surgery planning scheme during surgery to obtain an optimal robot planning scheme; and performing in-situ biological 3D printing according to the optimal robot planning scheme.

[0015] To achieve the above object and other related objects, the third aspect of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the in-situ biological 3D printing method.

[0016] To achieve the above object and other related objects, the fourth aspect of the present application provides an electronic terminal, which comprises: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the in-situ biological 3D printing method.

[0017] To achieve the above object and other related objects, the fifth aspect of the present application provides an in-situ biological 3D printing robot, which comprises: the in-situ biological 3D printing robot system as described above.

[0018] As described above, the in-situ biological 3D printing robot system, printing method, medium, terminal and robot of the present application have the following beneficial effects: the present application firstly performs optimal robot-assisted grinding and repair reconstruction path planning on three-dimensional structure space coordinates of a cartilage repair body obtained by constructing the cartilage repair body based on input medical image information, so as to obtain an optimal surgery planning scheme, and performs human-machine cooperation of robot obstacle avoidance trajectory according to the optimal surgery planning scheme during surgery to obtain an optimal robot planning scheme, and then performs in-situ biological 3D printing according to the optimal robot planning scheme. The present application controls the robot end by the robot body to perform in-situ biological 3D printing, which not only has high 3D printing precision, but also has high 3D printing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Fig. 1 shows a schematic diagram of an in-situ biological 3D printing robot system in an embodiment of the present application.

[0020] Figure 2 Fig. 3 shows a schematic diagram of an intelligent planning model in an embodiment of the present application.

[0021] Figure 3 Fig. 5 shows a schematic diagram of an optical guiding device in an embodiment of the present application.

[0022] Figure 4 A schematic diagram of an optical navigation dynamic reference frame and a marker ball is shown.

[0023] Figure 5 A schematic diagram of a method for training an intelligent planning model is shown.

[0024] Figure 6 A schematic diagram of a workstation with a display system is shown.

[0025] Figure 7 A schematic diagram of a grinding head power system, a biological 3D printing gun, and a puncture system is shown.

[0026] Figure 8 A schematic diagram of an in-situ biological 3D printing method is shown.

[0027] Figure 9 A schematic diagram of the structure of an electronic terminal is shown.

[0028] Figure 10 A schematic diagram of a knee cartilage-bone injury repair in-situ biological 3D printing robot system is shown. DETAILED DESCRIPTION

[0029] The present application will be described by specific, detailed embodiments, and one skilled in the art will readily recognize other advantages and purposes that the present application can be used for based on the disclosure herein. The present application can be embodied in other different embodiments and applied to other different scenarios, and each detail herein can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0030] It should be noted that in the following description, reference is made to the accompanying drawings, which form a part hereof, and in which are shown by way of illustration several embodiments of the present application. It is to be understood that other embodiments can be utilized and that mechanical, structural, electrical, and operational changes can be made without departing from the spirit and scope of the present application. The following detailed description is not to be interpreted as limiting the embodiments of the present application, and the scope of the embodiments of the present application are defined by the appended claims. The terminology used here is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. Spatially relative terms, such as "upper", "lower", "left", "right", "below", "beneath", "bottom", "top", and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures.

[0031] In this application, unless otherwise clearly indicated and limited, the terms "mounting", "connection", "connecting", "fixing", "holding" and the like should be interpreted broadly, for example, can be fixedly connected, can be detachably connected, or integrally connected; can be mechanically connected, or electrically connected; can be directly connected, or indirectly connected through an intermediate medium, or can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in this application can be understood according to the specific circumstances.

[0032] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of data herein so described can be interchanged, unless otherwise understood by those skilled in the art to be within the concept of the embodiments described herein. Also, the terms "comprise", "comprising", "include", "including", "contain", "containing", "have" and "having" and the like are meant to be inclusive, rather than mutually exclusive, unless the context clearly dictates otherwise. It will be further understood that the terms "or" and "and / or" as used herein, are to be interpreted as inclusive, or meaning either or any combination of items. Thus, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Only when the combination of elements, functions, or operations are inherently mutually exclusive is an exception to this definition.

[0033] Before the present application is further described, the nomenclature used in the description of the embodiments of the present application will be explained. The nomenclature used in the description of the embodiments of the present application is applicable as follows:

[0034] Biological 3D printing technology: 3D printing technology is a new application technology based on computer three-dimensional digital imaging technology and multi-layer continuous printing. Biological 3D printing technology is a 3D printing technology based on 3D printing, which prints biological materials or cells according to the requirements of biomorphic morphology, biological function, cell-specific microenvironment, etc. to print biomedical products such as biological three-dimensional structure with complex structure and function, three-dimensional biological function body in vitro, and regenerative medicine model.

[0035] Printing ink: biological materials (hydrogel, etc.) and biological units (cells, DNA, proteins, etc.) are manufactured by 3D printing according to the requirements of biomorphic morphology, biological function, cell growth microenvironment, etc. to manufacture biological functional structures with individualization.

[0036] 3D printing light solidification molding: light solidification molding is the earliest 3D printing molding technology, and is also the relatively mature 3D printing technology at present. The basic principle of the technology is to use the accumulation molding of materials, divide the shape of a three-dimensional target part into several plane layers, scan the liquid photosensitive resin with a light beam of a certain wavelength, so that the part of each layer of liquid photosensitive resin scanned is solidified, and the place not irradiated by the light beam is still liquid. Finally, each layer is accumulated to form the required target part, and the material utilization rate can be close to 100%.

[0037] In order to make the purpose, technical scheme and advantages of the present application clearer and more apparent, the technical scheme of the embodiments of the present application will be further described in detail below with reference to the following examples and in conjunction with the drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0038] As shown in Figure 1 , a kind of in-situ biological 3D printing robot system in the embodiments of the present application is shown.

[0039] The system comprises:

[0040] The robot body 1 is used to perform optimal robot-assisted grinding and repair reconstruction path planning according to the three-dimensional structure space coordinates of the cartilage repair body obtained by constructing the cartilage repair body based on input medical image information before surgery, so as to obtain an optimal surgical planning scheme, and perform human-machine cooperation according to the optimal surgical planning scheme during surgery to obtain an optimal robot planning scheme. Wherein, the input medical image information is CT image information obtained by CT scanning (computed tomography) of the knee joint cartilage region of the patient before surgery. More preferably, the knee joint cartilage region of the patient is scanned by spiral CT, and the CT image data information is in DICOM format. DICOM is the most original image data, which contains the most comprehensive, detailed and rich information. The robot-assisted grinding and repair reconstruction path includes the path for assisting grinding and repair of each defect site of the knee joint cartilage region of the patient.

[0041] The robot end 2 is connected with the robot body 1, and is used to perform in-situ biological 3D printing according to the optimal robot planning scheme; wherein, the in-situ biological 3D printing is to print the patient cells, biomaterials and bioactive factors as 3D printing ink at the defect site to precisely adapt to the target defect site and repair the cartilage-bone defect.

[0042] For the convenience of understanding, the process of the robot system for 3D printing will be described in detail as follows:

[0043] Before the surgery, the robot body 1 constructs the cartilage repair body based on the input medical image information of the patient to obtain the three-dimensional structure spatial coordinates of the cartilage repair body, and plans the robot-assisted grinding and repair reconstruction path according to the three-dimensional structure spatial coordinates of the cartilage repair body. The robot body 1 can generate multiple robot-assisted grinding and repair reconstruction paths according to the three-dimensional structure spatial coordinates, but not every robot-assisted grinding and repair reconstruction path can be applied in the surgical environment where the patient is located. In order to prevent the robot end 2 from colliding with each surgical instrument in the actual surgical environment and ensure the safety of the surgery, the robot body 1 needs to plan the optimal planning robot-assisted grinding and repair reconstruction path that is suitable for the actual surgical environment, so as to obtain the optimal surgical planning scheme. During the surgery, some information cannot be displayed by the medical image information, such as the information of the soft tissue pulled near the knee joint. Therefore, the human-computer cooperation of the robot obstacle avoidance trajectory needs to be performed according to the optimal surgical planning scheme to obtain the optimal robot planning scheme. After obtaining the optimal robot planning scheme, the robot body 1 controls the robot end 2 to perform in-situ biological 3D printing according to the optimal robot planning scheme.

[0044] In some embodiments of the present application, as shown in Figure 2 The robot body 1 includes an intelligent planning system 11, which is used to construct the cartilage repair body based on an intelligent planning model according to the input medical image information before the surgery to obtain the three-dimensional structure spatial coordinates of the cartilage repair body, and plan the optimal robot-assisted grinding and repair reconstruction path according to the input surgical area information and the three-dimensional structure spatial coordinates of the cartilage repair body, so as to obtain the optimal surgical planning scheme. The optimal surgical planning scheme includes the optimal robot-assisted grinding and repair reconstruction path, the volume of 3D printing ink, the 3D printing speed, and the light curing speed. The optimal robot-assisted grinding and repair reconstruction path is the path for in-situ biological 3D printing repair surgery of each defect site of the knee cartilage area of the patient. The volume of 3D printing ink is the amount of printing ink used for 3D printing of each defect site. The 3D printing speed is the printing speed when 3D printing each defect site. The light curing speed is the speed of light curing of the 3D printing ink. By planning the above surgical planning scheme for the patient, efficient and accurate in-situ biological 3D printing of each defect site of the knee cartilage area of the patient can be realized

[0045] The operation cooperation system 12 is connected with the intelligent planning system 11, and is used for obtaining an optimal robot planning scheme by human-robot cooperation of robot obstacle avoidance trajectory based on the reinforcement learning model according to the optimal operation planning scheme during the operation; wherein the optimal robot planning scheme comprises: an optimal robot assisted grinding and repair reconstruction path obtained by human-robot cooperation of robot obstacle avoidance trajectory, a volume of the 3D printing ink, a 3D printing speed and a light curing speed.

[0046] In some embodiments of the present application, the operation area information comprises: three-dimensional operation environment model information obtained by a three-dimensional modeling method based on a real object data set of a surgical instrument used by a doctor in an operation environment.

[0047] In some embodiments of the present application, as shown in Figure 2 The intelligent planning model 3 comprises: a primary planning module 31, which is used for constructing a three-dimensional structure space coordinate of the cartilage repair body according to input medical image information to obtain a plurality of robot assisted grinding and repair reconstruction paths corresponding to the cartilage repair body; and an optimization planning module 32 connected with the primary planning module 32, which is used for obtaining corresponding registration information generated based on different operation environments according to input operation area information, and confirming an optimal robot assisted grinding and repair reconstruction path from the plurality of robot assisted grinding and repair reconstruction paths obtained by the primary planning module based on the registration information, so as to obtain an optimal operation planning scheme; wherein the optimal operation planning scheme comprises: the optimal robot assisted grinding and repair reconstruction path, the volume of the 3D printing ink, the 3D printing speed and the light curing speed.

[0048] In some embodiments of the present application, the robot body further comprises: an optical guiding device as shown in Figure 3 and an optical navigation dynamic reference frame as shown in Figure 4 The registration information is generated by the optical navigation dynamic reference frame and the optical guiding device; wherein the optical guiding device is a binocular optical navigation system; the optical navigation dynamic reference frame is provided with a plurality of marker balls as shown in Figure 4 The optical navigation dynamic reference frame is arranged in the knee joint cartilage area to be repaired during the operation.

[0049] Specifically, the method for generating the registration information by the optical guiding device and the optical navigation dynamic reference frame comprises: confirming position data of each marker ball by the optical guiding device; and completing registration of the robot system and the actual operation environment based on the operation area information and the position data of each marker ball; wherein the registration of the robot system and the actual operation environment comprises: mapping virtual position information of each defect site to position information of each defect site in a real space, and mapping the position information of each defect site in the real space to a robot end coordinate system.

[0050] It should be noted that the position information of each defect site in the real space must be mapped to the robot end coordinate system, and the robot end can perform in-situ biological 3D printing according to the corresponding surgical planning scheme.

[0051] In some embodiments of the present application, as shown in Figure 5 The training method of the intelligent planning model includes:

[0052] Step S501: Constructing a sample set based on the medical image data of the patients corresponding to a plurality of successful surgery samples, the surgery region information and the surgery planning scheme point cloud data; wherein the sample set includes: a training sample set and a test sample set; wherein the medical image information includes: CT image information obtained by CT scanning of the knee joint cartilage region of the patient before surgery; the surgery region information includes: three-dimensional surgery environment model information obtained by a three-dimensional modeling method based on the surgery instrument real data set used by the doctor in the surgery environment; the surgery planning scheme point cloud data includes: robot-assisted grinding and repair reconstruction path, 3D printing ink volume, 3D printing speed and photocuring speed point cloud data.

[0053] Preferably, the training sample set and the test sample set are randomly sampled according to a ratio of 4:1.

[0054] Step S502: inputting the sample set into a neural network point cloud segmentation algorithm model to train and obtain the intelligent planning model. Specifically, the sample set is inputted into the neural network point cloud segmentation algorithm model for feature extraction of the sample set, and the intelligent planning model is trained and obtained based on the extracted features.

[0055] Preferably, when the sample set is feature extracted, a related deep learning algorithm is used to improve the local point cloud aggregation strategy to optimize the point embedding and enhance the knee joint prosthesis point cloud local feature extraction capability.

[0056] In some embodiments of the present application, after obtaining the optimal surgery planning scheme, due to the differences between the actual surgery environment and the modeled surgery environment and the fact that the medical image information cannot accurately display the soft tissue near the knee joint, it is necessary to optimize the optimal robot-assisted grinding and repair reconstruction path through a reinforcement learning model. The human-robot cooperation based on the reinforcement learning model for robot obstacle avoidance trajectory according to the optimal surgery planning scheme to obtain the optimal robot planning scheme includes: optimizing the robot-assisted grinding and repair reconstruction path based on the position information of each obstacle obtained by the trajectory path generated by the corresponding guide robot end avoiding each obstacle in the surgery environment to the starting point of the robot-assisted grinding and repair reconstruction path.

[0057] Specifically, the doctor guides the robot end to avoid the surgical instruments in the surrounding environment and the soft tissues near the knee joint to approach the starting point of the robot-assisted grinding and repair reconstruction path, obtains the position information of each obstacle through a non-contact sensing detection scheme based on the trajectory path of the robot end corresponding to each obstacle in the surgical environment to the starting point of the robot-assisted grinding and repair reconstruction path, and optimizes the optimal robot-assisted grinding and repair reconstruction path based on the position information of each obstacle by a reinforcement learning model.

[0058] In some embodiments of the present application, the non-contact sensing detection scheme can be visual detection or acoustic wave detection.

[0059] In some embodiments of the present application, an Actor-Critic architecture is used to build the reinforcement learning model, which includes two modules of local network and global network, and each sub-network includes an Actor and two Critic networks.

[0060] It should be noted that other sensing detection schemes and other types of reinforcement learning models can be used to optimize the optimal surgical path, and the present application does not limit this.

[0061] In some embodiments of the present application, the robot end includes a mechanical arm and an end biological printing system; wherein the mechanical arm is connected with the robot body and the end biological printing system respectively, and the mechanical arm is used to drive the end biological printing system to perform in-situ biological 3D printing according to the optimal robot planning scheme.

[0062] In some embodiments of the present application, the mechanical arm is a 6-DOF mechanical arm. It should be noted that other types of mechanical arms can be used to drive the end biological printing system to perform in-situ biological 3D printing, and the present application does not limit this.

[0063] In some embodiments of the present application, the optical guiding device has a navigation function in the surgical process to realize real-time tracking and monitoring of the mechanical arm; specifically, the robot body determines the positions of each defect site based on the position information of each optical marker ball obtained by the optical guiding device in real time to control the mechanical arm to drive the end biological printing system to perform in-situ biological 3D printing at each defect site.

[0064] In some embodiments of the present application, as shown in Figure 6 The robot body is a workstation with a display system; wherein the display system is used to display navigation image information; the navigation image information includes information of the optimal robot planning scheme; the optimal robot planning scheme information can be displayed in the form of an image through the display system so that the doctor can more intuitively see the optimal robot planning scheme information through the display system.

[0065] In some embodiments of the present application, since the defect site of the patient cannot be directly 3D printed sometimes, the defect site needs to be punctured and polished. For the above purpose, as shown in Figure 7 The end biological printing system further comprises a rapid end effector switching system, a polishing head power system, a puncture system and a biological 3D printing gun. The rapid end effector switching system is located on the mechanical arm and is connected with the polishing head power system, the puncture system and the biological 3D printing gun respectively, and is used for switching the polishing head power system, the puncture system and the biological 3D printing gun at different stages of the operation process. The rapid end effector switching system can automatically switch the polishing head power system, the puncture system and the biological 3D printing gun connected therewith during the operation process to meet different needs in the operation.

[0066] In some embodiments of the present application, the TCP of the robot system is fixed by a flange when designing the rapid end effector switching system.

[0067] In some embodiments of the present application, the doctor can also manually switch the polishing head power system, the puncture system and the biological 3D printing gun during the operation process.

[0068] In some embodiments of the present application, the biological 3D printing gun is a pneumatic portable biological 3D printing gun, which comprises a temperature control system, a gas pressure control system, a pushing rod and a needle cylinder. Specifically, the needle cylinder is used for storing 3D printing ink; the gas pressure control system is used for providing the gas pressure required for the flow output of the 3D printing ink; the temperature control system is used for keeping the internal temperature of the printing gun constant; and the pushing rod is used for extruding the 3D printing ink stored in the needle cylinder.

[0069] In some embodiments of the present application, the ink used by the biological 3D printing gun for 3D printing is a new type of bioactive gel ink. The preparation method of the new type of bioactive gel ink is as follows: collecting early primary cells (3-5 generations) of experimental objects to obtain a cell suspension; drying degummed silk to obtain a silk fibroin solution; adding a glycidyl methacrylate (GMA) solution to the silk fibroin solution to obtain a mixed solution; dialyzing a polymer solution obtained by stirring the mixed solution in deionized water, and storing a freeze-dried sample; adding the cell suspension to the filtered biological ink to obtain the new type of bioactive gel ink.

[0070] In some embodiments of the present application, an ultraviolet generator is further arranged on the biological 3D printing gun to perform light curing molding on the ink used for 3D printing.

[0071] As Figure 8As shown, a method for in-situ biological 3D printing is shown. The method is applied to an in-situ biological 3D printing robot system, comprising:

[0072] Step S801: obtaining medical image information of a patient;

[0073] Step S802: before surgery, performing optimal robot-assisted grinding and repair reconstruction path planning according to the three-dimensional structure space coordinates of the cartilage repair body obtained by constructing the cartilage repair body based on the input medical image information, to obtain an optimal surgical planning scheme, and performing human-machine cooperation of robot obstacle avoidance trajectory according to the optimal surgical planning scheme during surgery to obtain an optimal robot planning scheme;

[0074] Step S803: in-situ biological 3D printing according to the optimal robot planning scheme.

[0075] The application also provides a terminal 9, as shown by Figure 9 comprising: a processor 92 and a memory 91; the memory 91 is used to store a computer program; the processor 92 is used to execute the computer program stored in the memory, so that the terminal 9 executes the in-situ biological 3D printing method as shown by Figure 8 .

[0076] Optionally, the number of memories 91 can be one or more, the number of processors 92 can be one or more, and the number of terminals 9 can be one or more. Figure 9 In the description, one is taken as an example.

[0077] Optionally, the processor 92 in the control device will load one or more instructions corresponding to the process of the application program into the memory 91 according to the steps as shown by Figure 8 , and the processor 92 runs the application program stored in the first memory, so as to realize various functions in the in-situ biological 3D printing method as shown by Figure 8 .

[0078] Optionally, the memory 91 can include, but is not limited to, a high-speed random access memory, a nonvolatile memory, for example, one or more disk storage devices, a flash memory device, or other nonvolatile solid-state storage device; the processor 92 can include, but is not limited to, a central processing unit (CPU), a network processing unit (NP), etc., and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component.

[0079] Optionally, the processor 92 can be a general-purpose processor, including a central processing unit (CPU), a network processing unit (NP), etc., and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component.

[0080] The application also provides a computer readable storage medium storing a computer program, the computer program being configured to implement the in-situ biological 3D printing method when running. Figure 8 The computer readable storage medium can include, but is not limited to, a floppy disk, an optical disk, a CD-ROM (compact disk-read only memory), a magneto-optical disk, a ROM (read only memory), a RAM (random access memory), an EPROM (erasable programmable read only memory), an EEPROM (electrically erasable programmable read only memory), a magnetic card or an optical card, a flash memory, or other types of media / machine readable media suitable for storing machine executable instructions. The computer readable storage medium can be a product not connected to a computer device, or a component connected to a computer device.

[0081] In some embodiments of the present application, the computer readable storage medium can include read only memory, random access memory, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, U disk, mobile hard disk, or any other medium capable of storing desired program code in the form of instructions or data structures and capable of being accessed by a computer. In addition, any connection can be appropriately referred to as a computer readable medium. For example, if instructions are sent from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave is included in the definition of the medium. However, it should be understood that the computer readable storage medium and the data storage medium do not include connections, carriers, signals or other transitory media, but are intended for non-transitory, tangible storage media. As used in the application, magnetic disks and optical disks include compact disks (CD), laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, in which magnetic disks usually magnetically copy data, and optical disks optically copy data with laser.

[0082] In order to better illustrate the in-situ biological 3D printing robot system in the present application, the following Figure 10 The following specific embodiments are described.

[0083] Embodiment one: a specific process of knee cartilage-bone injury repair by in-situ biological 3D printing robot system.

[0084] Before surgery: first, the knee cartilage area of the patient is scanned by spiral CT, and the scanned data is saved in DICOM format, and the three-dimensional structure space coordinates of the cartilage repair body are obtained based on the scanned data to plan a plurality of corresponding robot-assisted grinding and repair reconstruction paths according to the three-dimensional structure space coordinates; then, a biological 3D printing gun, a puncture system and a grinding head power system are prepared, and a new type of biological active gel ink is injected into the biological 3D printing gun; the lower limbs of the patient are fixed on the corresponding operating bed, and the dynamic reference frame marker balls are set in the knee cartilage area of the patient, and the position information of each marker ball is recorded; the three-dimensional image reconstruction of the operating environment is carried out to obtain the operating environment information, the registration of the robot system and the actual operating environment is completed based on the operating environment information and the position information of each marker ball to obtain the registration information, and the optimal robot-assisted grinding and repair reconstruction path is determined in the plurality of corresponding robot-assisted grinding and repair reconstruction paths based on the registration information and the operating environment information to obtain the optimal surgical planning scheme.

[0085] In the operation process: the doctor drags the robot end to the operation starting point to cooperate with the robot to avoid obstacles according to the optimal operation planning scheme; when the doctor confirms that the robot end moves to the accurate drilling position planned before the operation, the robot stops the automatic motion mode and enters the follow-up motion state; after completing the polishing and drilling operation, the biological 3D printing gun is switched; the doctor observes the display and drags the printing gun according to the optimal robot planning scheme according to the navigation image, and if obstacles or resistance are encountered during the movement of the biological 3D printing gun, the end biological printing system can be moved out, the direction is modified, and then the movement is performed according to the set trajectory; the in-situ biological 3D printing is performed on the defective part, and the laser set on the biological 3D printing gun is used to perform light curing forming on the new biological active gel ink.

[0086] The application also provides an in-situ biological 3D printing robot, comprising: the in-situ biological 3D printing robot system in the above embodiment. The in-situ biological 3D printing robot system comprises:

[0087] The robot body is used for performing optimal robot-assisted grinding and repair reconstruction path planning on the three-dimensional structure space coordinates of the cartilage repair body obtained by constructing the cartilage repair body based on input medical image information before the operation, so as to obtain an optimal operation planning scheme, and performing human-computer cooperation of robot obstacle avoidance trajectory according to the optimal operation planning scheme in the operation process to obtain an optimal robot planning scheme.

[0088] The robot end is connected with the robot body and is used for performing in-situ biological 3D printing according to the optimal robot planning scheme.

[0089] Since the in-situ biological 3D printing robot system in the embodiment can realize all the functions of the in-situ biological 3D printing robot system in the above embodiment, no repeated description is given here.

[0090] In summary, the application provides an in-situ biological 3D printing robot system, a printing method, a medium, a terminal and a robot. The application first performs optimal robot-assisted grinding and repair reconstruction path planning on the three-dimensional structure space coordinates of the cartilage repair body obtained by constructing the cartilage repair body based on input medical image information, so as to obtain an optimal operation planning scheme, and then performs human-computer cooperation of robot obstacle avoidance trajectory according to the optimal operation planning scheme in the operation process to obtain an optimal robot planning scheme. Then, in-situ biological 3D printing is performed according to the optimal robot planning scheme. The robot end is controlled by the robot body to perform in-situ biological 3D printing, which not only has high 3D printing precision, but also has high 3D printing efficiency. Therefore, the application effectively overcomes the various shortcomings in the prior art and has high industrial utilization value.

[0091] The above embodiments are only illustrative of the principles of the present application and its effects, and are not intended to limit the present application. Any modification or change made by any person skilled in the art without departing from the spirit and scope of the present application shall be covered by the claims of the present application.

Claims

1. An in-situ bio 3D printing robot system, characterized in that, The application comprises: a robot body for optimal robot-assisted grinding and repair reconstruction path planning according to the three-dimensional structure spatial coordinates of the cartilage repair body obtained by constructing the cartilage repair body based on input medical image information before surgery, so as to obtain an optimal surgical planning scheme, and for human-robot cooperation in robot obstacle avoidance trajectory according to the optimal surgical planning scheme during surgery to obtain an optimal robot planning scheme; wherein the robot body comprises: an intelligent planning system for constructing the three-dimensional structure spatial coordinates of the cartilage repair body according to input medical image information before surgery based on an intelligent planning model, and for optimal robot-assisted grinding and repair reconstruction path planning according to input surgical area information and the three-dimensional structure spatial coordinates of the cartilage repair body, so as to obtain an optimal surgical planning scheme; wherein the optimal surgical planning scheme comprises: optimal robot-assisted grinding and repair reconstruction path, 3D printing ink volume, 3D printing speed and light curing speed; and wherein the intelligent planning model comprises: a primary planning module for planning a plurality of robot-assisted grinding and repair reconstruction paths according to the three-dimensional structure spatial coordinates of the cartilage repair body obtained by constructing the cartilage repair body based on input medical image information before surgery; an optimization planning module connected to the primary planning module for obtaining corresponding registration information generated based on different surgical environments according to input surgical area information, and for confirming an optimal robot-assisted grinding and repair reconstruction path from the plurality of robot-assisted grinding and repair reconstruction paths obtained by the primary planning module based on the registration information, so as to obtain an optimal surgical planning scheme; an operation cooperation system connected to the intelligent planning system for human-robot cooperation in robot obstacle avoidance trajectory according to the optimal surgical planning scheme based on a reinforcement learning model during surgery to obtain an optimal robot planning scheme; wherein the optimal robot planning scheme comprises: an optimal robot-assisted grinding and repair reconstruction path obtained by human-robot cooperation in robot obstacle avoidance trajectory, the 3D printing ink volume, the 3D printing speed and the light curing speed; a robot end connected to the robot body for in-situ biological 3D printing according to the optimal robot planning scheme.

2. The system of claim 1, wherein, A method for training the intelligent planning model comprises: constructing a sample set based on medical image data, surgical area information and surgical planning scheme point cloud data of patients corresponding to a plurality of successful surgery samples; inputting the sample set into a neural network point cloud segmentation algorithm model to train and obtain the intelligent planning model.

3. The system of claim 2, wherein, The human-robot cooperation in robot obstacle avoidance trajectory according to the optimal surgical planning scheme based on the reinforcement learning model to obtain the optimal robot planning scheme comprises: optimizing the robot-assisted grinding and repair reconstruction path based on position information of each obstacle obtained from a trajectory path generated by a corresponding guiding robot end avoiding each obstacle in the surgical environment to a starting point of the robot-assisted grinding and repair reconstruction path.

4. The system of claim 1, wherein, The robot end comprises: a mechanical arm and an end biological printing system; The mechanical arm is connected with the terminal biological printing system, and is used to drive the terminal biological printing system to perform in-situ biological 3D printing according to the optimal robot planning scheme.

5. The system of claim 4, wherein, The terminal biological printing system further comprises a quick terminal executor switching system, a grinding head power system, a puncture system and a biological 3D printing gun. The quick terminal executor switching system is located on the mechanical arm and is connected with the grinding head power system, the puncture system and the biological 3D printing gun respectively, and is used to switch the grinding head power system, the puncture system and the biological 3D printing gun according to different requirements in the surgical process.

6. The system of claim 5, wherein, The ink used by the biological 3D printing gun for 3D printing is a new type of bioactive gel ink.

7. The system of claim 6, wherein, An ultraviolet generator is arranged on the biological 3D printing gun, and is used to perform light curing forming on the ink used for 3D printing.

8. An in-situ bio 3D printing method, characterized in that, The in-situ biological 3D printing robot system is applied to the in-situ biological 3D printing robot system of any one of claims 1 to 7, and comprises: obtaining medical image information of a patient; performing optimal robot-assisted grinding and repair reconstruction path planning on three-dimensional structure space coordinates of the cartilage repair body obtained by constructing the cartilage repair body according to the input medical image information before surgery, so as to obtain an optimal surgical planning scheme, and performing human-computer cooperation of robot obstacle avoidance trajectory according to the optimal surgical planning scheme in the surgical process to obtain an optimal robot planning scheme; performing in-situ biological 3D printing according to the optimal robot planning scheme.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the method of claim 8.

10. A terminal, characterized by comprising: comprise: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the terminal executes the method of claim 8.

11. An in-situ bio 3D printing robot, characterized in that, comprise: the in-situ biological 3D printing robot system of any one of claims 1 to 7.

Images