Surgical planning method with constrained boundaries and surgical robot control method

Abstract

This application provides a surgical planning method and a surgical robot control method with constrained boundaries. The method includes: acquiring preoperative images of the patient's surgical area; selecting a predetermined number of boundary points from the preoperative images; generating multiple cutting planes using the boundary points; determining the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area; discretizing the inner and outer boundaries into inner boundary point sets and outer boundary point sets, respectively; generating constrained boundaries based on the inner and outer boundary point sets and the boundary points; and using the constrained boundaries as the boundary restrictions for surgical planning. This application is applicable to osteotomy planning of irregular bone surfaces, meets the needs of surgical planning under multimodal imaging, and improves the accuracy and flexibility of surgical planning.

Description

Technical Field

[0001] This application relates to the field of medical surgical robot technology, and in particular to a surgical planning method with constrained boundaries and a surgical robot control method. Background Technology

[0002] With the development of modern science and technology and computer technology, various computer-aided medical technologies and equipment are being used more and more widely in the medical field. Compared with manual operation by doctors, computer-aided surgical robot systems have great advantages in terms of accuracy and comfort. For example, the introduction of osteotomy robot systems in spinal osteotomy can greatly improve surgical efficiency and precision.

[0003] In robotic surgical systems, surgical planning is a crucial step in image-guided surgery. Taking osteotomy in spinal surgery as an example, surgical planning involves designing the osteotomy area based on the patient's affected site. Because the spinal surface is typically curved and irregular, spinal osteotomy presents significant challenges. Currently known osteotomy planning methods involve placing boxes in preoperative 3D images to plan the surgical area. This is achieved by rendering the cube into the image and adjusting its position and size. However, this method cannot account for changes in the curvature of the spine and can only adjust the planned surgical area in three-dimensional space.

[0004] Therefore, existing cube-based surgical planning methods are only suitable for surgical planning on planar or small-radius bone surfaces, and cannot be applied to osteotomy planning in spinal surgery. They can only achieve surgical planning in three-dimensional space, and cannot meet the needs of surgical planning based on multimodal images. Furthermore, the accuracy of surgical planning results is poor, and the surgical planning operation is not flexible enough. Summary of the Invention

[0005] In view of this, embodiments of this application provide a surgical planning method and a surgical robot control method with constrained boundaries to solve the problems of the existing technology having a small range of applicable scenarios, being unable to meet the needs of surgical planning based on different modal images, having poor accuracy of surgical planning results, and not being flexible enough in surgical planning operations.

[0006] A first aspect of this application provides a surgical planning method with constrained boundaries, comprising: acquiring preoperative images of a patient's surgical area; selecting a predetermined number of boundary points from the preoperative images; generating multiple cutting planes using the boundary points; determining the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area; discretizing the inner and outer boundaries into an inner boundary point set and an outer boundary point set, respectively; generating constrained boundaries based on the inner boundary point set, the outer boundary point set, and the boundary points; and using the constrained boundaries as boundary restrictions for surgical planning.

[0007] A second aspect of this application provides a surgical robot control method, comprising: acquiring preoperatively determined surgical planning results and preoperative planning images, wherein the surgical planning results are generated by using boundary points in the preoperative images to create a cutting plane, the cutting plane interacting with the bone structure of the surgical area to form an inner boundary and an outer boundary, the inner boundary and the outer boundary being discretized into point sets respectively, and a constraint boundary generated by using the point sets and boundary points; performing spatial registration to determine the coordinate transformation relationship between the intraoperative images, the patient, and the robotic arm end effector, and fusing the registered intraoperative images with the preoperative planning images as a reference to obtain intraoperative planning images; when the robotic arm end effector is subjected to external force, adjusting the parameters of the robotic arm in the next control cycle according to the force sensor signal and the position of the robotic arm end effector in the constraint space, and adjusting the position of the robotic arm in the next control cycle according to the adjusted parameters, so that the robotic arm end effector always cuts bone with compliant posture parameters within the constraint space, wherein the constraint space is the space determined based on the constraint boundary in the intraoperative planning images.

[0008] A third aspect of this application provides a surgical planning device with constrained boundaries, comprising: an acquisition module configured to acquire preoperative images of a patient's surgical area and select a predetermined number of boundary points from the preoperative images; a discretization module configured to generate multiple cutting planes using the boundary points, determine the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area, and discretize the inner and outer boundaries into an inner boundary point set and an outer boundary point set, respectively; and a generation module configured to generate constrained boundaries based on the inner boundary point set, the outer boundary point set, and the boundary points, and use the constrained boundaries as boundary restrictions for surgical planning.

[0009] A fourth aspect of this application provides a surgical robot control device, comprising: a discrete generation module configured to acquire preoperatively determined surgical planning results and preoperative planning images, wherein the surgical planning results are generated by using boundary points in the preoperative images to generate cutting planes, the cutting planes interacting with the bone structure of the surgical area to form inner and outer boundaries, the inner and outer boundaries being discretized into point sets respectively, and constraint boundaries generated by the point sets and boundary points; a registration and fusion module configured to perform spatial registration, determine the coordinate transformation relationship between the intraoperative images, the patient, and the robotic arm end effector, and fuse the registered intraoperative images with the preoperative planning images as a reference to obtain intraoperative planning images; and an adjustment control module configured to adjust the parameters of the robotic arm in the next control cycle according to the force sensor signal and the position of the robotic arm end effector in the constraint space when the robotic arm end effector is subjected to external force, and adjust the position of the robotic arm in the next control cycle according to the adjusted parameters, so that the robotic arm end effector always cuts bone with compliant posture parameters within the constraint space, wherein the constraint space is the space determined based on the constraint boundaries in the intraoperative planning images.

[0010] A fifth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a surgical planning method with constrained boundaries as described in the first aspect embodiment or a surgical robot control method as described in the second aspect embodiment.

[0011] A sixth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a surgical planning method with constrained boundaries as described in the first aspect embodiment or the steps of a surgical robot control method as described in the second aspect embodiment.

[0012] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0013] By acquiring preoperative images of the patient's surgical area, a predetermined number of boundary points are selected from the images. Multiple cutting planes are generated using these boundary points, and the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area are determined. These inner and outer boundaries are then discretized into inner and outer boundary point sets, respectively. Constraint boundaries are generated based on these inner and outer boundary point sets and the boundary points themselves, and these constraints serve as the boundaries for surgical planning. This application is applicable not only to surgical planning of bone surfaces with small curvature but also to surgical planning scenarios with large curvature bone surfaces. It allows for surgical planning and adjustment of the planning area not only within three-dimensional images but also within two-dimensional images. Therefore, this application significantly expands the surgical scenarios applicable to the surgical planning method, meets the needs of surgical planning based on multimodal images, and its surgical planning method is more flexible, improving the efficiency and accuracy of surgical planning results. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating a surgical planning method with constrained boundaries provided in an embodiment of this application.

[0016] Figure 2 This is a 3D schematic diagram of the spine for the planned osteotomy operation to be performed, provided in an embodiment of this application.

[0017] Figure 3 The embodiments of this application are based on Figure 2 A schematic diagram showing the selection of boundary points for the lesion in the spinal diagram shown;

[0018] Figure 4 This is a schematic diagram illustrating the generation of the cutting plane provided in an embodiment of this application;

[0019] Figure 5 This is a schematic diagram illustrating the generation of constraint points using a cutting plane, provided in an embodiment of this application.

[0020] Figure 6 This is a flowchart illustrating the surgical robot control method provided in an embodiment of this application;

[0021] Figure 7 This is a schematic diagram of the interaction control process between the surgeon and the surgical robot provided in an embodiment of this application;

[0022] Figure 8This is a schematic diagram of the control flow for implementing active constraint control of a robotic arm using an admittance controller, provided in an embodiment of this application.

[0023] Figure 9 This is a schematic diagram of the surgical planning device with constrained boundaries provided in an embodiment of this application;

[0024] Figure 10 This is a schematic diagram of the structure of the surgical robot control device provided in the embodiments of this application;

[0025] Figure 11 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0027] Traditional surgeon-manual surgeries demand a high level of experience and precision. Surgeons experience fatigue during prolonged procedures, and the unstable reaction forces generated during manipulation can cause arm tremors, leading to instrument vibration. Furthermore, maintaining constant force and precise cutting positions is difficult, resulting in reduced accuracy and increased surgical risks. The following section uses spinal surgery as an example to illustrate the problems inherent in current surgeon-manual procedures.

[0028] As is well known, the resection and grinding of structures such as the posterior wall of the spinal canal, lamina, facet joints, and vertebral bodies are the most basic bone tissue manipulation operations in spinal surgery, second only to drilling. These are crucial surgical steps in spinal surgeries such as nerve and spinal canal decompression, vertebral tumor resection, and scoliosis correction. Due to the distribution of important nerves and blood vessels around the spinal structures, spinal surgery carries a certain degree of high risk. Furthermore, during grinding and cutting operations, surgeons primarily use tools such as bone chisels, bone forceps, and grinding drills manually. Control of cutting depth and force relies on experience and feel. In addition, prolonged operation can cause wrist fatigue, leading to instrument tremors, all of which directly affect the surgical outcome and may even damage important structures.

[0029] In addition, regarding the use of surgical instruments, high-speed drills are currently the most commonly used power tools in spinal surgery, offering high cutting efficiency, but potentially causing serious complications due to scraping effects. In recent years, ultrasonic bone scalpels have gradually gained application and promotion in spinal surgery. Ultrasonic bone scalpels exhibit strong tissue selectivity during laminectomy, which can reduce nerve and dura mater damage to some extent, enhancing the safety and effectiveness of bone resection. However, their application also presents several problems, including: lower cutting efficiency; excessive heat generation during prolonged cutting; lack of clear reference markers leading to cutting deviation; difficulties in cutting extensive lesions and severe ligament ossification due to changes in the physiological curvature of the spine and varying thickness of ossifications; and inaccurate control of depth and boundaries during freehand cutting, potentially leading to dural tears, nerve root or spinal cord injury, increasing surgical risks and reducing surgical efficiency.

[0030] The power tools used in the embodiments of this application include, but are not limited to, high-speed power tools, ultrasonic bone scalpels, drills, and other surgical tools. In practical applications, power tools are also referred to as robotic arm end-effectors, end-effectors, cutting tools, powered cutting tools, etc. It should be noted that the type of power tools integrated into the surgical robot system does not constitute a limitation on the technical solution of this application.

[0031] With the development of surgical robots, surgical robot systems have begun to be gradually applied to spinal surgery. By using robots to guide surgeons in interactive procedures such as bone cutting and grinding, surgical efficiency is improved, and compared to manual methods, higher precision and stability are offered, thus greatly increasing the safety and outcome of spinal surgery. However, before using a surgical robot system, surgical planning is first required based on the patient's disease and affected area to determine the surgical scope when using the robot system. Therefore, surgical planning is the first step in image-guided surgical navigation and the use of a surgical robot system. The following section, using spinal surgery as an example, details existing surgical planning methods and their existing problems.

[0032] In the field of osteotomy in spinal surgery, spinal decompression surgery is a common osteotomy procedure. However, due to the typically curved and irregular shape of the spinal surface, osteotomy is quite challenging. For example, in procedures involving grinding or cutting bone around neural structures, especially in spinal neurodecompression surgery, current techniques plan the osteotomy area by placing a cube based on preoperative 3D images. This involves rendering the cube into the 3D image and adjusting its position and size to plan the osteotomy area. However, this surgical planning method cannot consider the curvature of the spine and can only plan and adjust the osteotomy area in 3D space. As a result, existing osteotomy planning methods are only suitable for planning on planar or small-radius bone surfaces, and are not applicable to osteotomy planning in spinal surgery. They can only plan the surgical area in 3D space, failing to meet the needs of surgical planning under multimodal imaging, and the accuracy of the surgical planning results is poor, and the surgical planning operation is not flexible enough.

[0033] In view of this, there is an urgent need to provide a surgical planning solution that can be applied to osteotomy on spinal surfaces with large curvatures, meets the needs of surgical planning based on multimodal imaging, and is flexible and labor-saving, thereby improving the efficiency and accuracy of surgical planning. This would expand the application of surgical robots in spinal osteotomy and enable ultrasonic bone scalpel osteotomy planning operations for different indications. In this way, it would meet the needs of using a surgical robot system to guide the surgeon in interactive operations such as cutting and grinding of the spinal bone surface during spinal decompression surgery.

[0034] It should be noted that the following embodiments of this application illustrate the surgical planning and control process of a surgical robot system integrating an ultrasonic bone scalpel. However, it should be understood that the surgical planning method and surgical robot control method provided in the embodiments of this application are not limited to bone cutting surgical planning and robot control scenarios, nor are they limited to spinal surgery scenarios (such as spinal nerve decompression surgery), and even less so to surgical scenarios using a surgical robot powered by an ultrasonic bone scalpel. Any other surgical operation scenario guided by a surgical robot and any surgical procedure involving bone cutting or drilling are also applicable to this solution, and the application scenarios of the following embodiments do not constitute a limitation on the technical solution of this application.

[0035] The following description uses robot-assisted spinal laminectomy as an example to illustrate the overall structure of the osteotomy robot system involved in the embodiments of this application.

[0036] The osteotomy surgical robot system described in this application, in a real-world scenario, comprises the following components: a main control carriage, a main unit carriage, and a toolbox. The main control carriage may include a main control computer and a binocular camera. The main unit carriage may include a robotic arm, a head tracker, and an end-effector tool (i.e., a power tool). The toolbox may include a patient tracker. In practical applications, the end-effector of the main unit carriage may consist of the following hardware structures: an end-effector joint, an end-effector flange, a six-dimensional force sensor, a power tool, and a head tracker. This application does not limit the specific structure of the osteotomy surgical robot system or the end-effector. Any surgical robot system incorporating a force sensor structure is applicable to the technical solution of this application, and the structure of the aforementioned osteotomy surgical robot system does not constitute a limitation on the technical solution of this application.

[0037] Figure 1 This is a flowchart illustrating a surgical planning method with constrained boundaries provided in an embodiment of this application. Figure 1 Surgical planning methods with constrained boundaries can be executed by computer equipment. For example... Figure 1 As shown, this surgical planning method with constrained boundaries may specifically include:

[0038] S101, acquire preoperative images of the patient's surgical area, and select a predetermined number of boundary points from the preoperative images;

[0039] S102, using boundary points to generate multiple cutting planes, respectively determining the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area, and discretizing the inner and outer boundaries into inner boundary point sets and outer boundary point sets respectively;

[0040] S103 generates constraint boundaries based on the inner boundary point set, outer boundary point set, and boundary points, and uses the constraint boundaries as the boundary restrictions for surgical planning.

[0041] Specifically, the surgical planning method in this application embodiment is based on preoperative images (such as CT or MRI). Of course, in practical applications, this solution can also be used to plan the surgery in intraoperative images (such as CBCT). The images used in the specific planning do not constitute a limitation on the technical solution of this application.

[0042] Furthermore, the boundary points in this application embodiment are selected based on the position of the patient's lesion in the preoperative image. Taking osteophytes (bone hyperplasia) as an example, several boundary points can be selected along the edge of the osteophyte as boundary points of the constraint boundary according to the position of the lesion in the preoperative image. The function of the constraint boundary is to enable the surgical robot to perform the task of cutting bones of irregular shapes.

[0043] In some embodiments, selecting a predetermined number of boundary points from preoperative images includes: selecting a predetermined number of boundary points from tomographic sections of preoperative images based on the location of the patient's lesion in the preoperative images; wherein the tomographic sections include transverse, sagittal, and coronal planes on the preoperative images.

[0044] Specifically, tomography refers to the transverse, sagittal, and coronal planes on preoperative images (CT or MRI). Taking a CT scan of the affected area as an example, the transverse plane of the patient's affected area is first scanned using CT, and then the sagittal and coronal planes are reconstructed based on a 3D reconstruction algorithm (i.e., voxel reconstruction algorithm). The selection of boundary points in this embodiment is an operation based on tomography. The process of selecting boundary points will be described in detail below with reference to the accompanying drawings and specific embodiments. Figure 2 This is a 3D schematic diagram of the spine for the planned osteotomy operation provided in an embodiment of this application. Figure 3 The embodiments of this application are based on Figure 2 This diagram illustrates the selection of boundary points for the lesion in the spinal spine diagram shown; as follows: Figures 2-3 As shown, the boundary point selection operation provided in this application embodiment may include the following:

[0045] Figure 2 The dashed lines in the 3D spinal diagram represent the patient's lesion. Tomographic data was obtained through preoperative CT scans and 3D reconstruction. Eight boundary points were sequentially selected from the tomographic data corresponding to the patient's lesion in the preoperative images, such as... Figure 3 As shown, the eight selected boundary points, from top to bottom, are named as follows: medial cephalic target point, medial cephalic entry point, lateral cephalic target point, lateral cephalic entry point, medial caudal target point, medial caudal entry point, lateral caudal target point, and lateral caudal entry point. In practical applications, these boundary points can be adjusted using points on the corresponding slices. The boundary points can be selected manually by the surgeon on the slices corresponding to the preoperative images, or automatically by the program.

[0046] It should be noted that since a cuboid can be defined using 8 boundary points, even when cutting relatively narrow bone structures, the cuts are made into an approximate cuboid structure. Therefore, under normal circumstances, the cuboid formed by 8 boundary points can cover all osteotomy planning. Thus, this embodiment of the application can select at least 8 boundary points for osteotomy planning. Of course, in practical applications, the number of boundary points is not limited to 8; more boundary points can be selected. For example, when using pentagonal, hexagonal, or cylindrical structures as constraint spaces, more boundary points need to be selected. For instance, the conical constraint in joint replacement can also be based on multi-point planning. This embodiment of the application does not limit the specific number of boundary points; different numbers of boundary points can be selected for surgical planning depending on different osteotomy and surgical scenarios. For ease of understanding and description, this embodiment of the application uses an 8-point constraint surgical planning scenario as an example.

[0047] In some embodiments, the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area are determined, including: forming a cross section by utilizing the interaction between each cutting plane and the bone structure of the surgical area in the three-dimensional image, with each cross section corresponding to an inner and outer boundary; wherein the inner and outer boundaries are respectively arc-shaped bone boundaries located on the bone structure of the surgical area facing the spinal canal and away from the spinal canal.

[0048] Specifically, in selecting such Figure 3 After setting the eight boundary points, the system automatically generates four cutting planes, corresponding to the head cutting plane, tail cutting plane, left cutting plane, and right cutting plane, respectively. These four cutting planes can be fine-tuned by adjusting the eight boundary points. In practical applications, the cutting planes can also be called bone-cutting planes. Figure 4 This is a schematic diagram of the generation of the cutting plane provided in an embodiment of this application, as shown below. Figure 4 As shown, the process of generating the cutting plane provided in this application embodiment may include the following:

[0049] Figure 4 The eight vertices of the cube are: HMIP (medial cephalometric target point), HMOP (medial cephalometric entry point), HLIP (lateral cephalometric target point), HLOP (lateral cephalometric entry point), TMIP (medial caudal cephalometric target point), TMOP (medial caudal cephalometric entry point), TLIP (lateral caudal cephalometric target point), and TLOP (lateral caudal cephalometric entry point). These correspond to four osteotomy planes: the cephalometric cutting plane formed by HLOP-HMOP-HMIP-HLIP, the caudal cutting plane formed by TLOP-TMOP-TMIP-TLIP, the left cutting plane formed by HLOP-TLOP-TLIP-HLIP, and the right cutting plane formed by HMOP-TMOP-TMIP-HMIP.

[0050] Furthermore, after generating four cutting planes using eight boundary points, each cutting plane will generate a series of intersection points when cutting the bone surface to be cut. Each intersection point of the cutting plane and the bone surface will form an arc, which can also be called an arc boundary, boundary curve, or intersection line. The intersection point between each cutting plane and the bone surface can be discretized into outer boundary points and inner boundary points. In practical applications, by interacting each cutting plane with the 3D bone structure of the surgical area to form a cross-section, this cross-section can be divided into an inner boundary curve and an outer boundary curve in 2D. The inner and outer boundary curves correspond to the inner and outer bone boundaries (i.e., the inner and outer boundaries in the above embodiment). By performing random and uniform sampling on the inner and outer boundary curves respectively, it can be discretized into the actual bone cutting entry point cluster (the point set formed by the outer boundary discrete points) and the target point cluster (the point set formed by the inner boundary discrete points).

[0051] In some embodiments, the inner boundary and outer boundary are discretized into an inner boundary point set and an outer boundary point set, respectively, including: randomly and uniformly sampling the inner boundary and outer boundary formed by each cutting plane, so as to collect a number of points from the inner boundary corresponding curve and the outer boundary corresponding curve to form a point set; wherein, the inner boundary point set and the outer boundary point set correspond to the target point cluster and the entry point cluster of the end power tool of the surgical robot's robotic arm, respectively.

[0052] Specifically, after determining the inner and outer boundaries formed by the interaction of each cutting plane with the bone structure, random and uniform sampling is performed along the curves corresponding to the inner and outer boundaries, respectively, to generate the inner and outer boundary point sets. These inner and outer boundary point sets together constitute the constraint points. The process of discretizing the inner and outer boundary curves to obtain constraint points is described in detail below with reference to the accompanying drawings and specific embodiments. Figure 5 This is a schematic diagram illustrating the generation of constraint points using a cutting plane, as provided in an embodiment of this application. Figure 5 As shown, the process of generating constraint points using a cutting plane in this application may include the following:

[0053] Figure 5 The upper dashed line box represents the surface formed by the vertices of the four cutting planes. The four edges within the upper dashed line box correspond to the four cutting planes respectively. The lower arc-shaped dashed line box represents the surface formed by the four arcs created when the four cutting planes cut the bone surface. The points on the surface are the discretized constraint points. Since each cutting plane interacts with the bone structure to form an inner boundary and an outer boundary, the constraint points corresponding to the inner boundary curves are discretized separately. The constraint points corresponding to the inner boundary curves are then grouped into an inner boundary point set (target point cluster), and the constraint points corresponding to the outer boundary curves are grouped into an outer boundary point set (entry point cluster). In practical applications, the corresponding point sets can be formed by collecting 10 points at equal intervals on both the inner and outer boundary curves.

[0054] Furthermore, by using four cutting planes to cut the bone structure, an inner boundary (i.e., an arc-shaped boundary near the spinal canal) and an outer boundary (i.e., an arc-shaped boundary away from the spinal canal) are formed. The outer and inner boundaries are then discretized into an ingress point cluster and a target point cluster, respectively. These ingress point clusters, target point clusters, and selected boundary points are collectively used to define the constraint boundary. This constraint boundary is sent to the robotic system, enabling the robot to complete bone-cutting tasks with irregular shapes. The constraint boundary allows for robot bone-cutting control under curved surface constraints. It should also be noted that specific constraint points can be added to the constraint boundary according to the actual bone-cutting requirements.

[0055] In the embodiments of this application, the inner boundary point set and outer boundary point set in the constraint boundary limit the boundaries that the robot-controlled power tool can break through. Taking spinal nerve decompression surgery as an example, when using a surgical robot to perform grinding or cutting operations on the bone around the nerve structure, since the purpose of grinding and bone cutting operations is nerve decompression, that is, to relieve the pressure of hyperplastic bone or ossified ligaments on the nerve, it is usually necessary to control the power tool to cut or grind to break through the inner cortical bone, but it cannot be allowed to continue cutting down to avoid damaging the dura mater under the cortical bone. Therefore, it is necessary to send the constraint boundary of the bone cutting operation to the robot so that the robot can achieve precise cutting depth control to avoid the power tool breaking through the boundary and damaging the dura mater.

[0056] In some embodiments, the number of boundary points selected in this application is no less than 8, and when the number of boundary points selected is 8, 4 cutting planes can be automatically generated using 8 boundary points. The selection of boundary points and the generation of cutting planes are described in the foregoing embodiments and will not be repeated here.

[0057] According to the technical solution provided in the embodiments of this application, this application provides a curved surface osteotomy planning method based on multiple boundary points. By selecting several boundary points in the 2D tomography corresponding to the preoperative image, multiple osteotomy planes are generated using these boundary points. Each osteotomy plane acts on the bone structure of the surgical area, forming an inner and outer boundary in 2D. The curves corresponding to the inner and outer boundaries are discretized to obtain an inner boundary point set (target point cluster) and an outer boundary point set (entry point cluster). The inner and outer boundary point sets and the boundary points together are used as the definition of the constraint boundary. This application achieves precise definition of the constraint boundary and allows for fine-tuning of the cutting plane by adjusting the boundary points, making the adjustment of the surgical planning area more flexible and simple, improving the efficiency of surgical planning and the accuracy of the planning results. The surgical planning method provided in the embodiments of this application is applicable to all types of osteotomy planning, including but not limited to laminectomy, laminectomy, laminectomy fenestration, vertebral body resection, and bone tumor resection.

[0058] The above embodiments provide a detailed description of the surgical planning method provided in this application. In addition, based on the above surgical planning method, this application also provides a surgical robot control method. The specific content of the surgical robot control method will be described below with reference to specific embodiments. Figure 6 This is a flowchart illustrating the surgical robot control method provided in the embodiments of this application. Figure 6 The control method for surgical robots can be executed by the surgical robot system. For example... Figure 6 As shown, the surgical robot control method may specifically include:

[0059] S601, Obtain the preoperative surgical planning results and preoperative planning images. The surgical planning results are generated by using the boundary points in the preoperative images to create a cutting plane. The cutting plane interacts with the bone structure of the surgical area to form an inner boundary and an outer boundary. The inner boundary and the outer boundary are discretized into point sets respectively. The constraint boundary is generated by using the point sets and the boundary points.

[0060] S602, spatial registration of the surgical robot is performed to determine the coordinate transformation relationship between intraoperative images, the patient and the end effector of the robotic arm. The registered intraoperative images are used as a reference and fused with the preoperative planning images to obtain the intraoperative planning images.

[0061] S603, when the robotic arm end-effector is subjected to an external force, the parameters of the robotic arm in the next control cycle are adjusted according to the force sensor signal and the position of the robotic arm end-effector in the constrained space, and the position of the robotic arm in the next control cycle is adjusted according to the adjusted parameters, so that the robotic arm end-effector always cuts bone with compliant posture parameters within the constrained space, where the constrained space is the space determined by the constrained boundary in the intraoperative planning image.

[0062] Specifically, the surgical robot control method of this application mainly includes three aspects: the first aspect is surgical planning; the second aspect is robot spatial registration and image fusion; and the third aspect is the relevant content of robot interactive control (i.e., human-machine collaborative control process). The first aspect has been described in detail in the preceding embodiments, and therefore will not be repeated in the following embodiments. The following embodiments mainly focus on the second and third aspects, especially the description of the human-machine collaborative control principle and process.

[0063] Furthermore, the surgical planning method provided in this application specifically includes the following:

[0064] In some embodiments, a cutting plane is generated using boundary points in preoperative images. The cutting plane interacts with the bone structure of the surgical area to form an inner boundary and an outer boundary. This includes: selecting a predetermined number of boundary points from the tomography of the preoperative images based on the location of the patient's affected area in the preoperative images; generating multiple cutting planes using the boundary points; and forming a cross-section by interacting each cutting plane with the bone structure of the surgical area in the three-dimensional images. Each cross-section corresponds to at least one inner boundary and an outer boundary in the two-dimensional images. The tomography includes transverse, sagittal, and coronal planes on the preoperative images, and the inner and outer boundaries are respectively arc-shaped bone boundaries located on the bone structure of the surgical area, facing towards and away from the spinal canal.

[0065] Specifically, the selection of boundary points based on 2D tomography, the generation of cutting planes and inner and outer boundaries in surgical planning have been described in detail in the aforementioned embodiments, and will not be repeated here.

[0066] In some embodiments, the inner boundary and outer boundary are discretized into point sets, and the constraint boundary generated by the point sets and boundary points includes: randomly and uniformly sampling the inner boundary and outer boundary formed by each cutting plane, so as to collect a number of points from the curves corresponding to the inner boundary and the curves corresponding to the outer boundary to form an inner boundary point set and an outer boundary point set, and generating a constraint boundary based on the inner boundary point set, the outer boundary point set and the boundary points; wherein, the inner boundary point set and the outer boundary point set correspond to the target point cluster and the entry point cluster of the end power tool of the surgical robot's robotic arm, respectively.

[0067] Specifically, the operation of uniformly sampling the inner and outer boundaries to generate discrete points (constraint points) in the surgical planning, and how to define constraint boundaries based on the inner and outer boundary point sets and boundary points, can be found in the aforementioned embodiments and will not be repeated here.

[0068] Based on the surgical planning method provided in the foregoing embodiments, the following detailed description of the robot's spatial registration, image fusion, and robot interactive control will be provided in conjunction with specific embodiments.

[0069] In some embodiments, spatial registration is performed to determine the coordinate transformation relationship between the intraoperative image, the patient, and the robotic arm end effector, including: scanning a scale plate installed at the end effector of the robotic arm using an imaging device to obtain the coordinates of a marker on the scale plate in the intraoperative image, and obtaining the coordinates of the marker in the coordinate system of the robotic arm end effector; establishing a first transformation matrix between the intraoperative image coordinate system and the robotic arm end effector coordinate system based on the coordinates of the marker in the intraoperative image and the coordinates of the marker in the coordinate system of the robotic arm end effector; establishing a third transformation matrix between the intraoperative image coordinate system and the patient coordinate system based on the first transformation matrix and a second transformation matrix between the robotic arm end effector coordinate system and the patient coordinate system, and using the first transformation matrix, the second transformation matrix, and the third transformation matrix as the result of spatial registration.

[0070] Specifically, when using a surgical robot system for bone cutting during surgery, the surgical robot is first spatially registered to establish the coordinate transformation relationship between the physical space of the surgical robot, the patient space, and the imaging space. In practical applications, since the patient space is located through the imaging space, there is a corresponding relationship between the patient space and the imaging space. Therefore, the spatial registration of the surgical robot can also be understood as establishing the coordinate transformation relationship between the physical space of the surgical robot and the imaging space, that is, establishing the transformation matrix between the intraoperative imaging coordinate system and the coordinate system of the robotic arm end effector.

[0071] Furthermore, the spatial registration method used in this application embodiment is point-to-point registration, that is, using imaging equipment to scan a scale plate installed on the tracer at the end of the robotic arm, and achieving registration by using the imaging positions of markers (such as steel balls) on the scale plate within the intraoperative image. However, this application is not limited to point-to-point registration in practical applications; other methods that can achieve spatial registration are also applicable to this application, such as CBCT-CT fusion registration and structured light-based CT point cloud registration. Among them, CBCT-CT fusion registration can also be considered a point-to-point registration method. CBCT-CT fusion is performed automatically through the ICP algorithm and can be manually adjusted. Structured light-based CT point cloud registration generates a point cloud by scanning the surface structure behind the vertebral body with a structured light scanning device, and then performs point cloud matching based on the ICP algorithm. This registration method is suitable for open surgery that can be planned based on preoperative CT.

[0072] In this embodiment of the application, the point-to-point registration method used in this application may include the following steps:

[0073] The first step is to rigidly fix the scale plate to the end effector of the robotic arm. The scale plate has multiple markers. During the operation, the scale plate is 3D scanned using imaging equipment (such as a C-arm) to obtain real-time images and determine the coordinates of each marker in the real-time images (such as CBCT real-time images).

[0074] The second step is to map the coordinates of the markers on the scale plate relative to the end of the robotic arm, since the coordinates of the markers in the real-time intraoperative image are known, and the coordinates of the markers relative to the end of the robotic arm are mapped to achieve point-to-point mapping. Based on the coordinate mapping relationship of the markers, the first transformation matrix between the image coordinate system and the end-of-arm coordinate system can be established, thereby achieving registration between the robot's physical space and the C-arm image space.

[0075] The third step is to establish the first transformation matrix. Since the physical space of the patient is the same as the physical space of the robot, the coordinates of the marker in the patient coordinate system can be determined. Since the coordinates of the marker in the real-time intraoperative image have been determined when the first transformation matrix is ​​established, the second transformation matrix between the patient coordinate system and the intraoperative image coordinate system can be directly established based on the above coordinates.

[0076] The fourth step involves determining the first and second transformation matrices, followed by a homogeneous transformation matrix to directly obtain the third transformation matrix between the robotic arm end-effector coordinate system and the patient coordinate system. Finally, the first, second, and third transformation matrices are used together as the result of spatial registration.

[0077] In some embodiments, the registered intraoperative image is used as a reference and fused with the preoperative planning image to obtain the intraoperative planning image, including: using the registered intraoperative image as a reference, the preoperative planning image is superimposed on the registered intraoperative image using preset bony landmarks, so as to map the preoperative surgical planning result to the intraoperative image to obtain the intraoperative planning image.

[0078] Specifically, after achieving spatial registration of the robot, the spatial resolution of intraoperative CBCT images is too low to meet the requirements of fine osteotomy planning. Therefore, it is necessary to fuse the preoperative planning images (i.e., preoperative CT planning images) with the intraoperative CBCT images to map the constraint boundaries in the preoperative planning images to the intraoperative CBCT images.

[0079] Furthermore, during image fusion, the intraoperatively registered CBCT image is used as the background (reference), and the preoperative planning image is used as a floating (moving image). Based on the position of bony markers in the image, the preoperative planning image is moved into the intraoperative CBCT image, causing them to overlap. This achieves fusion between the preoperative planning image and the intraoperative CBCT image, mapping the preoperative surgical planning results to the intraoperative CBCT image (i.e., mapping the osteotomy area to the CBCT), thus obtaining the intraoperative planning image. It should be noted that bony markers can be considered as feature points pre-marked on the bone surface. When the positions of feature points in the preoperative planning image and the feature points in the intraoperative CBCT image coincide, fusion between the two images can be achieved.

[0080] The final stage of the surgical robot control method provided in this application embodiment is robot interaction control. The robot interaction control method in this application embodiment adopts a robotic arm active constraint safety control strategy based on admittance control. This strategy is based on the kinematic and dynamic model of the robotic arm and controls the posture of the robotic arm in real time through active compliant control technology and robotic arm kinematics.

[0081] Before introducing the implementation principle and method of active constraint control of robotic arms based on admittance control, the interactive control process between the surgeon and the surgical robot provided in the embodiments of this application will be described in detail. Figure 7 This is a schematic diagram of the interaction control process between the surgeon and the surgical robot provided in an embodiment of this application. Figure 7 As shown, the interactive control process between the surgeon and the surgical robot may include the following:

[0082] Installation and calibration of the power tool; identification of six-dimensional force sensor parameters and real-time compensation of external force; load correction at the end of the robotic arm; acquisition of the constraint area generated during intraoperative planning; initial automatic positioning of the robotic arm; human-machine interaction based on active compliant control; determination of whether the robotic arm posture has deviated; determination of whether the power tool has entered the constraint area; control of the enabled or disabled power system; determination of whether the tip of the power tool has exceeded the constraint boundary; power system information feedback and visualization of the power tool position; finally, determination of whether the osteotomy operation has been completed.

[0083] The calibration of the power tool includes installing the power tool at the end of the robotic arm and then using the surgical robot navigation system to calibrate the power relationship of the power tool at the end of the robotic arm and the position of the power tool tip.

[0084] In practical applications, human-computer interaction based on active compliance control is implemented using an active constraint safety control strategy for robotic arms based on admittance control. For details on the active constraint safety control strategy for robotic arms based on admittance control, please refer to the examples below.

[0085] Furthermore, during the active constraint control of the robotic arm, it is determined whether the robotic arm's pose has deviated (by judging whether the tip of the power tool deviates from the trajectory of the constraint area). When the robotic arm's pose deviates, the pose is adjusted through constraint area tracking, so that the adjusted robotic arm drives the tip of the power tool back to the correct trajectory. When it is determined that the robotic arm's pose has not deviated, that is, when the tip of the power tool moves along the trajectory of the constraint area, it is further determined whether the tip of the power tool has entered the constraint area.

[0086] Furthermore, once it is determined that the tip of the power tool has entered the constraint area, the enabling power system is used to control the power tool to perform cutting or grinding. If the tip of the power tool has not yet entered the constraint area, the disabling power system is used to shut down the power tool, causing the power tool to temporarily stop working.

[0087] Furthermore, when using a robotic arm to control a power tool to cut or grind along the constrained area, it is determined in real time whether the tip of the power tool exceeds the constraint boundary (i.e., the constraint boundary generated by the 8-point constraint in the aforementioned embodiment). When it is determined that the tip of the power tool exceeds the constraint boundary, the power tool is shut down using the disabled power system, and the position of the robotic arm is readjusted so that the adjusted robotic arm can drive the tip of the power tool back to the position of the constraint boundary, restore the output of the power system, and allow the tip of the robotic arm to continue cutting or grinding along the constrained area.

[0088] Furthermore, the power system information is transmitted back, and the main control equipment displays the location of the power tool visually based on the power system information, which includes the status information of the power tool.

[0089] Based on the surgeon-surgical robot interaction control process provided in the above embodiments, the principle and method of robot interaction control based on admittance control will be specifically explained below with reference to specific embodiments.

[0090] In some embodiments, adjusting the parameters of the robotic arm in the next control cycle based on the force sensor signal and the position of the robotic arm end-effector within the constrained space includes: when the robotic arm end-effector is subjected to an external force, acquiring the force sensor signal generated by the six-dimensional force sensor, and adjusting at least one parameter of the inertial characteristics, damping characteristics, and stiffness characteristics of the robotic arm in the next control cycle based on the current position of the robotic arm end-effector within the constrained space using a second-order spring-damped model formed by modeling the robotic arm; wherein the force sensor signal includes the external force and external torque generated by the robotic arm end-effector when subjected to an external force.

[0091] Specifically, the active constraint control algorithm of this application can be integrated into the admittance controller of the main control device, which can also be considered a virtual control module. The principle and process of realizing active constraint control of the robotic arm using the admittance controller are explained below with reference to the accompanying drawings. Figure 8 This is a schematic diagram of the control flow for active constraint control of a robotic arm using an admittance controller, provided in an embodiment of this application. Figure 8 As shown, the process of using an admittance controller to achieve active constraint control of a robotic arm may include the following:

[0092] The force sensor signal generated when the robotic arm's end effector is dragged is sent to the admittance controller, and the robotic arm also sends the real-time position of the end effector within the constrained space to the admittance controller via a position sensor. Based on the acquired external force, external torque, and the position information of the end effector, the admittance controller uses a second-order spring-damped model to assess the inertial characteristics of the robotic arm in the next control cycle. M Damping characteristics Band stiffness characteristics K By adjusting at least one parameter in the control system, compliant and rigid control of the robotic arm can be achieved. For example, when the end effector of the robotic arm is within a constrained space, the damping characteristics of the robotic arm can be reduced. B and stiffness characteristics K This allows the robotic arm to maintain a large degree of freedom, thereby maximizing the smoothness of its movement within the constrained space. Furthermore, when the end effector is at the edge of the constrained space, the robotic arm's damping characteristics can be improved. B and stiffness characteristics K And reduce the inertial characteristics of the robotic arm. M This allows the robotic arm to maintain sufficient "rigidity" support, preventing the power tool from continuing to cut downwards.

[0093] In some embodiments, the second-order spring-damped model is represented as:

[0094]

[0095] in, This represents the external force acting on the robotic arm. Indicates external torque. M This indicates the inertial characteristics of the robotic arm. B This indicates the damping characteristics of the robotic arm. K This indicates the stiffness characteristics of the robotic arm. The Cartesian acceleration of the power tool at the end effector of the robotic arm. This indicates the speed of the power tool at the end of the robotic arm. x This indicates the displacement of the power tool at the end of the robotic arm. J This represents the moment of inertia of the robotic arm. This represents the Cartesian angular acceleration at the end of the robotic arm. This represents the angular velocity at the end of the robotic arm. This represents the angular displacement at the end of the robotic arm.

[0096] Specifically, during surgery, when the surgeon moves the power tool at the end of the robotic arm, a six-dimensional force sensor installed at the end of the robotic arm generates a corresponding force signal (i.e., a force sensor signal). This force signal includes the external force acting on the power tool and the resulting external torque. This embodiment models the robotic arm as a second-order spring-damped model and utilizes an active constraint control algorithm based on admittance control to achieve pose control of the robotic arm. This ensures that the power tool maintains "compliant" control within the constraint space and "rigid" control when the power tool is at the edge of the constraint space or may exceed the constraint space. The implementation process of the active constraint control of the robotic arm is described in detail below based on the above second-order spring-damped model.

[0097] In some embodiments, adjusting the position of the robotic arm in the next control cycle according to the adjusted parameters includes: using a second-order spring damping model to solve for the desired position or position change of the robotic arm end-effector in the next control cycle based on the adjusted parameters; sending the desired position or position change to the robotic arm controller, which adjusts the pose of the robotic arm according to the desired position or position change, so that the robotic arm automatically guides the robotic arm end-effector to move or fix in the desired position.

[0098] Specifically, after adjusting the parameters (inertia characteristics, damping characteristics, and / or stiffness characteristics) of the robotic arm in the next control cycle, based on the adjusted parameters, a second-order spring-damped model is used to solve for the desired position or position change of the robotic arm's end effector in the next control cycle, that is, using the adjusted inertia characteristics. M Damping characteristics B and / or stiffness characteristics K The second-order spring damping model described above is used again to solve the problem sequentially. x , and ,in, The first derivative of x ， Let x be the second derivative. After recalculating the velocity, acceleration, and displacement of the end effector of the robotic arm, the desired position or position change of the end effector tool in the next control cycle can be determined.

[0099] Furthermore, the calculated desired position or position change is sent to the robotic arm controller. The robotic arm controller automatically adjusts the position and attitude of the robotic arm based on the desired position or position change that the robotic arm end-effector is to reach in the next control cycle. This allows the robotic arm to automatically guide the robotic arm end-effector to move to the desired position according to the adjusted position and attitude. When the desired position of the robotic arm end-effector in the next control cycle is the same as the position in the previous control cycle, the robotic arm will fix the end-effector at the desired position to ensure that the robotic arm end-effector remains within the constrained space during the surgery.

[0100] This application's embodiment employs an active constraint control strategy for robotic arms based on admittance control. This strategy ensures that when the robotic arm's end effector is within the constraint space, it provides sufficient degrees of freedom while maintaining operational stability and accuracy, thereby maximizing the smoothness of dragging the robotic arm within the constraint space. When the end effector reaches a certain constraint depth (e.g., at the edge of the constraint space), the robotic arm's parameters are adjusted to provide sufficient rigidity, ensuring that the surgical tool (i.e., the power tool) does not come into contact with or collide with the critical area surrounding the surgical site. This application's embodiment achieves both compliant and rigid human-machine collaborative control, thereby improving the smoothness of switching surgical tools between different constraint areas and enhancing surgical safety and the user experience.

[0101] This application embodiment achieves compliant control by responding to the external force applied to the end effector of the robotic arm by the surgeon during the operation. Simultaneously, this application embodiment monitors the position of the robotic arm's end-effector in real time. Based on the position of the end-effector within the constrained space, the admittance controller adjusts the parameters to ensure the safety, stability, and compliance of the robotic arm during the surgery, maximizing the surgeon's experience. Furthermore, when the end-effector is at the boundary of the constrained space, the admittance controller ensures sufficient stability and rigidity of the robotic arm, guaranteeing that the end-effector does not exceed the safe zone, thus ensuring the safe and stable conduct of the surgery.

[0102] According to the technical solutions provided in the embodiments of this application, the embodiments of this application have at least the following advantages:

[0103] 1) This application uses an irregular constraint method based on 8 boundary points and discrete points, which can realize the surgical planning of osteotomy surgery with curved or arc-shaped bone surfaces. The adjustment of the planning area is flexible and simple, which makes it easy for the surgeon to clearly define the bone cutting boundary on 2D tomography, thus improving the application scenarios and scope of the surgical planning method.

[0104] 2) The active constraint definition and real-time control method based on boundary points and custom sampling points in this application is applicable to various regular and irregular bone-cutting boundary methods, including boundary contours such as cylinders, cones, rectangles, and curves. Because the constraint sampling points can be customized, it can reduce computational resources while ensuring safety and guaranteeing the reliability of real-time constraints.

[0105] 3) This application leverages the high precision of the robot and the advantage of the ultrasonic bone scalpel's ability to handle hard surfaces without compromising softness. Utilizing an active constraint control algorithm based on admittance control and a second-order spring damping model, it provides sufficient degrees of freedom in the upper space within the constraint space. When the surgical tool reaches a certain constraint depth, it provides sufficient rigid support to prevent further cutting, achieving a compliant yet rigid human-machine collaborative control. This makes high-risk osteotomy safer, especially promising for applications in spinal surgery. Furthermore, the system architecture of this application, based on multimodal imaging and flexible intraoperative registration methods (including point-to-point registration), makes the overall robot-assisted surgical process more reliable.

[0106] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0107] Figure 9 This is a schematic diagram of the surgical planning device with constrained boundaries provided in an embodiment of this application. Figure 9 As shown, the surgical planning device with constrained boundaries includes:

[0108] The acquisition module 901 is configured to acquire preoperative images of the patient's surgical area and select a predetermined number of boundary points from the preoperative images.

[0109] Discretization module 902 is configured to generate multiple cutting planes using boundary points, determine the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area, and discretize the inner and outer boundaries into inner boundary point sets and outer boundary point sets, respectively.

[0110] The generation module 903 is configured to generate constraint boundaries based on the inner boundary point set, the outer boundary point set, and the boundary points, and use the constraint boundaries as the boundary restrictions for surgical planning.

[0111] In some embodiments, Figure 9 The acquisition module 901 selects a predetermined number of boundary points from the tomography of the preoperative image based on the location of the patient's lesion in the preoperative image; wherein the tomography includes the transverse, sagittal and coronal planes on the preoperative image.

[0112] In some embodiments, Figure 9 The discrete module 902 uses the interaction between each cutting plane and the bone structure of the surgical area in the three-dimensional image to form a cross section. Each cross section corresponds to an inner boundary and an outer boundary. The inner boundary and the outer boundary are respectively the arc-shaped bone boundaries on the bone structure of the surgical area that face the spinal canal and face away from the spinal canal.

[0113] In some embodiments, Figure 9The discrete module 902 randomly and uniformly samples the inner and outer boundaries formed by each cutting plane so as to collect a number of points from the curves corresponding to the inner and outer boundaries to form a point set; wherein, the inner boundary point set and the outer boundary point set correspond to the target point cluster and the entry point cluster of the end power tool of the surgical robot's robotic arm, respectively.

[0114] Figure 10 This is a schematic diagram of the surgical robot control device provided in an embodiment of this application. Figure 10 As shown, the surgical robot control device includes:

[0115] The discrete generation module 1001 is configured to obtain the preoperative surgical planning results and preoperative planning images. The surgical planning results are generated by using the boundary points in the preoperative images to generate a cutting plane. The cutting plane interacts with the bone structure of the surgical area to form an inner boundary and an outer boundary. The inner boundary and the outer boundary are discretized into point sets respectively. The constraint boundary is generated by using the point sets and the boundary points.

[0116] The registration and fusion module 1002 is configured to perform spatial registration, determine the coordinate transformation relationship between intraoperative images, patients and robotic arm ends, and fuse the registered intraoperative images with the preoperative planning images as a reference to obtain intraoperative planning images.

[0117] The adjustment control module 1003 is configured to adjust the parameters of the robotic arm in the next control cycle based on the force sensor signal and the position of the robotic arm end-effector within the constraint space when the robotic arm end-effector is subjected to an external force, and to adjust the position of the robotic arm in the next control cycle based on the adjusted parameters, so that the robotic arm end-effector always performs bone cutting with compliant posture parameters within the constraint space, wherein the constraint space is the space determined based on the constraint boundary within the intraoperative planning image.

[0118] In some embodiments, Figure 10 The discrete generation module 1001 selects a predetermined number of boundary points from the tomography of the preoperative image based on the location of the patient's affected area in the preoperative image; it generates multiple cutting planes using the boundary points, and uses the interaction between each cutting plane and the bone structure of the surgical area in the three-dimensional image to form a cross section, with each cross section corresponding to an inner boundary and an outer boundary; wherein, the tomography includes the transverse, sagittal and coronal planes on the preoperative image, and the inner boundary and outer boundary are respectively the arc-shaped bone boundaries on the bone structure of the surgical area facing the spinal canal and away from the spinal canal.

[0119] In some embodiments, Figure 10The discrete generation module 1001 randomly and uniformly samples the inner and outer boundaries formed by each cutting plane, so as to collect a number of points from the curves corresponding to the inner and outer boundaries to form an inner boundary point set and an outer boundary point set, and generate a constraint boundary based on the inner boundary point set, the outer boundary point set, and the boundary points; wherein, the inner boundary point set and the outer boundary point set correspond to the target point cluster and the entry point cluster of the end power tool of the surgical robot's robotic arm, respectively.

[0120] In some embodiments, Figure 10 The registration and fusion module 1002 uses imaging equipment to scan the scale plate installed at the end of the robotic arm to obtain the coordinates of the marker on the scale plate in the intraoperative image and the coordinates of the marker in the coordinate system of the robotic arm end. Based on the coordinates of the marker in the intraoperative image and the coordinates of the marker in the coordinate system of the robotic arm end, a first transformation matrix is ​​established between the intraoperative image coordinate system and the coordinate system of the robotic arm end. Based on the first transformation matrix and the second transformation matrix between the coordinate system of the robotic arm end and the patient coordinate system, a third transformation matrix is ​​established between the intraoperative image coordinate system and the patient coordinate system. The first transformation matrix, the second transformation matrix and the third transformation matrix are used as the result of spatial registration.

[0121] In some embodiments, Figure 10 The registration and fusion module 1002 uses the registered intraoperative image as a reference and superimposes the preoperative planning image onto the registered intraoperative image using preset bony markers, so as to map the preoperative surgical planning result to the intraoperative image and obtain the intraoperative planning image.

[0122] In some embodiments, Figure 10 When the end-effector of the robotic arm is subjected to an external force, the adjustment control module 1003 acquires the force sensor signal generated by the six-dimensional force sensor. Based on the current position of the end-effector in the constrained space, it uses a second-order spring-damped model formed by modeling the robotic arm to adjust at least one parameter of the inertial characteristics, damping characteristics, and stiffness characteristics of the robotic arm in the next control cycle. The force sensor signal includes the external force and external torque generated by the end-effector when subjected to an external force.

[0123] In some embodiments, Figure 10 The adjustment control module 1003 represents the second-order spring damping model as follows:

[0124]

[0125] in, This represents the external force acting on the robotic arm. Indicates external torque. M This indicates the inertial characteristics of the robotic arm. B This indicates the damping characteristics of the robotic arm.K This indicates the stiffness characteristics of the robotic arm. The Cartesian acceleration of the power tool at the end effector of the robotic arm. This indicates the speed of the power tool at the end of the robotic arm. x This indicates the displacement of the power tool at the end of the robotic arm. J This represents the moment of inertia of the robotic arm. This represents the Cartesian angular acceleration at the end of the robotic arm. This represents the angular velocity at the end of the robotic arm. This represents the angular displacement at the end of the robotic arm.

[0126] In some embodiments, Figure 10 The adjustment control module 1003 uses a second-order spring damping model to solve for the desired position or position change of the end effector tool in the next control cycle based on the adjusted parameters. The desired position or position change is sent to the robot arm controller, which adjusts the robot arm's pose according to the desired position or position change, so that the robot arm automatically guides the end effector tool to move or fix in the desired position.

[0127] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0128] Figure 11 This is a schematic diagram of the structure of the electronic device 11 provided in an embodiment of this application. Figure 11 As shown, the electronic device 11 of this embodiment includes: a processor 1101, a memory 1102, and a computer program 1103 stored in the memory 1102 and executable on the processor 1101. When the processor 1101 executes the computer program 1103, it implements the steps in the various method embodiments described above. Alternatively, when the processor 1101 executes the computer program 1103, it implements the functions of each module / unit in the various device embodiments described above.

[0129] For example, computer program 1103 may be divided into one or more modules / units, which are stored in memory 1102 and executed by processor 1101 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 1103 in electronic device 11.

[0130] Electronic device 11 may be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 11 may include, but is not limited to, processor 1101 and memory 1102. Those skilled in the art will understand that... Figure 11 This is merely an example of electronic device 11 and does not constitute a limitation on electronic device 11. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0131] The processor 1101 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0132] The memory 1102 can be an internal storage unit of the electronic device 11, such as a hard disk or RAM of the electronic device 11. The memory 1102 can also be an external storage device of the electronic device 11, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, FlashCard, etc., equipped on the electronic device 11. Furthermore, the memory 1102 can include both internal and external storage units of the electronic device 11. The memory 1102 is used to store computer programs and other programs and data required by the electronic device. The memory 1102 can also be used to temporarily store data that has been output or will be output.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0134] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0135] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0139] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0140] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A surgical planning method with constrained boundaries, characterized in that, include: Acquire preoperative images of the patient's surgical area, and select a predetermined number of boundary points from the preoperative images; Multiple cutting planes are generated using the boundary points. The inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area are determined. The inner and outer boundaries are discretized into an inner boundary point set and an outer boundary point set, respectively. Based on the inner boundary point set, the outer boundary point set, and the boundary points, a constraint boundary is generated, and the constraint boundary is used as the boundary limit for surgical planning. The step of determining the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area includes: Each cutting plane is used to form a cross section by interacting with the bone structure of the surgical area in the three-dimensional image, and each cross section corresponds to an inner boundary and an outer boundary; The inner boundary and the outer boundary are respectively the arc-shaped bone boundaries on the bone structure of the surgical area, facing towards and away from the spinal canal.

2. The method according to claim 1, characterized in that, The step of selecting a predetermined number of boundary points from the preoperative images includes: Based on the location of the patient's affected area in the preoperative image, a predetermined number of boundary points are selected from the tomography of the preoperative image. The tomography includes the transverse, sagittal, and coronal sections on the preoperative images.

3. The method according to claim 1, characterized in that, Discretizing the inner boundary and the outer boundary into an inner boundary point set and an outer boundary point set, respectively, includes: Random and uniform sampling is performed on the inner and outer boundaries formed by each cutting plane so as to collect a number of points from the curves corresponding to the inner and outer boundaries to form a point set; The inner boundary point set and the outer boundary point set correspond to the target point cluster and entry point cluster of the end effector of the surgical robot's robotic arm, respectively.

4. The method according to any one of claims 1-3, characterized in that, The number of boundary points is no less than 8, and when the number of boundary points is 8, 4 cutting planes are automatically generated using the boundary points.

5. A surgical planning device with constrained boundaries, characterized in that, include: The acquisition module is configured to acquire preoperative images of the patient's surgical area and select a predetermined number of boundary points from the preoperative images. The discrete module is configured to generate multiple cutting planes using the boundary points, determine the inner and outer boundaries formed by the interaction between each cutting plane and the bone structure of the surgical area, and discretize the inner and outer boundaries into an inner boundary point set and an outer boundary point set, respectively. The generation module is configured to generate constraint boundaries based on the inner boundary point set, the outer boundary point set, and the boundary points, and use the constraint boundaries as boundary restrictions for surgical planning. The discrete module utilizes the interaction between each cutting plane and the bone structure of the surgical area in the three-dimensional image to form a cross-section, and each cross-section corresponds to an inner boundary and an outer boundary; wherein the inner boundary and the outer boundary are respectively arc-shaped bone boundaries located on the bone structure of the surgical area facing towards the spinal canal and away from the spinal canal.

6. A surgical robot control device, characterized in that, include: A discrete generation module is configured to acquire the preoperative surgical planning results and preoperative planning images, wherein the surgical planning results are generated by using boundary points in the preoperative images to generate a cutting plane, the cutting plane interacts with the bone structure of the surgical area to form an inner boundary and an outer boundary, the inner boundary and the outer boundary are discretized into point sets respectively, and a constraint boundary is generated by using the point sets and boundary points. The registration and fusion module is configured to perform spatial registration, determine the coordinate transformation relationship between intraoperative images, patients and robotic arm ends, and fuse the registered intraoperative images with the preoperative planning images as a reference to obtain intraoperative planning images. The adjustment control module is configured to adjust the parameters of the robotic arm in the next control cycle based on the force sensor signal and the position of the robotic arm end-effector within the constraint space when the robotic arm end-effector is subjected to an external force, and to adjust the position of the robotic arm in the next control cycle based on the adjusted parameters, so that the robotic arm end-effector always performs bone cutting with compliant pose parameters within the constraint space, wherein the constraint space is the space determined based on the constraint boundary within the intraoperative planning image; The surgical robot control method based on the surgical robot control device includes: Obtain the preoperative surgical planning results and preoperative planning images, wherein the surgical planning results are generated by using boundary points in the preoperative images to generate cutting planes, the cutting planes interact with the bone structure of the surgical area to form inner and outer boundaries, the inner and outer boundaries are discretized into point sets respectively, and the constraint boundaries are generated by using the point sets and boundary points; Spatial registration is performed to determine the coordinate transformation relationship between the intraoperative image, the patient, and the end effector of the robotic arm. The registered intraoperative image is then fused with the preoperative planning image to obtain the intraoperative planning image. When the robotic arm end-effector is subjected to an external force, the parameters of the robotic arm in the next control cycle are adjusted according to the force sensor signal and the position of the robotic arm end-effector within the constraint space. The position of the robotic arm in the next control cycle is also adjusted according to the adjusted parameters, so that the robotic arm end-effector always performs bone cutting with compliant posture parameters within the constraint space, wherein the constraint space is the space determined based on the constraint boundary in the intraoperative planning image. The method of generating a cutting plane using boundary points in preoperative images, wherein the cutting plane interacts with the bone structure of the surgical area to form inner and outer boundaries, includes: Based on the location of the patient's affected area in the preoperative image, a predetermined number of boundary points are selected from the tomography of the preoperative image. Multiple cutting planes are generated using the boundary points, and each cutting plane interacts with the bone structure of the surgical area in the three-dimensional image to form a cross section, with each cross section corresponding to an inner boundary and an outer boundary; The tomography includes the transverse, sagittal, and coronal planes on the preoperative images, and the inner and outer boundaries are respectively the arc-shaped bone boundaries on the bone structure of the surgical area, facing towards and away from the spinal canal.

7. The apparatus according to claim 6, characterized in that, The step of discretizing the inner and outer boundaries into point sets, and using the point sets and boundary points to generate constraint boundaries, includes: Random and uniform sampling is performed on the inner and outer boundaries formed by each cutting plane so as to collect a number of points from the inner boundary corresponding curve and the outer boundary corresponding curve to form an inner boundary point set and an outer boundary point set, and the constraint boundary is generated based on the inner boundary point set, the outer boundary point set and the boundary points. The inner boundary point set and the outer boundary point set correspond to the target point cluster and entry point cluster of the end effector of the surgical robot's robotic arm, respectively.

8. The apparatus according to claim 6, characterized in that, The spatial registration process, which determines the coordinate transformation relationship between intraoperative images, the patient, and the robotic arm's end effector, includes: The scale plate installed at the end of the robotic arm is scanned using imaging equipment to obtain the coordinates of the marker on the scale plate in the intraoperative image, and the coordinates of the marker in the coordinate system of the end of the robotic arm are obtained. Based on the coordinates of the marker in the intraoperative image and the coordinates of the marker in the coordinate system of the robotic arm end effector, a first transformation matrix is ​​established between the intraoperative image coordinate system and the coordinate system of the robotic arm end effector. Based on the first transformation matrix and the second transformation matrix between the robotic arm end-effector coordinate system and the patient coordinate system, a third transformation matrix between the intraoperative image coordinate system and the patient coordinate system is established, and the first transformation matrix, the second transformation matrix, and the third transformation matrix are used as the result of spatial registration.

9. The apparatus according to claim 6, characterized in that, The process of fusing the registered intraoperative images as a reference with the preoperative planning images to obtain intraoperative planning images includes: Using the registered intraoperative images as a reference, the preoperative planning images are superimposed onto the registered intraoperative images using preset bony landmarks, so as to map the preoperative surgical planning results to the intraoperative images and obtain the intraoperative planning images.

10. The apparatus according to claim 6, characterized in that, The step of adjusting the parameters of the robotic arm in the next control cycle based on the force sensor signal and the position of the end effector within the constrained space includes: When the end effector of the robotic arm is subjected to an external force, the force sensor signal generated by the six-dimensional force sensor is acquired. Based on the current position of the end effector of the robotic arm in the constrained space, at least one parameter of the inertial characteristics, damping characteristics, and stiffness characteristics of the robotic arm in the next control cycle is adjusted using a second-order spring-damped model formed by modeling the robotic arm. The force sensor signal includes the external force and external torque generated by the end effector of the robotic arm when subjected to an external force.

11. The apparatus according to claim 10, characterized in that, The second-order spring damping model is expressed as follows: in, This represents the external force acting on the robotic arm. Indicates external torque. M This indicates the inertial characteristics of the robotic arm. B This indicates the damping characteristics of the robotic arm. K This indicates the stiffness characteristics of the robotic arm. The Cartesian acceleration of the power tool at the end effector of the robotic arm. This indicates the speed of the power tool at the end of the robotic arm. x This indicates the displacement of the power tool at the end of the robotic arm. J This represents the moment of inertia of the robotic arm. This represents the Cartesian angular acceleration at the end of the robotic arm. This represents the angular velocity at the end of the robotic arm. This represents the angular displacement at the end of the robotic arm.

12. The apparatus according to claim 10, characterized in that, The step of adjusting the position of the robotic arm in the next control cycle according to the adjusted parameters includes: Based on the adjusted parameters, the desired position or position change of the end effector of the robotic arm in the next control cycle is solved using the second-order spring damping model. The desired position or position change is sent to the robotic arm controller, which adjusts the robotic arm's pose based on the desired position or position change, so that the adjusted robotic arm automatically guides the end-effector to move or fix at the desired position.

13. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 4.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

Images