A machine vision-based iliac bone harvesting device and method

Abstract

The application provides a kind of iliac bone taking device and method based on machine vision, it is related to medical mechanical technical field.The device includes: power box, bone grain placement table, bone taking robot, depth camera, motor, bone taking drill, force sensor and computer;Wherein the bone grain placement table is fixed on the upper surface of power box by bolt, the bone taking robot is fixed on the upper surface of power box, the depth camera is erected on power box by fixed support, and ensure that the bone grain placement table is located in the shooting range of depth camera, the motor is installed at the end of bone taking robot, the bone taking drill is installed on the motor by bone taking drill fixator, the force sensor and computer are electrically connected with bone taking robot, the position information of iliac bone taking point is identified using computer vision technology, three-dimensional model is established and bone taking is carried out using control bone taking robot, reduce the time of bone taking operation, reduce the working strength of doctor at the same time, improve the efficiency of bone taking operation.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, specifically to an iliac bone harvesting device and method based on machine vision. Background Technology

[0002] Iliac crest bone harvesting is an adjunctive surgical procedure often used in orthopedic surgeries, such as those for fractures, nonunion, bone defects, and bone tumors. Taking bone defects as an example, a bone defect is a loss of bone tissue caused by various factors, with the tibia being the most common site for such defects. The most common treatment for bone defects is autologous bone grafting, while iliac crest bone harvesting is a relatively common and minimally invasive medical approach to addressing bone defects.

[0003] Autologous bone grafting is widely used in orthopedic surgery to effectively repair damage caused by orthopedic procedures. The iliac bone is the primary harvesting site for autologous bone grafting because it contains both cancellous and compact bone, with abundant cancellous bone and a good blood supply. Harvesting it has minimal impact on function, making it the most commonly used harvesting site for both traditional and blood-supply bone grafts. However, current methods of iliac bone harvesting have drawbacks such as long operation time, excessive physical exertion on medical staff, and sometimes bone waste. Therefore, researching iliac bone harvesting methods to reduce operation time and bone waste is of great significance. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the present invention provides an iliac bone harvesting device and method based on machine vision, which can reduce the time of bone harvesting surgery, reduce the workload of doctors, and improve the efficiency of bone harvesting surgery.

[0005] To address the aforementioned technical problems, this invention provides a machine vision-based iliac bone harvesting device. The device includes: a power supply unit, a servo driver, a motion controller, a bone fragment placement stage, a bone harvesting robotic arm, a depth camera, a motor, a bone harvesting drill bit, a force sensor, and a computer. The servo driver and motion controller are bolted to the inside of the power supply unit. The bone fragment placement stage is bolted to the upper surface of the power supply unit, and the bone harvesting robotic arm is also bolted to the upper surface of the power supply unit. The depth camera is mounted on the power supply unit via a mounting bracket, ensuring that the bone fragment placement stage is within the camera's field of view. The motor is mounted at the end of the bone harvesting robotic arm, and the bone harvesting drill bit is mounted on the motor via a drill bit retainer. The force sensor is mounted inside the drill bit retainer. The servo driver, motion controller, depth camera, bone harvesting robotic arm, force sensor, and computer are all electrically connected. The power supply enclosure contains a power supply for powering the servo driver, motion controller, bone harvesting robotic arm, depth camera, motor and force sensor. The motion controller is used to send control commands to the servo driver; The servo driver is used to control the bone retrieval robotic arm to move along a planned motion trajectory under the control of the motion controller. The bone particle placement platform is used to place the removed bone particles; The depth camera is used to capture images of the iliac bone region and the area where the bone granulation stage is located, and then send these images to the computer. The motor is used to drive the bone-harvesting drill bit to rotate and complete the bone-harvesting action; The bone harvesting drill bit is used to harvest bone at the bone harvesting location in the iliac bone. The force sensor is used to acquire the cutting force at the iliac bone harvesting site. The data is then transmitted to a computer, which processes the images returned by the depth camera during the bone harvesting process and combines this with the cutting force at the iliac bone harvesting location. Determine the motion plan for the bone harvesting robotic arm; Furthermore, the power supply box is placed inside the operating room, and is positioned below the operating table according to its location. The fixed bracket consists of two vertical fixed profiles and one horizontal fixed profile, which are mounted on the power supply box. A depth camera is installed in the middle of the horizontal fixed profile to ensure that the operating table is within the shooting range of the depth camera; wherein the length of the vertical fixed profile is greater than that of the horizontal fixed profile. The bone-harvesting robotic arm has six degrees of freedom relative to the power supply housing, including: a first linear motion joint, a second linear motion joint, a linear connection plate, a first rotary motion joint, a second rotary motion joint, a third rotary motion joint, a fourth rotary motion joint, three equal-length fixed profiles, and several square connecting plates; two of the fixed profiles are vertically fixed to the upper surface of the power supply housing via square connecting plates and bolts, with the horizontal distance between them less than the length of the fixed profiles; the other fixed profile is horizontally fixed to one side of the two fixed profiles via square connecting plates and bolts, and the first linear motion joint is horizontally mounted to the other side of the two fixed profiles via square connecting plates and bolts; the second linear motion joint is horizontally mounted to the other side of the two fixed profiles via square connecting plates and bolts; the third linear motion joint is horizontally mounted to the other side of the two fixed profiles via square connecting plates and bolts. A connecting plate and bolts are mounted on the first linear motion joint and are perpendicular to it. The first rotary motion joint is connected to the second linear motion joint via an L-shaped joint connecting plate and bolts, and then via a square connecting plate. The second rotary motion joint and the first rotary motion joint are connected by a linear connecting plate and bolts. The third rotary motion joint and the second rotary motion joint are connected by an L-shaped joint connecting plate and bolts. The fourth rotary motion joint and the third rotary motion joint are connected by an L-shaped joint connecting plate and bolts. The servo driver, motion controller, and each joint in the bone harvesting robotic arm are electrically connected. The joints of the bone harvesting robotic arm cooperate with each other to achieve movement of the bone harvesting robotic arm in six directions. The bone-harvesting drill bit is mounted on the main shaft of the motor and includes: a vacuum plate, a spring, and a main cutter head; wherein the vacuum plate is connected to the main cutter head via a cylindrical pin; and the spring is connected to the inner wall structure hole of the vacuum plate via a hook on the spring. The servo driver, motion controller, depth camera, force sensor, and each joint in the bone harvesting robotic arm are all electrically connected to the computer via wired communication. Another aspect of the present invention provides a machine vision-based method for iliac bone harvesting, the method comprising the following steps: Step 1: Determine the location of the iliac bone harvesting device based on the position of the operating table, turn on the switch of the power supply box in the harvesting device, and start the iliac bone harvesting device; Step 2: Initialize the pose information of the bone retrieval robotic arm and record the current position of the bone retrieval robotic arm as the initial pose; Step 3: Set the range of feedback force for the iliac bone harvesting robotic arm at the bone harvesting location during bone retrieval. and bone granule integrity % Step 4: Capture an image of the iliac bone location region with distance information and identify the iliac bone harvesting location region. Use a random algorithm to fit the iliac bone harvesting location point in the iliac bone harvesting location region, obtain the location information of the iliac bone harvesting location point, and encapsulate it together with the image of the iliac bone location region with distance information. Step 5: Establish a simulation environment obstacle model by calling the encapsulated information, and use the fast search random tree (RRT) algorithm to perform inverse kinematics calculations on the bone harvesting robot arm to plan the motion trajectory of the bone harvesting robot arm from the initial pose to the iliac bone harvesting location. Step 6: The robotic arm for bone retrieval begins to move according to the planned trajectory. When the robotic arm reaches the iliac bone retrieval point, the distance is... When the position is reached, the bone-removing drill bit begins to rotate; Step 7: The bone retrieval robotic arm continues to move along its trajectory. When the robotic arm reaches the iliac bone retrieval location, it performs a trial cut to obtain the cutting force at the iliac bone retrieval location. ; Step 8: Determine the cutting force at this location. Is it within the set feedback force range? If not, the bone retrieval drill stops rotating, and the bone retrieval robotic arm returns to the initial position along the original trajectory and repeats step 4; if yes, the direction of the bone retrieval drill remains unchanged and continues to advance a preset distance along that direction before stopping rotating, and then returns to the iliac bone retrieval position point along that direction. Step 9: Obtain the current position coordinates of the bone retrieval robotic arm and the position coordinates of the bone particle placement platform. Based on the obtained position coordinates, re-plan the motion trajectory of the bone retrieval robotic arm from the current position to the bone particle placement platform. The bone retrieval robotic arm moves to the bone particle placement platform according to the new motion trajectory. The operator manually removes the bone particles carried on the bone retrieval drill bit and places them on the bone particle placement platform. Step 10: Use a depth camera to capture images of the bone granules and use machine vision technology to identify the integrity of the bone granules. ,judge If the condition is not met, step 4 is repeated until the condition is met; if the condition is met, the bone retrieval robot arm is moved to the initial pose to complete the bone retrieval process. Furthermore, the bone retrieval process performed by the robotic arm in step 7 includes: keeping the direction of the bone retrieval drill unchanged, the robotic arm advances along the axis of the bone retrieval drill; when the bone retrieval drill enters the iliac bone to a depth of 1 mm, the force sensor acquires the cutting force at that position as the cutting force for the iliac bone retrieval position. .

[0006] The beneficial effects of adopting the above technical solution are as follows: Compared with existing technologies, this invention proposes a machine vision-based iliac bone harvesting device and method. It utilizes a computer in conjunction with a depth camera to identify the location information of the iliac bone harvesting point through visual technology. The computer performs 3D modeling analysis on the location and distance information extracted by the depth camera, and uses the harvesting point location information and the 3D modeling analysis results to control the harvesting angle of the bone harvesting drill bit mounted on the robotic arm. Through force sensor feedback signals, the computer can receive real-time feedback on the force information at the harvesting location, and the computer program analyzes the magnitude of the real-time force at different locations to determine whether the location is suitable for bone harvesting. The drill bit structure designed in this invention for iliac bone harvesting not only ensures the quality of the extracted cancellous bone but also further reduces the time of the bone harvesting operation, while simultaneously reducing the workload of the surgeon, thereby improving the efficiency of the bone harvesting operation. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the structure of an iliac bone harvesting device based on machine vision in this embodiment. Figure 2 This is a side view of an iliac bone harvesting device based on machine vision in this embodiment; Figure 3 This is a schematic diagram of the bone-harvesting drill bit structure in this embodiment; Figure 4 Figure (a) is a front view of the angle of the bone-harvesting drill and the position of the operating table in this embodiment; Figure (b) is a side view of the angle of the bone-harvesting drill and the position of the operating table. Figure 5 This is a flowchart of a machine vision-based iliac bone harvesting method in this embodiment; Among them, 1—power supply box, 2—servo driver, 3—motion controller, 4—bone particle placement stage, 5—fixation bracket, 6—depth camera, 7—first linear motion joint, 8—second linear motion joint, 9—linear connecting plate, 10—first rotary motion joint, 11—second rotary motion joint, 12—third rotary motion joint, 13—fourth rotary motion joint, 14—motor, 15—bone harvesting drill bit fixator, 16—bone harvesting drill bit, 17—vacuum plate, 18—spring, 19—main cutter head, 20—operating table. Detailed Implementation

[0008] To facilitate understanding of this application, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following embodiments are illustrative of the invention but are not intended to limit its scope. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0009] This embodiment provides a machine vision-based iliac bone harvesting device, such as... Figure 1 As shown, the device includes: a power supply housing 1, a servo driver 2, a motion controller 3, a bone particle placement stage 4, a bone retrieval robotic arm, a depth camera 6, a motor 14, a bone retrieval drill bit 16, a force sensor, and a computer; wherein the servo driver 2 and the motion controller 3 are both bolted to the inside of the power supply housing 1; the bone particle placement stage 4 is fixed to the upper surface of the power supply housing 1 by bolts, the bone retrieval robotic arm is fixed to the upper surface of the power supply housing 1, the depth camera 6 is mounted on the power supply housing 1 by a fixing bracket 5, ensuring that the bone particle placement stage 4 is within the shooting range of the depth camera 6, the motor 14 is installed at the end of the bone retrieval robotic arm, the bone retrieval drill bit 16 is mounted on the motor 14 by a bone retrieval drill bit retainer 15, the force sensor is installed inside the bone retrieval drill bit retainer 15, and the servo driver 2, the motion controller 3, the depth camera 6, the bone retrieval robotic arm, the force sensor, and the computer are all electrically connected.

[0010] The power supply housing 1 contains a power supply for powering the servo driver 2, motion controller 3, bone harvesting robotic arm, depth camera 6, motor 14, and force sensor.

[0011] The motion controller 3 is used to send control commands to the servo driver 2.

[0012] The servo driver 2 is used to control the bone retrieval robotic arm to move along a planned motion trajectory under the control of the motion controller 3.

[0013] The bone particle placement platform 4 is used to place the removed bone particles.

[0014] The depth camera 6 is used to capture images of the iliac bone region and the area where the bone granulation stage is located, and then send these images to the computer.

[0015] The motor 14 is used to drive the bone-harvesting drill bit 16 to rotate and complete the bone-harvesting action.

[0016] The bone-harvesting drill bit 16 is used to harvest bone at the bone-harvesting location in the iliac bone.

[0017] The force sensor and computer are used to acquire the cutting force at the iliac bone harvesting site. ; The computer processes the images returned by the depth camera during the bone harvesting process and combines them with the cutting force at the iliac bone harvesting location. Determine the motion plan for the bone harvesting robotic arm.

[0018] The power supply box 1 is placed in the operating room and positioned below the operating table 20 according to its position. The servo driver 2 and motion controller 3 are both bolted to the inside of the power supply box 1 to drive and control the iliac bone harvesting device.

[0019] The fixed bracket 5 consists of two vertical fixed profiles and one horizontal fixed profile, which are mounted on the power supply box 1. A depth camera 6 is installed in the middle of the horizontal fixed profile to ensure that the operating table 20 is within the shooting range of the depth camera 6. The length of the vertical fixed profile is greater than that of the horizontal fixed profile.

[0020] In this embodiment, the fixed bracket 5 is composed of two vertical 1200mm fixed profiles and one horizontal 900mm fixed profile, which are mounted on the power supply box 1. The depth camera 6 is installed in the middle of the 900mm fixed profile to ensure that the operating table 20 is within the shooting range of the depth camera 6.

[0021] The bone-harvesting robotic arm has six degrees of freedom relative to the power supply housing, including: a first linear motion joint 7, a second linear motion joint 8, a linear connection plate 9, a first rotary motion joint 10, a second rotary motion joint 11, a third rotary motion joint 12, a fourth rotary motion joint 13, three fixed profiles of equal length, and several square connecting plates; two of the fixed profiles are vertically fixed to the upper surface of the power supply housing 1 by square connecting plates and bolts, and the horizontal distance between them is less than the length of the fixed profiles; the other fixed profile is horizontally fixed to one side surface of the two fixed profiles by square connecting plates and bolts, and the first linear motion joint 7 is horizontally mounted on the other side surface of the two fixed profiles by square connecting plates and bolts; the second linear motion joint 8 is horizontally mounted to the other side surface of the two fixed profiles by square connecting plates and bolts. Bolts are installed on the first linear motion joint 7 and are perpendicular to the first linear motion joint 7; the first rotary motion joint 10 is connected to the second linear motion joint 8 via an L-shaped joint connecting plate and bolts, and then via a square connecting plate; the second rotary motion joint 11 and the first rotary motion joint 10 are connected via a linear connecting plate 9 and bolts; the third rotary motion joint 12 and the second rotary motion joint 11 are connected via an L-shaped joint connecting plate and bolts; the fourth rotary motion joint 13 and the third rotary motion joint 12 are connected via an L-shaped joint connecting plate 11 and bolts; wherein the servo driver 2 and the motion controller 3 are electrically connected to each joint in the bone retrieval robotic arm; the joints of the bone retrieval robotic arm cooperate with each other to realize the movement of the bone retrieval robotic arm in six directions.

[0022] In this embodiment, such as Figure 1 and Figure 2As shown, two 460mm fixed profiles are connected to the upper surface of the power supply box 1 by a square connecting plate and bolts, and are 190mm apart. A third 460mm fixed profile is perpendicularly connected to the first two 460mm fixed profiles by three square connecting plates and bolts at different positions. The first linear motion joint 7 is bolted to the first two 460mm fixed profiles using two square connecting plates. The first linear motion joint 7 and the second linear motion joint 8 are bolted together by two square connecting plates. The second linear motion joint 8 and the first rotary motion joint 10 are connected by square connecting plates, L-shaped joint connecting plates, and bolts. The first rotary motion joint 10 and the second rotary motion joint 10 are bolted together by a linear connecting plate 9. The second rotary motion joint 11 and the third rotary motion joint 12 are bolted together by an L-shaped joint connecting plate. The third rotary motion joint 12 and the fourth rotary motion joint 13 are bolted together by an L-shaped joint connecting plate 11, forming a bone retrieval robotic arm. The bone retrieval robotic arm has six degrees of freedom relative to the power supply box 1.

[0023] The bone-harvesting drill bit 16 is mounted on the spindle of the motor 14 and includes: a vacuum plate 17, a spring 18, and a main cutter head 19; wherein the vacuum plate 17 is connected to the main cutter head 19 by a cylindrical pin; and the spring 18 is connected to the inner wall structure hole of the vacuum plate 17 by a hook on the spring.

[0024] In this embodiment, such as Figure 3 As shown, the vacuum plate 17 is connected to the main blade head 19 by a cylindrical pin with a diameter of 0.15 mm; the computer is placed near the power supply box 1 according to the spatial location inside the operating room.

[0025] The servo driver 2, motion controller 3, depth camera 6, force sensor, and each joint in the bone harvesting robotic arm are all electrically connected to the computer via wired communication.

[0026] In this embodiment, such as Figure 4 As shown, the iliac bone harvesting device controls the angle between the bone harvesting drill 16 and the operating table 20 in different directions. and To determine the position and orientation of the bone-harvesting drill 16 during surgery; the operating table 20 is placed on one side of the upper surface of the power supply box 1.

[0027] This embodiment proposes a machine vision-based method for iliac bone harvesting, employing a machine vision-based iliac bone harvesting device, such as... Figure 5 As shown, it includes the following steps: Step 1: Determine the location of the iliac bone harvesting device based on the position of the operating table, turn on the power switch in the power box of the harvesting device, and start the iliac bone harvesting device.

[0028] In this embodiment, the power supply unit 1 is placed under the operating table, and the depth camera 6 located above the operating table is moved to ensure that the iliac bone appears within the shooting range of the depth camera. The power supply unit 1 is turned on, and the power supply unit 1, servo motion unit 2 and motion controller 3 are started, thereby driving the bone harvesting robotic arm to complete the bone harvesting action, ensuring that the bone harvesting robotic arm moves smoothly in the subsequent bone harvesting process; the computer is started to ensure that the images returned by the depth camera 6 are processed and the motion planning of the bone harvesting robotic arm is calculated during the bone harvesting process.

[0029] Step 2: Initialize the pose information of the bone retrieval robotic arm and record the current position of the bone retrieval robotic arm as the initial pose.

[0030] In this embodiment, the pose initialization of the bone retrieval robotic arm is performed so that the computer obtains the initial pose information of the bone retrieval robotic arm before performing trajectory calculation.

[0031] Step 3: Set the range of feedback force for the iliac bone harvesting robotic arm at the bone harvesting location during bone retrieval. and bone granule integrity %.

[0032] In this embodiment, since the hard bone above the iliac bone needs to be removed by medical staff before bone harvesting, the bone harvesting effect will be poor if the hard bone is not removed. Therefore, the operator needs to make a manual judgment based on the bone quality of the iliac bone to limit the range of feedback force at the bone harvesting location and ensure that the target point is the area where the doctor has removed the hard bone.

[0033] Step 4: Capture an image of the iliac bone location region with distance information and identify the iliac bone harvesting location region. Use a random algorithm to fit the iliac bone harvesting location point in this iliac bone harvesting location region, obtain the location information of the iliac bone harvesting location point, and encapsulate it together with the iliac bone location region image with distance information.

[0034] In this embodiment, an image of the iliac bone location region with distance information captured by the depth camera 6 is transmitted to the computer. The computer uses machine vision technology, namely the YOLOv5 model, to identify the iliac bone harvesting location region based on the iliac bone location region image, and fits the iliac bone harvesting location point in the region using a random algorithm. At the same time, the computer calculates the normal direction information of the iliac bone harvesting location point and the distance information of the iliac bone harvesting location point relative to the camera based on the distance information contained in the iliac bone location region image, and encapsulates the iliac bone location region image with distance information and the position, distance and direction information of the iliac bone harvesting location point for later use.

[0035] Step 5: Establish a simulation environment obstacle model by calling the encapsulated information, and use the Fast Search Random Tree (RRT) algorithm to perform inverse kinematics calculations on the bone retrieval robot arm to plan the motion trajectory of the bone retrieval robot arm from the initial pose to the iliac bone retrieval location.

[0036] In this embodiment, the computer uses the image of the iliac bone location region with distance information captured by the depth camera 6 and calls the encapsulated information of the iliac bone harvesting location point in step 4 to perform simulated environment obstacle modeling. Based on the established simulated environment obstacle model, the computer uses the Rapidly-Exploring Random Trees (RRT) algorithm to automatically perform inverse kinematics calculations of the bone harvesting robot arm and plans the motion trajectory of the bone harvesting robot arm from the initial pose to the iliac bone harvesting location point.

[0037] Step 6: The robotic arm for bone retrieval begins to move according to the planned trajectory. When the robotic arm reaches the iliac bone retrieval point, the distance is... When the position is reached, the bone-removing drill bit begins to rotate.

[0038] In this embodiment, the bone harvesting robotic arm, under the control of the motion controller 3, moves along a planned trajectory to a distance in front of the iliac bone harvesting location. When the position is 10mm, the motor 14 drives the bone retrieval drill 16 to start rotating. When the bone retrieval drill 16 rotates, it uses centrifugal force to overcome the tension of the spring 18 and support the vacuum plate 17 on the cutter head, so that the bone retrieval drill 16 can smoothly rotate into the cancellous bone of the iliac bone.

[0039] Step 7: The bone retrieval robotic arm continues to move along its trajectory. When the robotic arm reaches the iliac bone retrieval location, it performs a trial cut to obtain the cutting force at the iliac bone retrieval location. .

[0040] The process of the bone harvesting robotic arm performing trial cuts includes: keeping the direction of the bone harvesting drill 16 unchanged, the bone harvesting robotic arm advances along the axis of the bone harvesting drill 16, and when the bone harvesting drill 16 enters the iliac bone to a depth of 1mm, the force sensor obtains the cutting force at that position as the cutting force for the iliac bone harvesting position. .

[0041] In this embodiment, the bone-harvesting robotic arm begins a trial cut to harvest bone. Based on the location information of the iliac bone harvesting location point obtained by the depth camera 6 and processed by the computer, it enters a position with a fixed depth of 1 mm. The force sensor feeds back the cutting force F to the computer.

[0042] Step 8: Determine the cutting force at this location. Is it within the set feedback force range? If not, the bone retrieval drill stops rotating, and the bone retrieval robotic arm returns to its initial position along the original trajectory, and steps 4 are executed again; if yes, the direction of the bone retrieval drill remains unchanged and continues to advance a preset distance in that direction before stopping rotating, and then returns to the iliac bone retrieval location in that direction.

[0043] In this embodiment, the cutting force at that position is determined. Is it within the set feedback force range? If it is not within the range of the feedback force magnitude If the motor 14 stops rotating the bone-retrieving drill 16, the bone-retrieving robotic arm returns to its initial position along the planned motion trajectory and repeats step 4; if the feedback force is within the range... Inside, the direction of the bone-harvesting drill bit 16 remains unchanged, and after advancing 10mm along the direction of the drill bit, the bone-harvesting drill bit 16 driven by the motor 14 stops rotating, and then returns to the iliac bone harvesting position point in the original direction.

[0044] Step 9: Obtain the current position coordinates of the bone retrieval robotic arm and the position coordinates of the bone particle placement platform. Based on the obtained position coordinates, replan the motion trajectory of the bone retrieval robotic arm from the current position to the bone particle placement platform. The bone retrieval robotic arm moves to the bone particle placement platform according to the new motion trajectory. The operator manually removes the bone particles carried by the bone retrieval drill bit and places them on the bone particle placement platform.

[0045] In this embodiment, the computer uses machine vision technology to identify and calculate the center coordinates of the bone fragment placement platform 4 based on the encapsulation information of the iliac bone harvesting location in step 4 and the image of the location of the bone fragment placement platform 4 captured by the depth camera 6. Based on the obtained position coordinates and the distance of 300mm between the fixed position and the upper surface of the power supply box 1 in the vertical direction, the computer re-plans the motion trajectory of the bone harvesting robotic arm. The bone harvesting robotic arm moves to the bone fragment placement platform 4 according to the new motion trajectory, and the operator manually removes the bone fragments carried on the bone harvesting drill bit and places them on the bone fragment placement platform 4.

[0046] Step 10: Use a depth camera to capture images of the bone granules and use machine vision technology to identify the integrity of the bone granules. ,judge If the condition is not met, step 4 is repeated until the condition is met; if the condition is met, the bone retrieval robot arm is moved to the initial pose to complete the bone retrieval process.

[0047] In this embodiment, a depth camera 6 captures an image of the bone grain, and the image is transmitted to a computer. The computer's machine vision technology is then used to identify the integrity of the bone grain. and judge If the condition is not met, step 4 is repeated until the condition is met; if the condition is met, the bone retrieval robot arm is returned to its initial position, and the bone retrieval process ends.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A machine vision-based iliac bone harvesting device, characterized in that, The device includes: a power supply housing, a servo driver, a motion controller, a bone particle placement stage, a bone retrieval robotic arm, a depth camera, a motor, a bone retrieval drill bit, a force sensor, and a computer. The servo driver and motion controller are bolted to the inside of the power supply housing. The bone particle placement stage is bolted to the upper surface of the power supply housing, and the bone retrieval robotic arm is also bolted to the upper surface of the power supply housing. The depth camera is mounted on the power supply housing via a mounting bracket, ensuring that the bone particle placement stage is within the camera's field of view. The motor is mounted at the end of the bone retrieval robotic arm, and the bone retrieval drill bit is mounted on the motor via a drill bit retainer. The force sensor is mounted inside the drill bit retainer. The servo driver, motion controller, depth camera, bone retrieval robotic arm, force sensor, and computer are all electrically connected. The power supply enclosure contains a power supply for powering the servo driver, motion controller, bone harvesting robotic arm, depth camera, motor and force sensor. The motion controller is used to send control commands to the servo driver; The servo driver is used to control the bone retrieval robotic arm to move along a planned motion trajectory under the control of the motion controller. The bone particle placement platform is used to place the removed bone particles; The depth camera is used to capture images of the iliac bone region and the area where the bone granulation platform is located, and then send these images to the computer. The process of capturing images of the iliac bone region and the area where the bone graft placement stage is located includes: capturing an image of the iliac bone location region with distance information and identifying the iliac bone harvesting location region; using a random algorithm to fit the iliac bone harvesting location point in the iliac bone harvesting location region; obtaining the location information of the iliac bone harvesting location point and encapsulating it together with the image of the iliac bone location region with distance information. The motor is used to drive the bone-harvesting drill bit to rotate and complete the bone-harvesting action; The bone harvesting drill bit is used to harvest bone at the bone harvesting location in the iliac bone. The force sensor is used to acquire the cutting force at the iliac bone harvesting site. The data is then transmitted to a computer, which processes the images returned by the depth camera during the bone harvesting process and combines this with the cutting force at the iliac bone harvesting location. Determine the motion plan for the bone harvesting robotic arm; The cutting force used to determine the iliac bone harvesting location This includes: the bone harvesting robotic arm begins to move according to the planned motion trajectory; when the robotic arm reaches the iliac bone harvesting location point, the distance is... When the bone is positioned, the bone retrieval drill begins to rotate; the robotic arm continues to move along its trajectory. When the robotic arm reaches the iliac bone retrieval location, it performs a trial cut to obtain the cutting force at the iliac bone retrieval location. ; The computer processes the images returned by the depth camera during the bone harvesting process and combines them with the cutting force at the iliac bone harvesting location. Determining the motion planning of the bone harvesting robotic arm includes: establishing a simulation environment obstacle model by calling encapsulated information; performing inverse kinematics calculations on the robotic arm using the Fast Random Tree Search (RRT) algorithm to plan the motion trajectory of the robotic arm from its initial pose to the iliac bone harvesting location; and determining the cutting force at the current iliac bone harvesting location. Is it within the set feedback force range? If not, the bone retrieval drill stops rotating, and the robotic arm returns to its initial position along its original trajectory. It then re-captures images of the iliac bone region and the area where the bone graft placement platform is located, and fits the iliac bone retrieval location point. If yes, the direction of the bone retrieval drill remains unchanged, and it continues forward a preset distance along that direction before stopping and returning to the iliac bone retrieval location point. The current position coordinates of the robotic arm and the position coordinates of the bone graft placement platform are obtained. Based on these coordinates, the robotic arm's trajectory from its current position to the bone graft placement platform is re-planned. The robotic arm moves to the bone graft placement platform according to the new trajectory, and the operator manually removes the bone graft carried on the bone retrieval drill and places it on the platform. The image of the bone graft captured by the depth camera is then used to identify the integrity of the bone graft using machine vision technology. ,judge Whether it is valid, If the set bone fragment integrity is not met, images of the iliac bone region and the area where the bone fragment placement platform is located are re-captured, and the iliac bone harvesting location point is fitted until it is met; if it is met, the bone harvesting robotic arm is moved to the initial pose to complete the bone harvesting process.

2. The iliac bone harvesting device based on machine vision according to claim 1, characterized in that, The power supply box is placed inside the operating room, and is positioned below the operating table according to its location.

3. The iliac bone harvesting device based on machine vision according to claim 2, characterized in that, The fixed bracket consists of two vertical fixed profiles and one horizontal fixed profile, which are mounted on the power supply box. A depth camera is installed in the middle of the horizontal fixed profile to ensure that the operating table is within the shooting range of the depth camera. The length of the vertical fixed profile is greater than that of the horizontal fixed profile.

4. The iliac bone harvesting device based on machine vision according to claim 2, characterized in that, The bone-harvesting robotic arm has six degrees of freedom relative to the power supply housing, including: a first linear motion joint, a second linear motion joint, a linear connection plate, a first rotary motion joint, a second rotary motion joint, a third rotary motion joint, a fourth rotary motion joint, three fixed profiles of equal length, and several square connecting plates; two of the fixed profiles are vertically fixed to the upper surface of the power supply housing via square connecting plates and bolts, with the horizontal distance between them less than the length of the fixed profiles; the other fixed profile is horizontally fixed to one side of the two fixed profiles via square connecting plates and bolts, and the first linear motion joint is horizontally mounted on the other side of the two fixed profiles via square connecting plates and bolts; the second linear motion joint is horizontally mounted via square connecting plates and bolts. An L-shaped connecting plate and bolts are mounted on the first linear motion joint and are perpendicular to it. The first rotary motion joint is connected to the second linear motion joint via an L-shaped joint connecting plate and bolts, and then via a square connecting plate. The second rotary motion joint and the first rotary motion joint are connected by a linear connecting plate and bolts. The third rotary motion joint and the second rotary motion joint are connected by an L-shaped joint connecting plate and bolts. The fourth rotary motion joint and the third rotary motion joint are connected by an L-shaped joint connecting plate and bolts. The servo driver, motion controller, and each joint in the bone harvesting robotic arm are electrically connected. The joints of the bone harvesting robotic arm cooperate with each other to realize the movement of the bone harvesting robotic arm in six directions.

5. The iliac bone harvesting device based on machine vision according to claim 1, characterized in that, The bone-harvesting drill bit is mounted on the main shaft of the motor and includes: a vacuum plate, a spring, and a main cutter head; wherein the vacuum plate is connected to the main cutter head via a cylindrical pin; and the spring is connected to the inner wall structure hole of the vacuum plate via a hook on the spring.

6. The iliac bone harvesting device based on machine vision according to claim 4, characterized in that, The servo driver, motion controller, depth camera, force sensor, and various joints in the bone harvesting robotic arm are all electrically connected to the computer via wired communication.

Images