Surgical robot anti-collision system and anti-collision method based on binocular vision
By using binocular vision and convolutional neural networks to identify the key tracking points of the surgical robot, the problem of monocular cameras failing to detect in inaccurate three-dimensional depth information and complex environments is solved, achieving high-precision, real-time collision warning and avoidance, and improving surgical safety and adaptability.
Patent Information
- Application Number
- CN202510864736.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-26
- Publication Date
- 2025-09-26
AI Technical Summary
Existing medical robots based on monocular cameras cannot accurately obtain depth information in three-dimensional space, making it difficult to cope with complex surgical environments. In addition, collision detection is prone to fail in environments with insufficient light or poor texture, resulting in false alarms or missed alarms.
A binocular vision-based anti-collision system is used, which uses a binocular camera and a convolutional neural network to identify the key tracking points of the surgical arm. Combined with an anti-collision algorithm, it calculates the three-dimensional position and motion trajectory in real time, provides collision alerts through the display screen and voice broadcast module, and avoids collisions by adjusting the movement of the robotic arm.
It improves the spatial recognition accuracy and safety of surgical robot operations, enhances the adaptability and robustness of the system under different lighting conditions, provides a user-friendly interactive interface, and realizes self-learning and optimized anti-collision strategies.
Smart Images
Figure CN120694754A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical robot anti-collision technology, and in particular to a binocular vision-based surgical robot anti-collision system and anti-collision method. Background Art
[0002] With advances in medical technology, surgical robots have been widely used in a variety of complex surgeries, improving surgical outcomes and reducing medical risks with their high precision and stability. However, safety during surgical robot operation remains a key design consideration. Especially in crowded operating room environments, ensuring that the robotic arm does not collide with external objects is crucial for ensuring surgical safety and improving efficiency.
[0003] Currently, most commercial collision avoidance systems use monocular camera technology, relying on a constant velocity model or lateral acceleration model to detect potential collision risks by identifying texture keypoints. However, this monocular camera-based approach has certain limitations, such as inaccurate acquisition of three-dimensional depth information and insufficient adaptability to complex surgical scenarios, which limits its application in the field of surgical robotics.
[0004] Furthermore, collision detection based on texture keypoints can easily fail in low-light or texture-poor environments, leading to false positives or false negatives, increasing surgical risks. Therefore, it is urgent and necessary to develop a system that can more accurately detect the position and motion of surgical robot arms in three-dimensional space, thereby achieving efficient and accurate collision warnings. Summary of the Invention
[0005] The technical problems to be solved by the present invention are:
[0006] In order to solve the problem that medical robots based on monocular cameras cannot obtain accurate depth information in three-dimensional space, are difficult to cope with complex surgical environments, and collision detection based on texture key points is prone to failure in environments with insufficient light or poor texture, resulting in false positives or missed positives.
[0007] The present invention is to solve the above technical problems using the following technical solutions:
[0008] The present invention provides a surgical robot anti-collision system based on binocular vision, comprising a trolley, a binocular camera and a surgical arm.
[0009] The trolley includes a traveling motor and traveling wheels, which are used to drive the traveling wheels to move and thus drive the surgical robot to move. The trolley is provided with at least one binocular camera; the top of the trolley is connected to multiple surgical arms through a connecting rod. The surgical arms are three-axis surgical arms, and each surgical arm is provided with multiple key tracking points. The binocular camera is facing the multiple surgical arms, and is used to collect images of the multiple surgical arms;
[0010] It also includes a central processing unit, which includes an image processing module, a data analysis module and an alarm module. The image processing module is used to use a convolutional neural network to process the stereo images collected by the binocular camera, and identify the surgical arm and key tracking points on the surgical arm; the data analysis module is used to calculate the three-dimensional position and motion trajectory of the surgical arm for the key tracking points identified by the image processing module, and use an anti-collision algorithm to evaluate the collision risk; the alarm module is used to issue an alarm when the distance between the surgical arm and the obstacle is less than the safe distance.
[0011] Furthermore, it also includes a display screen for issuing warnings to the operator through a user interface on the display screen based on the collision risk assessment results, and can display the real-time position, expected trajectory and potential collision area of the surgical arm.
[0012] Furthermore, it also includes a voice broadcast module for reminding the operator through voice broadcast when an alarm occurs.
[0013] Furthermore, the number of the surgical arms is set to three, and the number of key tracking points on each surgical arm is set to three.
[0014] Furthermore, it also includes a communication module for wirelessly connecting the binocular camera, surgical arm, display screen and voice broadcast module with the central processing unit, and wirelessly connecting the central processing unit with the remote terminal.
[0015] A collision avoidance method for a surgical robot collision avoidance system based on binocular vision, comprising the following steps:
[0016] S100, collecting stereoscopic image information of multiple surgical arms in the surgical area in real time using a binocular camera;
[0017] S200, the image processing module is used to process the stereo image information collected by the binocular camera using a convolutional neural network to identify the surgical arm and key tracking points on the surgical arm;
[0018] S300, the data analysis module calculates the three-dimensional position and motion trajectory of the surgical arm based on the key tracking points identified in step S200, and evaluates the collision risk;
[0019] S400: When an alarm is triggered, an alarm signal is issued through a display screen or a voice alarm model, and a collision is avoided by controlling the robotic arm to stop moving or move in the reverse direction.
[0020] Furthermore, the anti-collision algorithm is based on the velocity Jacobian matrix to obtain the component of the motion velocity direction of the current key tracking point toward the line connecting the same key points of the two adjacent manipulators, that is, the velocity vector in the C1 space and the acceleration vector along the C1C2 component, that is, 、 and the C2 velocity vector component towards C2C1 、 ; Emergency response time of equipment operated by a large number of sampling personnel , calculate the minimum collision distance in real time:
[0021]
[0022] Set a safety factor on the minimum collision distance S , and obtain the final safe distance , when the distance between any two points is less than the final safe distance The alarm is triggered.
[0023] Compared with the prior art, the present invention has the following beneficial effects:
[0024] Improved accuracy and safety: By utilizing binocular stereo vision technology, it can provide more accurate depth information than traditional monocular camera systems, significantly improving the spatial recognition accuracy and safety during surgical robot operations.
[0025] Real-time dynamic tracking: Combined with convolutional neural networks (CNN), it can identify and track key tracking points on each surgical arm of the surgical robot in real time, effectively preventing collisions.
[0026] High adaptability: The image processing module can work accurately under different lighting conditions and surgical environments, enhancing the adaptability and robustness of the system in actual surgery.
[0027] User-friendly interactive interface: By providing intuitive warnings and virtual model displays on the display, it enhances the operator experience and enables them to quickly respond to potential collision risks.
[0028] Self-learning and optimization: The introduction of machine learning algorithms enables the system to learn from historical operations, automatically adjust anti-collision strategies, and improve surgical safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 This is a three-dimensional diagram of a surgical robot anti-collision system based on binocular vision in an embodiment of the present invention;
[0030] Figure 2 3D diagram of multiple surgical arms with key tracking points in an embodiment of the present invention. DETAILED DESCRIPTION
[0031] In the description of the present invention, it should be noted that the terminology in each embodiment, such as "up", "down", "front", "back", "left", "right", etc., which indicate directions, are only for simplifying the description of the positional relationship based on the drawings in the specification, and do not mean that the referred elements and devices must be operated in accordance with the specific directions and defined operations and methods and structures in the specification. Such directional nouns do not constitute a limitation to the present invention.
[0032] In the description of the present invention, it should be noted that the terms "first," "second," and "third" mentioned in the embodiments of the present invention are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include one or more of such features.
[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0034] Specific implementation plan 1: Combined Figure 1 and 2 As shown, the present invention provides a surgical robot anti-collision system based on binocular vision, including a trolley, a binocular camera and a surgical arm.
[0035] The trolley includes a traveling motor and traveling wheels, which are used to drive the traveling wheels to move and thus drive the surgical robot to move. The trolley is provided with at least one binocular camera; the top of the trolley is connected to multiple surgical arms through a connecting rod. The surgical arms are three-axis surgical arms, and each surgical arm is provided with multiple key tracking points. The binocular camera is facing the multiple surgical arms, and is used to collect images of the multiple surgical arms;
[0036] The system also includes a central processing unit (CPU), which includes an image processing module, a data analysis module, and an alarm module. The image processing module is used to process the stereo images collected by the binocular camera using a convolutional neural network (CNN) or other deep learning model to identify the surgical arm and the key tracking points on the surgical arm (calculating the absolute distance between the robotic arms relies on binocular monitoring of the key points; the binoculars can detect the absolute spatial position of the key points, and knowing the absolute position of the key points can calculate the absolute distance). Figure 2A1, B1, C1, ..., An, Bn, Cn, where n is the number of surgical arms, and multiple robotic arms are extended accordingly; the data analysis module is used to calculate the three-dimensional position and motion trajectory of the surgical arm based on the key tracking points identified by the image processing module, and use the anti-collision algorithm to assess the collision risk; the alarm module is used to issue an alarm when the distance between the surgical arm and the obstacle is less than the safe distance;
[0037] Key tracking points include but are not limited to three or more pre-set points on each surgical arm. These key tracking points have the same relative position on different surgical arms but are assigned unique identifiers. Based on the depth detection function of the binocular camera, the spatial position of the current key point relative to the binocular camera can be calculated. If the coordinates of two points in space are known, the relative distance between the two points can be calculated.
[0038] The anti-collision algorithm is based on the current motion state of the surgical arm and the velocity Jacobian matrix to obtain the component of the motion velocity direction of the current key tracking point toward the direction of the line connecting the same key points of the two adjacent robotic arms, that is, the velocity vector in the C1 space and the acceleration vector along the C1C2 component, that is, 、 and the C2 velocity vector component towards C2C1 、 ; Emergency response time of equipment operated by a large number of sampling personnel , calculate the minimum collision distance in real time , set the safety factor on the minimum collision distance S , and obtain the final safe distance , when the distance between any two points is less than the final safe distance The alarm is triggered when
[0039] It also includes a display screen. Based on the collision risk assessment results, the user interface on the surgical robot's display screen issues a warning to the operator. The interface can display the real-time position, expected trajectory, and potential collision area of the surgical arm (by giving parameter labels, then through data training, and ultimately identifying target data through the training data. For example, the above parameters are calculated through 500 actual surgical motion trajectory data, and then trained based on collision events and 0.5 seconds in advance of the collision event (for example, if this time can be extended for higher safety), the trained model can predict whether a collision will occur within the next 0.5 seconds based on the current position curve, so as to provide a prompt or avoidance; for example, using CNN to predict the possibility of a collision, the motion trajectory of the robotic arm and the area where the two arms will collide are predicted based on the current velocity vector and acceleration vector).
[0040] It also includes a voice broadcast module for reminding the operator by voice broadcast when an alarm occurs;
[0041] The system can also automatically adjust the trajectory of the robot arm based on its speed and direction to avoid collisions; this adjustment is done automatically by the algorithm without human intervention;
[0042] The system's image processing unit regularly self-calibrates (by adaptively adjusting exposure intensity because the ambient light around the camera is unstable and may be obscured; and the camera needs to focus because the position of the robotic arm relative to the binoculars is constantly changing) to adapt to lighting changes in the operating room and the reflective properties of the robotic arm's surface, ensuring tracking accuracy.
[0043] The user interface set on the display allows the operator to customize warning levels and prompt methods, such as sound, color or flashing frequency, to adapt to different surgical needs and operator preferences; it also allows interactive operations such as zooming in and rotating the view so that the operator can examine the potential collision area from multiple angles.
[0044] The number of the surgical arms is set to three, and the number of key tracking points on each surgical arm is set to three.
[0045] The binocular camera uses a high-resolution sensor to improve image clarity and depth measurement accuracy. To reduce system response time, the image processing unit and data analysis unit use high-performance computing hardware to support real-time data processing.
[0046] It also includes a communication module for wirelessly connecting the binocular camera, surgical arm, display screen and voice broadcast module to the central processing unit, and for wirelessly connecting the central processing unit to the remote terminal.
[0047] The system provides a multi-language user interface to meet the needs of users in different countries and regions.
[0048] The system design includes a low-power mode that reduces energy consumption and extends the life of the system when the surgical robot is in standby mode.
[0049] Specific implementation scheme 2: The present invention provides an anti-collision method of a surgical robot anti-collision system based on binocular vision, comprising the following steps:
[0050] S100, collecting stereoscopic image information of multiple surgical arms in the surgical area in real time using a binocular camera;
[0051] S200, the image processing module is used to process the stereo image information collected by the binocular camera using a convolutional neural network (CNN) or other deep learning models to identify the surgical arm and key tracking points on the surgical arm;
[0052] S300, the data analysis module calculates the three-dimensional position and motion trajectory of the surgical arm based on the key tracking points identified in step S200, and evaluates the collision risk;
[0053] That is, based on the velocity Jacobian matrix, the component of the motion velocity direction of the current key tracking point toward the line connecting the same key points of the two adjacent manipulators is obtained, that is, the velocity vector in the C1 space and the acceleration vector along the C1C2 component are 、 and the C2 velocity vector component towards C2C1 、 ; Emergency response time of equipment operated by a large number of sampling personnel , calculate the minimum collision distance in real time , set the safety factor on the minimum collision distance S , and obtain the final safe distance , when the distance between any two points is less than the final safe distance The alarm is triggered when
[0054] S400: When an alarm is triggered, an alarm signal is issued through a display screen or a voice alarm model, and a collision is avoided by controlling the robotic arm to stop moving or move in the reverse direction.
[0055] The other combinations and connection relationships of this embodiment are the same as those of the first embodiment.
[0056] Although the present invention is disclosed as above, the scope of protection disclosed by the present invention is not limited thereto. Those skilled in the art of the present invention may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the scope of protection of the present invention.
Claims
1. A surgical robot anti-collision system based on binocular vision, characterized by: Including trolley, binocular camera and surgical arm, The trolley includes a traveling motor and traveling wheels, which are used to drive the traveling wheels to move and thus drive the surgical robot to move. The trolley is provided with at least one binocular camera; the top of the trolley is connected to multiple surgical arms through a connecting rod. The surgical arms are three-axis surgical arms, and each surgical arm is provided with multiple key tracking points. The binocular camera is facing the multiple surgical arms, and is used to collect images of the multiple surgical arms; It also includes a central processing unit, which includes an image processing module, a data analysis module and an alarm module. The image processing module is used to use a convolutional neural network to process the stereo images collected by the binocular camera, and identify the surgical arm and key tracking points on the surgical arm; the data analysis module is used to calculate the three-dimensional position and motion trajectory of the surgical arm for the key tracking points identified by the image processing module, and use an anti-collision algorithm to evaluate the collision risk; the alarm module is used to issue an alarm when the distance between the surgical arm and the obstacle is less than the safe distance.
2. The binocular vision-based surgical robot anti-collision system according to claim 1, characterized in that: It also includes a display screen for issuing warnings to the operator through a user interface on the display screen based on the collision risk assessment results, and can display the real-time position, expected trajectory and potential collision area of the surgical arm.
3. The binocular vision-based surgical robot collision avoidance system according to claim 2, characterized in that: It also includes a voice broadcast module, which is used to remind the operator through voice broadcast when an alarm occurs.
4. The binocular vision-based surgical robot collision avoidance system according to claim 3, characterized in that: The number of the surgical arms is set to three, and the number of key tracking points on each surgical arm is set to three.
5. The binocular vision-based surgical robot anti-collision system according to claim 4, characterized in that: It also includes a communication module for wirelessly connecting the binocular camera, surgical arm, display screen and voice broadcast module to the central processing unit, and for wirelessly connecting the central processing unit to the remote terminal.
6. An anti-collision method for a surgical robot anti-collision system based on binocular vision according to any one of claims 1 to 5, characterized in that: The following steps are involved: S100, collecting stereoscopic image information of multiple surgical arms in the surgical area in real time using a binocular camera; S200, the image processing module is used to process the stereo image information collected by the binocular camera using a convolutional neural network to identify the surgical arm and key tracking points on the surgical arm; S300, the data analysis module calculates the three-dimensional position and motion trajectory of the surgical arm based on the key tracking points identified in step S200, and evaluates the collision risk; S400: When an alarm is triggered, an alarm signal is issued through a display screen or a voice alarm model, and a collision is avoided by controlling the robotic arm to stop moving or move in the reverse direction.
7. The anti-collision method of the binocular vision-based surgical robot anti-collision system according to claim 6, characterized in that: The anti-collision algorithm is based on the velocity Jacobian matrix to obtain the component of the motion velocity of the current key tracking point in the direction of the line connecting the same key points of the two adjacent manipulators, that is, the velocity vector in the C1 space and the acceleration vector along the C1C2 component, that is, 、 and the C2 velocity vector component towards C2C1 、 ; Emergency response time of human-operated equipment through large-scale sampling , calculate the minimum collision distance in real time:
8. Set a safety factor on the minimum collision distance S , and obtain the final safe distance , when the distance between any two points is less than the final safe distance The alarm is triggered.