Surgical robot mechanical arm control method based on binocular structured light camera and large model

Abstract

The invention provides an orthopedic surgery robot mechanical arm control method, system and device based on a binocular structured light camera and a large model and a computer readable storage medium. The control method for the mechanical arm of the orthopedic surgery robot based on the binocular structured light camera and the large model comprises the steps that video information of the surrounding environment is obtained in real time based on the binocular structured light camera of the orthopedic surgery robot; extracting depth information in the video information by using a large model, and determining a spatial three-dimensional coordinate of a target object in the surrounding environment of the orthopedic surgical robot; according to the spatial three-dimensional coordinates of the target object, motion path planning of a mechanical arm of the orthopedic surgery robot is conducted; and the mechanical arm is controlled to move according to the planned motion path. According to the embodiment of the invention, the mechanical arm can be controlled more accurately.

Description

Technical Field

[0001] The present application belongs to the field of robotic arm control, and in particular, relates to a method, system, device and computer-readable storage medium for controlling the robotic arm of an orthopedic surgical robot based on a binocular structured light camera and a large model. Background Art

[0002] At present, the intelligent manufacturing model represented by robotics technology is developing rapidly. Machine vision perception is a high-precision, non-contact, widely used technology that can greatly improve the automation and intelligence of production. However, traditional robotic arms use trial teaching or offline programming to control the robotic arms to complete processing, assembly, handling, grasping and other tasks according to prescribed actions. In complex and changing environments, the workpiece posture cannot be determined, and it is even impossible to locate the moving workpiece on the industrial production line in real time. Providing the workpiece posture to the robotic arm based on visual information and guiding the robotic arm to track the workpiece to complete related tasks can greatly improve the flexibility of the production process.

[0003] At present, many scholars at home and abroad have studied the tracking, grasping, and handling of robotic arms. The target tracking algorithm is the core of the robot to obtain the target position in real time to implement tracking control, and is divided into generative algorithms and discriminative algorithms. Generative algorithms mainly include the classic linear Kalman filter and its nonlinear improved algorithm, the Meanshift algorithm and its Camshift algorithm based on color features, and the particle filter algorithm and improved algorithm suitable for nonlinear non-Gaussian systems studied by many scholars; discriminative algorithms are the mainstream direction of target tracking algorithms in recent years, such as the correlation filter algorithm, which uses machine learning methods to classify the target and the background, train the classifier, and use the trained classifier to find the optimal target area in the next frame.

[0004] Han Jiuqiang et al. from Xi'an Jiaotong University disclosed "Visual robotic arm control device and method based on Camshift visual tracking and DH modeling algorithm", patent number 201510394465.4. Although the algorithm complexity is low using the traditional Camshift algorithm, the tracking effect is poor for objects with complex backgrounds or rich textures.

[0005] Therefore, how to control the robotic arm more accurately is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0006] The embodiments of the present application provide a method, system, device and computer-readable storage medium for controlling the manipulator arm of an orthopedic surgical robot based on a binocular structured light camera and a large model, which can more accurately control the manipulator arm.

[0007] In a first aspect, an embodiment of the present application provides a method for controlling an orthopedic surgical robot arm based on a binocular structured light camera and a large model, comprising:

[0008] Based on the binocular structured light camera of the orthopedic surgical robot, real-time video information of the surrounding environment is obtained;

[0009] The large model is used to extract depth information from the video information and determine the spatial three-dimensional coordinates of the target object in the environment surrounding the orthopedic surgical robot;

[0010] Plan the motion path of the robot arm of the orthopedic surgery robot according to the three-dimensional spatial coordinates of the target object;

[0011] Control the robot arm to move according to the planned motion path.

[0012] Optionally, before obtaining the video information of the surrounding environment in real time, the following is further included:

[0013] Calibrate the binocular structured light camera to determine the camera parameters;

[0014] Through hand-eye calibration, the spatial relationship between the camera coordinate system and the robotic arm coordinate system is determined.

[0015] Optionally, calibrate the binocular structured light camera, including:

[0016] Collect chessboard images: When calibrating the binocular structured light camera, the left and right cameras shoot at the same time. The chessboard needs to occupy more of the photo screen and be shot from multiple angles. The number of photos should be more than 5 pairs;

[0017] Use the stereoCalibrate function provided by Opencv to obtain the camera's intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficient;

[0018] Decompose the extrinsic parameter matrix solved by Opencv into the matrices of half rotation and half translation of the left and right cameras;

[0019] The pixel coordinate system of the source image is transformed into the camera coordinate system through the intrinsic parameter matrix, parallel epipolar correction is performed through the matrix, and the camera coordinates of the image are corrected through the distortion coefficient;

[0020] The corrected camera coordinate system is converted back into the image pixel coordinate system, and the new image coordinates are assigned according to the pixel values ​​of the source image coordinates.

[0021] Optionally, the large model is used to extract depth information from the video information to determine the spatial three-dimensional coordinates of the target object in the surrounding environment of the orthopedic surgical robot, including:

[0022] Use stereo matching algorithm to calculate the disparity map between the left and right camera images and perform hole filling to obtain more accurate disparity information;

[0023] The depth map is calculated through the disparity map. Each pixel value in the depth map represents the distance information of the corresponding point in the physical world.

[0024] Using the disparity map and transformation matrix, the two-dimensional image coordinates are converted into corresponding three-dimensional world coordinates to obtain the spatial three-dimensional coordinates of the target object.

[0025] Optionally, the motion path planning of the mechanical arm of the orthopedic surgery robot is performed according to the spatial three-dimensional coordinates of the target object, including:

[0026] Convert the spatial three-dimensional coordinates of the target object from the camera coordinate system to the robot arm coordinate system;

[0027] Select appropriate path planning algorithms based on the environment and task requirements, including search-based planning algorithms or sampling-based planning algorithms;

[0028] Using the selected path planning algorithm, a path from the starting point to the end point is planned according to the spatial three-dimensional coordinates of the target object and the current position of the robotic arm.

[0029] Optionally, controlling the robot arm to move according to the planned motion path includes:

[0030] Determine the speed and acceleration of each joint of the robot arm to ensure smooth and efficient movement;

[0031] Convert high-level motion plans into low-level motor control signals;

[0032] During the movement of the robot arm, its position and posture are monitored in real time;

[0033] When the robot arm reaches the target position, it controls the end effector to perform corresponding operations.

[0034] Optionally, also include:

[0035] During the movement of the robot arm, its position and posture are monitored in real time;

[0036] If you encounter an unexpected obstacle or abnormal situation, you need to stop immediately or change the path to ensure safety.

[0037] In a second aspect, the embodiment of the present application provides an orthopedic surgery robot arm control system based on a binocular structured light camera and a large model, including:

[0038] The video information acquisition module is used to acquire video information of the surrounding environment in real time based on the binocular structured light camera of the orthopedic surgical robot;

[0039] A depth information extraction module is used to extract depth information from video information using a large model to determine the spatial three-dimensional coordinates of a target object in the surrounding environment of the orthopedic surgical robot;

[0040] A motion path planning module, used to plan the motion path of the mechanical arm of the orthopedic surgery robot according to the spatial three-dimensional coordinates of the target object;

[0041] The robot control module is used to control the robot to move according to the planned motion path.

[0042] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;

[0043] When the processor executes the computer program instructions, an orthopedic surgery robot arm control method based on a binocular structured light camera and a large model is implemented.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, a method for controlling an orthopedic surgical robot arm based on a binocular structured light camera and a large model is implemented.

[0045] The orthopedic surgical robot arm control method, system, device and computer-readable storage medium based on a binocular structured light camera and a large model in the embodiments of the present application can control the robot arm more accurately.

[0046] The orthopedic surgery robot arm control method based on binocular structured light camera and large model includes:

[0047] Based on the binocular structured light camera of the orthopedic surgical robot, real-time video information of the surrounding environment is obtained;

[0048] The large model is used to extract depth information from the video information and determine the spatial three-dimensional coordinates of the target object in the environment surrounding the orthopedic surgical robot;

[0049] Plan the motion path of the robot arm of the orthopedic surgery robot according to the three-dimensional spatial coordinates of the target object;

[0050] Control the robot arm to move according to the planned motion path. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a flowchart of a method for controlling an orthopedic surgical robot arm based on a binocular structured light camera and a large model provided by an embodiment of the present application;

[0053] Figure 2 An orthopedic surgical robot based on a binocular structured light camera and a large model is provided in one embodiment of the present application;

[0054] Figure 3 It is a structural schematic diagram of an orthopedic surgical robot arm control system based on a binocular structured light camera and a large model provided by an embodiment of the present application;

[0055] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0056] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0057] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0058] In order to solve the problems of the prior art, the embodiments of the present application provide a method, system, device and computer-readable storage medium for controlling the mechanical arm of an orthopedic surgical robot based on a binocular structured light camera and a large model. The following first introduces the method for controlling the mechanical arm of an orthopedic surgical robot based on a binocular structured light camera and a large model provided in the embodiments of the present application.

[0059] Figure 1 A flow chart of a method for controlling an orthopedic surgical robot arm based on a binocular structured light camera and a large model provided in one embodiment of the present application is shown. Figure 2 An orthopedic surgical robot based on a binocular structured light camera and a large model is provided in one embodiment of the present application. Figure 1 As shown, the orthopedic surgery robot arm control method based on binocular structured light camera and large model includes:

[0060] S101, binocular structured light camera based on orthopedic surgical robot, real-time video information of the surrounding environment;

[0061] S102, using the large model to extract depth information from the video information, and determining the spatial three-dimensional coordinates of the target object in the surrounding environment of the orthopedic surgical robot;

[0062] S103, planning the motion path of the mechanical arm of the orthopedic surgery robot according to the spatial three-dimensional coordinates of the target object;

[0063] S104: Control the robot arm to move according to the planned motion path.

[0064] In one embodiment, before acquiring the video information of the surrounding environment in real time, the method further includes:

[0065] Calibrate the binocular structured light camera to determine the camera parameters;

[0066] Through hand-eye calibration, the spatial relationship between the camera coordinate system and the robotic arm coordinate system is determined.

[0067] In one embodiment, calibrating a binocular structured light camera includes:

[0068] Collect chessboard images: When calibrating the binocular structured light camera, the left and right cameras shoot at the same time. The chessboard needs to occupy more of the photo screen and be shot from multiple angles. The number of photos should be more than 5 pairs;

[0069] Use the stereoCalibrate function provided by Opencv to obtain the camera's intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficient;

[0070] Decompose the extrinsic parameter matrix solved by Opencv into the matrices of half rotation and half translation of the left and right cameras;

[0071] The pixel coordinate system of the source image is transformed into the camera coordinate system through the intrinsic parameter matrix, parallel epipolar correction is performed through the matrix, and the camera coordinates of the image are corrected through the distortion coefficient;

[0072] The corrected camera coordinate system is converted back into the image pixel coordinate system, and the new image coordinates are assigned according to the pixel values ​​of the source image coordinates.

[0073] In one embodiment, the large model is used to extract depth information from the video information to determine the spatial three-dimensional coordinates of the target object in the surrounding environment of the orthopedic surgical robot, including:

[0074] Use stereo matching algorithm to calculate the disparity map between the left and right camera images and perform hole filling to obtain more accurate disparity information;

[0075] The depth map is calculated through the disparity map. Each pixel value in the depth map represents the distance information of the corresponding point in the physical world.

[0076] Using the disparity map and transformation matrix, the two-dimensional image coordinates are converted into corresponding three-dimensional world coordinates to obtain the spatial three-dimensional coordinates of the target object.

[0077] In one embodiment, the motion path planning of the robot arm of the orthopedic surgery robot is performed according to the spatial three-dimensional coordinates of the target object, including:

[0078] Convert the spatial three-dimensional coordinates of the target object from the camera coordinate system to the robot arm coordinate system;

[0079] Select appropriate path planning algorithms based on the environment and task requirements, including search-based planning algorithms or sampling-based planning algorithms;

[0080] Using the selected path planning algorithm, a path from the starting point to the end point is planned according to the spatial three-dimensional coordinates of the target object and the current position of the robotic arm.

[0081] In one embodiment, controlling the robot arm to move according to the planned motion path includes:

[0082] Determine the speed and acceleration of each joint of the robot arm to ensure smooth and efficient movement;

[0083] Convert high-level motion plans into low-level motor control signals;

[0084] During the movement of the robot arm, its position and posture are monitored in real time;

[0085] When the robot arm reaches the target position, it controls the end effector to perform corresponding operations.

[0086] In one embodiment, it further includes:

[0087] During the movement of the robot arm, its position and posture are monitored in real time;

[0088] If you encounter an unexpected obstacle or abnormal situation, you need to stop immediately or change the path to ensure safety.

[0089] Figure 3 It is a structural schematic diagram of an orthopedic surgical robot arm control system based on a binocular structured light camera and a large model provided by an embodiment of the present application;

[0090] The orthopedic surgery robot arm control system based on binocular structured light camera and large model includes:

[0091] The video information acquisition module 201 is used to acquire video information of the surrounding environment in real time based on the binocular structured light camera of the orthopedic surgery robot;

[0092] A depth information extraction module 202 is used to extract depth information from the video information using the large model to determine the spatial three-dimensional coordinates of the target object in the surrounding environment of the orthopedic surgical robot;

[0093] A motion path planning module 203 is used to plan the motion path of the mechanical arm of the orthopedic surgery robot according to the spatial three-dimensional coordinates of the target object;

[0094] The robot arm control module 204 is used to control the robot arm to move according to the planned motion path.

[0095] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.

[0096] The electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0097] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0098] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 302 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 302 may be inside or outside the electronic device. In a particular embodiment, the memory 302 may be a non-volatile solid-state memory.

[0099] In one embodiment, the memory 302 may be a read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0100] The processor 301 reads and executes the computer program instructions stored in the memory 302 to implement any one of the orthopedic surgery robot arm control methods based on a binocular structured light camera and a large model in the above embodiments.

[0101] In one example, the electronic device may further include a communication interface 303 and a bus 310. Figure 4 As shown, the processor 301, the memory 302, and the communication interface 303 are connected via a bus 310 and communicate with each other.

[0102] The communication interface 303 is mainly used to implement communication between various modules, systems, units and / or devices in the embodiments of the present application.

[0103] Bus 310 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 310 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0104] In addition, in combination with the orthopedic surgery robot manipulator control method based on binocular structured light camera and large model in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the orthopedic surgery robot manipulator control methods based on binocular structured light camera and large model in the above embodiment is implemented.

[0105] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0106] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0107] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or systems. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0108] The above reference is according to the method of the embodiment of the present application, the flow chart of system (system) and computer program product and / or block diagram and describe various aspects of the present application.It should be understood that each square frame in the flow chart and / or block diagram and the combination of each square frame in the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer, special-purpose computer or other programmable data processing system, to produce a kind of machine, so that these instructions executed by the processor of computer or other programmable data processing system enable the realization of the function / action specified in one or more square frames of flow chart and / or block diagram.Such processor can be but not limited to general-purpose processor, special-purpose processor, special application processor or field programmable logic circuit.It can also be understood that each square frame in the block chart and / or flow chart and the combination of square frames in the block chart and / or flow chart can also be realized by the special-purpose hardware that performs the specified function or action, or can be realized by the combination of special-purpose hardware and computer instructions.

[0109] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for controlling an orthopedic surgical robot arm based on a binocular structured light camera and a large model, characterized in that: include: Based on the binocular structured light camera of the orthopedic surgical robot, real-time video information of the surrounding environment is obtained; The large model is used to extract depth information from the video information and determine the spatial three-dimensional coordinates of the target object in the environment surrounding the orthopedic surgical robot; Plan the motion path of the robot arm of the orthopedic surgery robot according to the three-dimensional spatial coordinates of the target object; Control the robot arm to move according to the planned motion path.

2. The method for controlling the mechanical arm of an orthopedic surgical robot based on a binocular structured light camera and a large model according to claim 1, characterized in that: Before obtaining the video information of the surrounding environment in real time, it also includes: Calibrate the binocular structured light camera to determine the camera parameters; Through hand-eye calibration, the spatial relationship between the camera coordinate system and the robotic arm coordinate system is determined.

3. The method for controlling the mechanical arm of an orthopedic surgical robot based on a binocular structured light camera and a large model according to claim 2, characterized in that: Calibrate the binocular structured light camera, including: Collect chessboard images: When calibrating the binocular structured light camera, the left and right cameras shoot at the same time. The chessboard needs to occupy more of the photo screen and be shot from multiple angles. The number of photos should be more than 5 pairs; Use the stereoCalibrate function provided by Opencv to obtain the camera's intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficient; Decompose the extrinsic parameter matrix solved by Opencv into the matrices of half rotation and half translation of the left and right cameras; The pixel coordinate system of the source image is transformed into the camera coordinate system through the intrinsic parameter matrix, parallel epipolar correction is performed through the matrix, and the camera coordinates of the image are corrected through the distortion coefficient; The corrected camera coordinate system is converted back into the image pixel coordinate system, and the new image coordinates are assigned according to the pixel values ​​of the source image coordinates.

4. The method for controlling the mechanical arm of an orthopedic surgical robot based on a binocular structured light camera and a large model according to claim 3, characterized in that: The large model is used to extract the depth information in the video information and determine the spatial three-dimensional coordinates of the target object in the surrounding environment of the orthopedic surgical robot, including: Use stereo matching algorithm to calculate the disparity map between the left and right camera images and perform hole filling to obtain more accurate disparity information; The depth map is calculated through the disparity map. Each pixel value in the depth map represents the distance information of the corresponding point in the physical world. Using the disparity map and transformation matrix, the two-dimensional image coordinates are converted into corresponding three-dimensional world coordinates to obtain the spatial three-dimensional coordinates of the target object.

5. The method for controlling the mechanical arm of an orthopedic surgical robot based on a binocular structured light camera and a large model according to claim 4, characterized in that: The motion path planning of the robot arm of the orthopedic surgery robot is performed according to the spatial three-dimensional coordinates of the target object, including: Convert the spatial three-dimensional coordinates of the target object from the camera coordinate system to the robot arm coordinate system; Select appropriate path planning algorithms based on the environment and task requirements, including search-based planning algorithms or sampling-based planning algorithms; Using the selected path planning algorithm, a path from the starting point to the end point is planned according to the spatial three-dimensional coordinates of the target object and the current position of the robotic arm.

6. The method for controlling the mechanical arm of an orthopedic surgical robot based on a binocular structured light camera and a large model according to claim 5, characterized in that: Control the robot arm to move according to the planned motion path, including: Determine the speed and acceleration of each joint of the robot arm to ensure smooth and efficient movement; Convert high-level motion plans into low-level motor control signals; During the movement of the robot arm, its position and posture are monitored in real time; When the robot arm reaches the target position, it controls the end effector to perform corresponding operations.

7. The method for controlling the mechanical arm of an orthopedic surgical robot based on a binocular structured light camera and a large model according to claim 6, characterized in that: Also includes: During the movement of the robot arm, its position and posture are monitored in real time; If you encounter an unexpected obstacle or abnormal situation, you need to stop immediately or change the path to ensure safety.

8. An orthopedic surgery robot arm control system based on a binocular structured light camera and a large model, characterized in that: The system comprises: The video information acquisition module is used to acquire video information of the surrounding environment in real time based on the binocular structured light camera of the orthopedic surgical robot; A depth information extraction module is used to extract depth information from video information using a large model to determine the spatial three-dimensional coordinates of a target object in the surrounding environment of the orthopedic surgical robot; A motion path planning module, used to plan the motion path of the mechanical arm of the orthopedic surgery robot according to the spatial three-dimensional coordinates of the target object; The robot control module is used to control the robot to move according to the planned motion path.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the orthopedic surgical robot arm control method based on a binocular structured light camera and a large model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the orthopedic surgical robot arm control method based on a binocular structured light camera and a large model as described in any one of claims 1 to 7.

Images