Surgical robotic pedal control system, method, readable medium, and surgical robot

Abstract

The application provides a surgical robot pedal control system, method, readable medium and surgical robot, and the surgical robot pedal control system comprises an image acquisition device and a control device; the image acquisition device is used for acquiring real-time three-dimensional images of a pedal of a surgical robot system and an adjacent area of the pedal, and transmitting the real-time three-dimensional images to the control device; the control device is configured to identify the pedal and a target object according to the acquired real-time three-dimensional images; and obtain real-time relative position information of the pedal and the target object in space based on the identified pedal and target object; the control device is further configured to judge according to the real-time relative position information, and if the target object is in a pre-trigger area of the pedal, switch a mapping relationship between the pedal and a slave device of the surgical robot system when a displacement amount of the target object in a preset direction reaches a threshold value. In this way, the same pedal can realize different functions, and the complexity and difficulty of pedal operation are simplified.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a surgical robot pedal control system, method, readable medium, and surgical robot. Background Technology

[0002] The application of surgical robot systems has addressed the clinical needs for minimally invasive and precise surgical procedures. Generally, a surgical robot system consists of a master device (such as the surgeon's console) and a slave device (such as the patient's surgical platform). During surgery, the surgeon observes the tissue characteristics within the patient's body through a two-dimensional or three-dimensional display at the surgeon's console and remotely controls the master hand and pedals on the console via a master-slave mapping. This drives the robotic arms and surgical instruments on the patient's surgical platform to complete the surgical procedure. Surgeons can perform the procedure in a manner and with similar sensations to traditional surgery, significantly reducing the difficulty of the procedure, improving efficiency and safety, and enabling groundbreaking progress in remote surgery. Surgery using surgical robot systems results in smaller incisions, less bleeding, and faster recovery for patients, greatly shortening postoperative hospital stays and significantly improving postoperative survival and recovery rates. It is favored by both doctors and patients and is now widely used in various clinical surgeries as a high-end medical device.

[0003] However, the pedal system of the surgeon's console in a surgical robot system typically includes multiple foot switches (e.g., six or more), each corresponding to a different function controlling the patient's surgical platform, such as electrocautery, electrocoagulation, switching of manipulator arms, and clutch engagement. Having so many foot switches is cumbersome to operate, and requires the surgeon to memorize the function of each switch, increasing the operational difficulty. Furthermore, the existing pedal system in surgical robot systems does not allow surgeons to visually identify the exact location of each foot switch during surgery, potentially leading to accidental presses and seriously compromising surgical safety. The need for surgeons to repeatedly check the pedal positions can also undermine their confidence and negatively impact their experience. Summary of the Invention

[0004] The purpose of this invention is to provide a surgical robot pedal control system, method, readable medium, and surgical robot to solve the problem of multiple foot switches in existing surgical robot systems, which leads to operational difficulties.

[0005] To solve the above-mentioned technical problems, the present invention provides a surgical robot pedal control system, which includes: an image acquisition device and a control device;

[0006] The image acquisition device is used to acquire real-time three-dimensional images of the pedals and adjacent areas of the surgical robot system, and transmit them to the control device.

[0007] The control device is configured to identify the pedal and the target object based on the acquired real-time three-dimensional image; and to obtain the real-time relative position information of the pedal and the target object in space based on the identified pedal and the target object.

[0008] The control device is further configured to, based on the real-time relative position information, determine whether the target object is within the pre-trigger area of ​​the pedal, and when the displacement of the target object along a preset direction reaches a threshold, switch the mapping relationship between the pedal and the slave device of the surgical robot system.

[0009] Optionally, in the surgical robot pedal control system, the pre-trigger area is a three-dimensional space area enclosed by a horizontal boundary, a first height boundary, and a second height boundary. The horizontal boundary is a boundary formed by extending horizontally with the pedal as the center. The first height boundary is higher than the second height boundary in the vertical direction. The first height boundary and the second height boundary are boundaries formed by extending vertically with the vertical height of the pedal's tread surface as a reference.

[0010] Optionally, in the surgical robot pedal control system, the control device is further configured to send an excitation signal to the master device when the pedal is pressed, so that the master device drives the corresponding instrument of the slave device to perform a corresponding operation according to the current mapping relationship.

[0011] Optionally, in the surgical robot pedal control system, the control device includes an image synthesis module, which is used to output a menu image corresponding to the current mapping relationship and / or a prompt message corresponding to the current mapping relationship to the display device of the main device.

[0012] Optionally, in the surgical robot pedal control system, if the target object is outside the pre-trigger area of ​​the pedal, the image synthesis module is used to hide the menu image.

[0013] Optionally, in the surgical robot pedal control system, the control device is further configured to store the current mapping relationship when the target object leaves the pre-trigger area of ​​the pedal; and to use the stored mapping relationship as the initial mapping relationship when the target object re-enters the pre-trigger area of ​​the pedal.

[0014] Optionally, in the surgical robot pedal control system, the control device is further configured to, when connected to the master device of the surgical robot system, acquire the current mapping relationship of the master device and use the acquired mapping relationship as the initial mapping relationship.

[0015] To address the aforementioned technical problems, the present invention also provides a surgical robot pedal control method, which is applied to the surgical robot pedal control system described above; the surgical robot pedal control method includes:

[0016] Acquire real-time three-dimensional images of the pedal and its adjacent area;

[0017] Based on the acquired real-time 3D image, the pedal and the target object are identified; and based on the identified pedal and the target object, the real-time relative position information of the pedal and the target object in space is obtained.

[0018] Based on the real-time relative position information, if the target object is within the pre-trigger area of ​​the pedal, then when the displacement of the target object along the preset direction reaches a threshold, the mapping relationship with the slave device of the surgical robot system is switched.

[0019] To address the aforementioned technical problems, the present invention also provides a readable storage medium storing a program thereon, which, when executed, implements the steps of the surgical robot pedal control method as described above.

[0020] To address the aforementioned technical problems, the present invention also provides a surgical robot system, which includes the surgical robot pedal control system described above, and further includes a master device. The master device includes a pedal, the preset direction is perpendicular to the triggering direction of the pedal, and the preset direction is perpendicular to the facing direction of the master device.

[0021] Optionally, in the surgical robot system, the master device includes up to two of the pedals.

[0022] In summary, the surgical robot pedal control system, surgical robot pedal control method, readable storage medium, and surgical robot system provided by this invention include: an image acquisition device and a control device; the image acquisition device is used to acquire real-time three-dimensional images of the pedal of the surgical robot system and the adjacent area of ​​the pedal, and transmit them to the control device; the control device is configured to identify the pedal and a target object based on the acquired real-time three-dimensional images; and based on the identified pedal and the target object, obtain real-time relative position information of the pedal and the target object in space; the control device is further configured to, based on the real-time relative position information, determine whether, if the target object is within the pre-trigger area of ​​the pedal, the mapping relationship between the pedal and the slave device of the surgical robot system is switched when the displacement of the target object along a preset direction reaches a threshold.

[0023] With this configuration, the operator can switch the mapping relationship between the pedal and the slave device by moving their foot in a preset direction, so that the same pedal can perform different functions. This reduces the number of physical pedals, simplifies the complexity and difficulty of pedal operation, and also avoids accidental stepping during surgery, thus improving the safety of the operation. Attached Figure Description

[0024] Those skilled in the art will understand that the accompanying drawings are provided to better understand the invention and do not constitute any limitation on the scope of the invention. Wherein:

[0025] Figure 1 This is a schematic diagram of a surgical robot system according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the doctor's console according to an embodiment of the present invention;

[0027] Figure 3 This is a hardware structure block diagram of the surgical robot pedal control system according to an embodiment of the present invention;

[0028] Figure 4 This is a software structure block diagram of the surgical robot pedal control system according to an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram illustrating the mapping relationship between the pedal and the slave device according to an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of the surgical robot pedal control system according to an embodiment of the present invention;

[0031] Figure 7a and Figure 7b This is a schematic diagram of the left and right movement of the foot according to an embodiment of the present invention;

[0032] Figure 8 This is a top view of the pre-trigger area according to an embodiment of the present invention;

[0033] Figures 9a-9c This is a side view showing the relative relationship between the foot and the pre-trigger area in an embodiment of the present invention;

[0034] Figure 10 This is a schematic diagram illustrating the display status of a display device according to an embodiment of the present invention;

[0035] Figure 11 This is a schematic diagram illustrating the switching of menu images according to an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, advantages, and features of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to facilitate and clarify the explanation of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structures. In particular, different figures may emphasize different aspects and may sometimes use different scales.

[0037] As used in this invention, the singular forms “a,” “an,” and “the” include plural objects; the term “or” is generally used to mean “and / or”; the term “a number” is generally used to mean “at least one”; and the term “at least two” is generally used to mean “two or more”. Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first,” “second,” or “third” may explicitly or implicitly include one or at least two of that feature; “one end” and “the other end,” and “proximal end” and “distal end” generally refer to two corresponding parts, which include not only endpoints. Furthermore, the terms "installed," "connected," and "attached," as used in this invention, and the term "set" on one element from another, should be interpreted broadly. They generally only indicate a connection, coupling, cooperation, or transmission relationship between the two elements, which can be direct or indirect through an intermediate element. They should not be construed as indicating or implying a spatial relationship between the two elements, meaning one element can be located inside, outside, above, below, or to one side of another element, unless otherwise explicitly stated. Those skilled in the art can understand the specific meaning of these terms in this invention based on the specific circumstances. Additionally, directional terms such as above, below, up, down, upward, downward, left, and right are used relative to exemplary embodiments as shown in the figures, with upward or upper directions pointing towards the top of the corresponding figure, and downward or lower directions pointing towards the bottom of the corresponding figure.

[0038] The purpose of this invention is to provide a surgical robot pedal control system, control method, readable medium, and surgical robot to solve the problem of multiple foot switches in existing surgical robot systems, which leads to operational difficulties. The following description refers to the accompanying drawings.

[0039] This invention provides a surgical robot system. Figure 1 and Figure 2 The application scenario of the surgical robot system is illustrated. The surgical robot system includes a master-slave teleoperated surgical robot, specifically, the surgical robot system includes a master device 100 (e.g., a doctor's console), a slave device 200 (e.g., a patient surgical platform), and a support device 400 (e.g., an operating table) for supporting the surgical object (e.g., a patient) during surgery. The master device 100 includes a doctor's host 111 (…). Figure 1 Not shown in the image, see Figure 3 and Figure 4 ) and visual host 112 ( Figure 1 Not shown in the image, see Figure 3 and Figure 4It should be noted that in some embodiments, the support device 400 may be replaced with other surgical operating platforms, and the present invention is not limited thereto.

[0040] like Figure 2 As shown, the master device 100 is the operating end of a teleoperated surgical robot, and includes a master hand 101 and several pedals 500 mounted thereon. The master hand 101 is used to receive hand movement information from the operator (e.g., a doctor), and the pedals 500 are used to receive foot movement information from the operator to complete the input of related operation commands such as clutch engagement, electrocautery, electrocoagulation, and endoscopic movement control. Preferably, the master device 100 also includes a display device 102, which is connected to a vision host 112. The vision host 112 is also connected to the endoscope of the slave device 200 to acquire surgical field images of the cavity (referring to the patient's body cavity) captured by the endoscope. Furthermore, the vision host 112 is used to perform image processing on the surgical field images acquired by the endoscope and transmit them to the display device 102 for display, so that the operator can observe the surgical field images. The surgical field images include the type and number of surgical instruments, their position in the abdomen, the morphology and arrangement of the patient's organs and tissues, and the blood vessels of the surrounding organs and tissues. It should be understood that the image displayed by the display device 102 can be a two-dimensional or three-dimensional image.

[0041] The slave device 200 is the specific execution platform for the remotely operated surgical robot, including a base 201 and a surgical execution component mounted thereon. The surgical execution component includes an instrument arm 210 and an instrument 221, with the instrument 221 mounted or connected to the end of the instrument arm 210. Further, the instrument 221 includes surgical instruments for performing specific surgical operations and endoscopes for auxiliary observation. In one embodiment, the surgical instruments are used to perform specific surgical operations, such as clamping, cutting, and scissing.

[0042] The doctor-side host 111 is communicatively connected to the slave device 200. The doctor-side host 111 is used to control the movement of the surgical execution component based on the movement of the master manipulator 101 and the pedal 500. Specifically, the doctor-side host 111 includes a master-slave mapping module 113. The master-slave mapping module 113 is used to acquire the end-effector pose information of the master manipulator 101, obtain the desired end-effector pose of the surgical execution component according to a predetermined mapping relationship, and then control the instrument arm 210 to drive the instrument to the desired end-effector pose. The master-slave mapping module 113 is also used to receive the pedal 500's pedal input information, and obtain instrument function operation instructions (such as electrocautery, electrocoagulation, and other related operation instructions) according to a predetermined mapping relationship, and control the energy driver of the surgical instrument 221 to release energy to realize surgical operations such as electrocautery and electrocoagulation.

[0043] Furthermore, the surgical robot system also includes an image carriage 300. Optionally, the image carriage 300 further includes an auxiliary display screen 302. The auxiliary display screen 302 is communicatively connected to the vision host 112 and is used to provide real-time display of surgical field images or other auxiliary display information to the assistant operator (e.g., a nurse).

[0044] Optionally, in some surgical applications, the surgical robot system may also include auxiliary components such as a ventilator and anesthesia machine 410 and an instrument table 420 for use during surgery. Those skilled in the art can select and configure these auxiliary components according to existing technology, which will not be described in detail here.

[0045] It should be noted that the surgical robot system disclosed in the above examples is only an example of an application scenario and not a limitation on the application scenario of the surgical robot system. The surgical robot system is not limited to a master-slave teleoperated surgical robot, but can also be a single-end surgical robot system in which the operator directly operates the surgical robot to perform surgery. This invention is not limited to this.

[0046] To address the issue of an excessive number of pedals 500 included in the main device 100 in the existing technology, please refer to... Figures 3 to 11This invention provides a surgical robot pedal control system, comprising: an image acquisition device 600 and a control device 700; the image acquisition device 600 is used to acquire real-time three-dimensional images of the pedal 500 and its adjacent areas, and transmit them to the control device 700; the control device 700 is configured to identify the pedal 500 and a target object based on the acquired real-time three-dimensional images; and based on the identified pedal 500 and the target object, obtain real-time relative position information of the pedal 500 and the target object in space; the control device 700 is further configured to, based on the real-time relative position information, determine whether, if the target object is within a pre-trigger area of ​​the pedal 500, the mapping relationship between the pedal 500 and the slave device 200 of the surgical robot system is switched when the displacement of the target object along a preset direction reaches a threshold. It should be noted that the target object here can refer to the operator's foot, or it can be an object such as a foot prosthesis model used for calibration and testing; this embodiment is not limited to this. For ease of explanation, the following description uses the operator's foot as an example. The pre-trigger area refers to a certain spatial range adjacent to the pedal 500. When the foot moves within this pre-trigger area, it will be captured by the image acquisition device 600 and monitored by the control device 700. If the displacement of the foot in the pre-trigger area along a preset direction reaches a threshold, the control device 700 outputs pedal function switching information to the master-slave mapping module 113, thereby switching the mapping relationship between the pedal 500 and the slave device 200. It can be understood that parameters such as the pre-trigger area, preset direction, and displacement threshold can be set according to the actual size and layout of the pedal 500, and the pre-trigger area should be completely within the shooting range of the image acquisition device 600.

[0047] With this configuration, the operator can switch the mapping relationship between the pedal 500 and the slave device 200 by moving their foot in a preset direction. This allows the same pedal 500 to perform different functions, reducing the number of physical pedals, simplifying the complexity and difficulty of operating the pedal 500, and also preventing accidental stepping during surgery, thus improving the safety of the surgery.

[0048] Please refer to Figure 3 and Figure 4 This exemplifies the hardware and software structure of the surgical robot pedal control system provided in this embodiment. Please refer to the references. Figure 6The image acquisition device 600, for example, is a structured light depth camera, used to capture real-time 3D images of the area enclosed by the bottom support of the main device 100. Understandably, this area enclosed by the bottom support of the main device 100 includes the pedal 500 and its adjacent area. Optionally, the image acquisition device 600 is connected to the control device 700 via a USB cable. The pedal 500 is preferably detachably connected to the control device 700, for example, via a USB cable. The pedal 500 is configured to send pedaling information to the control device 700 when it is pressed. Optionally, in Figure 3 and Figure 4 In the illustrated example, the control device 700, as an integrated hardware device independent of the host device 100, is detachably connected to the doctor's host 111, for example, via a network cable. Optionally, the control device 700 can be detachably connected to the vision host 112 and the display device 102, respectively, such as via an HDMI cable. The control device 700 processes real-time 3D images from the vision host 112 and the image acquisition device 600, and synthesizes new images for output to the display device 102 for display.

[0049] Please refer to Figure 4 The software structure of the surgical robot pedal control system includes a foot position and posture recognition module 701, a foot pedal configuration module 702, a foot pedal indication activation module 703, and an image synthesis module 704. These modules can be integrated into the control device 700 in terms of hardware. Real-time three-dimensional images acquired by the image acquisition device 600 are input to the foot position and posture recognition module 701. The foot position and posture recognition module 701 identifies the position of the foot and pedal 500, as well as the foot's movement posture. Based on the threshold configured by the foot pedal configuration module 702, it outputs a signal to the foot pedal indication activation module 703. The foot pedal indication activation module 703, based on the signal sent by the foot position and posture recognition module 701, determines whether the foot is within the pre-trigger area and whether the displacement of the foot along a preset direction reaches the threshold. If so, it obtains pedal function switching information.

[0050] Furthermore, the foot pedal indicator activation module 703 sends pedal function switching information to the master-slave mapping module 113 on one hand, and sends a menu image 810 corresponding to the current mapping relationship and / or activation prompt information 820 corresponding to the current mapping relationship to the image synthesis module 704 on the other hand. The master-slave mapping module 113 is mainly used to update and switch the original pedal processing logic and the various functions of the indicator module menu according to the received pedal function switching information.

[0051] In one aspect, the control device 700 is further configured to send an activation signal to the master device 100 when the pedal 500 is depressed, so that the master device 100 drives the corresponding instrument of the slave device 200 to perform a corresponding operation according to the current mapping relationship. Specifically, when the pedal 500 is depressed, a pedal signal is sent to the control device 700, and the control device 700 then sends an activation signal to the master-slave mapping module 113 of the master device 100 according to the pedal signal, thereby driving the corresponding instrument of the slave device 200 to perform a corresponding operation according to the current mapping relationship.

[0052] In another aspect, the image synthesis module 704 is used to output the menu image 810 corresponding to the current mapping relationship and / or the activation prompt information 820 corresponding to the current mapping relationship to the display device 102 of the master device 100. Optionally, the image synthesis module 704 is also used to acquire the surgical field image from the vision host 112, and synthesize the menu image 810, the activation prompt information 820, and the surgical field image, and finally output them to the display device 102 for display. Optionally, the foot pedal configuration module 702 can be displayed on the display device 102, for example, the threshold can be set in the settings page on the doctor's end.

[0053] Please refer to Figure 5 This illustrates an example of the mapping relationship between pedal 500 and slave device 200. Solid line 114 indicates that when pedal 500 is depressed, the control device 700 and master-slave mapping module 113 will drive the first arm of slave device 200 to perform an electrical cut. The remaining dashed lines 115 indicate the selectable functions of slave device 200 when pedal 500 is depressed. It can be understood that, based on the pedal function switching information, the function mapped when pedal 500 is depressed can be selected from the various selectable functions of slave device 200.

[0054] Optionally, the control device 700 is further configured to store the current mapping relationship when the target object leaves the pre-trigger area of ​​the pedal 500; and to use the stored mapping relationship as the initial mapping relationship when the target object re-enters the pre-trigger area of ​​the pedal 500. When the operator's foot leaves the pre-trigger area and re-enters it, for ease of operation, the mapping relationship selected when the foot leaves can be stored and recorded. When the operator's foot re-enters the pre-trigger area, the stored mapping relationship is directly used as the current initial mapping relationship. In some embodiments, the master device 100 can store the mapping relationship when the foot leaves the pre-trigger area, and when the operator's foot re-enters the pre-trigger area, the control device 700 retrieves the stored mapping relationship from the master device 100. Of course, in other embodiments, the control device 700 can also store the mapping relationship when the foot leaves the pre-trigger area; this embodiment is not limited to this.

[0055] Furthermore, in some embodiments, the control device 700 is detachably connected to the master device 100. The control device 700 is also configured to, upon connecting to the master device 100 of the surgical robot system, acquire the current mapping relationship of the master device 100 and use the acquired mapping relationship as the initial mapping relationship. Since the control device 700 and the master device 100 are detachably connected, they need to be matched and synchronized when connecting. In one example, the control device 700 can initiate a connection request to the doctor's host 111 of the master device 100. The doctor's host 111 listens for connections from the control device 700. If a connection is established, it sends the current mapping relationship of the master device 100 to the control device 700. Upon receiving this mapping relationship, the control device 700 uses it as the current initial mapping relationship. Furthermore, the image synthesis module 704 can obtain the corresponding menu image 810 and / or trigger prompt information 820 according to the initial mapping relationship, and then process, re-render and output to the display device 102 for display.

[0056] Please refer to the following. Figures 6 to 7b It illustrates an example of a surgical robot pedal control system. Figure 6 In the illustrated example, the master device 100 includes two pedals 500, preferably corresponding to the operator's two feet respectively, which may correspond to a portion of the optional functions of the slave device 200.

[0057] Preferably, the preset direction is perpendicular to the triggering direction of the pedal 500, and the preset direction is perpendicular to the facing direction of the main device 100. Please refer to the reference. Figure 1 and Figure 2In one example, the main device 100 is a doctor's console. The operator preferably faces the console while seated, meaning the main device 100 faces the operator. The trigger direction of the pedal 500 refers to the direction in which the pedal 500 is activated when it is pressed. Figure 1 and Figure 2 In the example, this is a vertical direction or a direction slightly angled to the vertical (e.g., tilting forward or backward, or tilting left or right). Therefore, the preset direction extends along the left-right direction of the main device 100. Please refer to the reference. Figure 7a and Figure 7b Since the operator is preferably seated, the left and right movement of the feet is very convenient and it is not easy for the foot position and posture recognition module 701 to misidentify. Of course, in some other embodiments, the preset direction can also be set along the vertical direction or along the facing direction of the main device 100 (i.e., the front and back direction). This embodiment does not limit this, and those skilled in the art can set it according to the actual needs of the operator.

[0058] Furthermore, to improve the robustness of the surgical robot's pedal control system and avoid false triggering, the foot position and posture recognition module 701 can integrate a foot recognition unit. The foot recognition unit is used to identify features contained in the real-time 3D image to confirm whether certain features belong to the operator's feet, preventing similar objects from entering the adjacent area of ​​the pedal 500 and causing false triggering. The foot recognition unit can employ some existing image recognition algorithms, such as the SURF algorithm, which can be understood by those skilled in the art. Optionally, the foot recognition unit may include, for example, a trained foot model library. The recognition steps of the foot recognition unit include: recognition step 1: acquiring the real-time 3D image acquired by the image acquisition device 600; recognition step 2: extracting features from the real-time 3D image; recognition step 3: matching features with the trained foot model library and outputting the recognition result. The foot model library can be trained in advance based on a large amount of foot data. The specific training algorithm can refer to existing technologies, such as using neural network algorithms, which will not be elaborated in this embodiment.

[0059] Optionally, for ease of understanding, in the following example, the target object (the operator's foot) is abstracted as recognition point A located at the front of the foot. It is understood that in practice, any point, any area, or the entire foot can be used as a recognition point or recognition area for comparison and calculation; this embodiment is not limited to this.

[0060] Please refer to Figure 6In one example, the image acquisition device 600 is mounted on the column 104 of the main device 100. Optionally, the image acquisition device 600 is a structured light depth camera. The image acquisition device 600 includes an infrared projector 610, an infrared camera 620, and a regular camera 630 arranged sequentially at intervals. The infrared projector 610 is used to project infrared structured light onto the target object (such as the pedal 500 and feet), the infrared camera 620 is used to capture a first image reflecting infrared structured light information, and the regular camera 630 is used to capture a second image reflecting visible light information. The origin O is defined by the intersection of the left and inner supports with the ground (see reference). Figure 2 and Figure 6 A three-dimensional spatial coordinate system is established, with the left-right direction (length direction) of the main device 100 as the X-axis, the front-back direction (width direction) of the main device 100 as the Y-axis, and the vertical direction (height direction) as the Z-axis. Since the distance between the infrared camera 620 and the ordinary camera 630 is known, by superimposing the first and second images, and based on the two-dimensional coordinates of a point in space in the first and second images, depth information is calculated according to the principle of triangulation, thus obtaining the coordinates of that point in the three-dimensional spatial coordinate system. For the specific measurement principle, please refer to existing technologies; it will not be elaborated here.

[0061] Optional, please refer to Figures 8 to 9c The pre-trigger area is a three-dimensional space area enclosed by a horizontal boundary 710, a first height boundary 721, and a second height boundary 722. The horizontal boundary 710 is a boundary formed by extending horizontally with the pedal 500 as the center. The first height boundary 721 is higher than the second height boundary 722 in the vertical direction. The first height boundary 721 and the second height boundary 722 are boundaries formed by extending vertically with the vertical height of the pedal surface of the pedal 500 as the reference.

[0062] exist Figures 8 to 9c In the illustrated example, the left pedal 500 is used as an example. The coordinate range of the left pedal 500 is (X15, X25), (Y40, Y55), Z20; the coordinate range of the horizontal boundary 710 is (X0, X40), (Y35, Y70); the coordinates of the first height boundary 721 are Z50, and the coordinates of the second height boundary 722 are Z20. Taking the recognition point A at the front of the foot as an example, the foot position and posture recognition module 701 detects the movement of recognition point A. When the three-dimensional coordinates of recognition point A are located within the pre-trigger area, the foot pedal indication activation module 703 determines that the foot is within the pre-trigger area. Then, in conjunction with the reference... Figure 7a and Figure 7bIf the identification point A moves along a preset direction (such as the left-right direction of the master device 100) and the displacement W reaches a threshold, the foot pedal indication activation module 703 determines that the operator's intention is to switch the mapping relationship, that is, obtains the pedal function switching information, and sends the pedal function switching information to the master-slave mapping module 113. Optionally, the specific ranges of the horizontal boundary 710, the first height boundary 721, and the second height boundary 722 can be configured through the foot pedal configuration module 702.

[0063] Furthermore, to more clearly and intuitively reflect the mapping relationship switched by foot movement, please refer to... Figure 10 The example illustrates a menu image 810 and a trigger prompt message 820 corresponding to the mapping relationship. For the current pedal 500, the selectable functions corresponding to the slave device 200 are functions one through four. The menu image 810 displays functions one through four, with function one currently selected for mapping. This function can be displayed, for example, using dark font or a selection box. Unselected functions two through four can be displayed using light font.

[0064] Furthermore, the trigger prompt information 820 includes four prompt areas ① to ④, which correspond to functions one through four, respectively. When the pedal 500 is pressed, the prompt information is triggered according to the currently selected mapping relationship (…). Figure 10 (For Function 1), the corresponding instrument of the slave device 200 is activated and performs the corresponding operation. At the same time, the prompt area ① corresponding to Function 1 will be activated to prompt, for example, the prompt area ① can flash or be displayed in red, to prompt the operator that the instrument corresponding to the current Function 1 is being activated and performing the corresponding operation.

[0065] In an example of changing the mapping relationship, for instance, when the foot moves to the left and the displacement reaches a threshold, the selected mapping function switches from function one to function two. Correspondingly, the menu content moves to the left, placing function two at the selected position, and is displayed using dark font or a selection box, such as... Figure 11 As shown. It is understandable that this corresponds to... Figure 11 The mapping relationship means that when pedal 500 is pressed, the corresponding prompt area ② for function two will be activated. Of course... Figure 10 and Figure 11 The menu image 810 and the trigger prompt information 820 shown are merely examples and not limitations on the menu image 810 and the trigger prompt information 820.

[0066] Preferably, if the target object is outside the pre-trigger area of ​​the pedal 500, the image compositing module 704 hides the menu image 810. To alert the operator, the menu image 810 can be hidden when the foot is outside the pre-trigger area, and displayed only when the foot is within the pre-trigger area. Figures 9a to 9c As shown in the example, Figure 9a In the image, the Z-axis coordinate of identification point A is Z60, which is above the coordinate Z50 of the first height boundary 721 of the pre-trigger area. This indicates that the operator's foot is raised to a very high height, exceeding the height of the pre-trigger area. At this time, the menu image 810 is hidden. Even if the operator moves his foot left or right, the mapping relationship will not change. Figure 9b The Z-axis coordinate of the identification point A is located between the coordinates Z50 of the first height boundary 721 and the coordinates Z20 of the second height boundary 722. At this time, the menu image 810 is displayed on the display device 102. If the displacement of the operator's foot along the preset direction reaches the threshold, the mapping relationship will be changed. Figure 9c When the operator's foot is placed down, the Z-axis coordinate of the identification point A is Z15, which is lower than the coordinate Z20 of the second height boundary 722. At this time, the operator's foot cannot and does not need to move left or right, and the menu image 810 is hidden.

[0067] Preferably, the master device 100 includes at most two pedals 500. To simplify the structure of the surgical robot pedal control system, at most two pedals 500 are provided corresponding to the operator's two feet, with each pedal 500 mapping to a portion of the functions. This effectively solves the problem of multiple foot switches in existing surgical robot systems, which leads to operational difficulties. Furthermore, the inclusion of pedals 500, compared to completely eliminating pedals and integrating all existing pedal functions onto the operating handle, reduces the operator's hand workload and does not affect the user experience. Of course, in some embodiments, the surgical robot pedal control system may also include only one pedal 500, which can map all the optional functions of the slave device 200.

[0068] This invention also provides a surgical robot pedal control method, which is applied to the surgical robot pedal control system described above; the surgical robot pedal control method includes:

[0069] Step S1: Obtain real-time three-dimensional images of the pedal 500 and its adjacent areas;

[0070] Step S2: Based on the acquired real-time 3D image, identify the pedal 500 and the target object; and based on the identified pedal 500 and the target object, obtain the real-time relative position information of the pedal 500 and the target object in space;

[0071] Step S3: Based on the real-time relative position information, if the target object is within the pre-trigger area of ​​the pedal 500, then when the displacement of the target object along the preset direction reaches a threshold, switch the mapping relationship between the pedal 500 and the slave device 200 of the surgical robot system.

[0072] The specific principles and execution process of this method can be found in the above description of the surgical robot pedal control system, and will not be repeated here.

[0073] Furthermore, embodiments of the present invention also provide a readable storage medium storing a program thereon, which, when executed, implements the steps of the surgical robot pedal control method described above. Even further, embodiments of the present invention also provide a computer device including a processor and the readable storage medium described above, the processor being used to execute the program stored on the readable storage medium. The readable storage medium can be set independently or integrated into the surgical robot system; the present invention is not limited in this regard. Still further, embodiments of the present invention also provide a surgical robot system including the surgical robot pedal indication system described above. The structure and principle of other components of the surgical robot system can be referred to the prior art, and the present invention will not elaborate on them one by one.

[0074] In summary, the surgical robot pedal control system, surgical robot pedal control method, readable storage medium, and surgical robot system provided by this invention include: an image acquisition device and a control device; the image acquisition device is used to acquire real-time three-dimensional images of the pedal of the surgical robot system and the adjacent area of ​​the pedal, and transmit them to the control device; the control device is configured to identify the pedal and a target object based on the acquired real-time three-dimensional images; and based on the identified pedal and the target object, obtain real-time relative position information of the pedal and the target object in space; the control device is further configured to, based on the real-time relative position information, determine whether, if the target object is within the pre-trigger area of ​​the pedal, the mapping relationship between the pedal and the slave device of the surgical robot system is switched when the displacement of the target object along a preset direction reaches a threshold.

[0075] With this configuration, the operator can switch the mapping relationship between the pedal and the slave device by moving their foot in a preset direction, so that the same pedal can perform different functions. This reduces the number of physical pedals, simplifies the complexity and difficulty of pedal operation, and also avoids accidental stepping during surgery, thus improving the safety of the operation.

[0076] It should be noted that the above embodiments can be combined with each other. The above description is only a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A surgical robot pedal control system, characterized in that, include: Image acquisition device and control device; The image acquisition device is used to acquire real-time three-dimensional images of the pedal of the surgical robot system and the adjacent area of ​​the pedal, and transmit them to the control device. The control device is configured to identify the pedal and the target object based on the acquired real-time three-dimensional image; and to obtain the real-time relative position information of the pedal and the target object in space based on the identified pedal and the target object. The control device is further configured to, based on the real-time relative position information, determine whether the target object is within the pre-trigger area of ​​the pedal, and when the displacement of the target object along a preset direction reaches a threshold, switch the mapping relationship between the pedal and the slave device of the surgical robot system.

2. The surgical robot pedal control system according to claim 1, characterized in that, The pre-trigger area is a three-dimensional space area enclosed by a horizontal boundary, a first height boundary, and a second height boundary. The horizontal boundary is a boundary formed by extending horizontally with the pedal as the center. The first height boundary is higher than the second height boundary in the vertical direction. The first height boundary and the second height boundary are boundaries formed by extending vertically with the vertical height of the pedal's tread surface as a reference.

3. The surgical robot pedal control system according to claim 1, characterized in that, The control device is also configured to send an excitation signal to the master device when the pedal is pressed, so that the master device drives the corresponding instrument of the slave device to perform a corresponding operation according to the current mapping relationship.

4. The surgical robot pedal control system according to claim 1, characterized in that, The control device includes an image synthesis module, which is used to output a menu image corresponding to the current mapping relationship and / or a prompt message corresponding to the current mapping relationship to the display device of the main device.

5. The surgical robot pedal control system according to claim 4, characterized in that, If the target object is outside the pre-trigger area of ​​the pedal, the image synthesis module is used to hide the menu image.

6. The surgical robot pedal control system according to claim 1, characterized in that, The control device is further configured to store the current mapping relationship when the target object leaves the pre-trigger area of ​​the pedal; and to use the stored mapping relationship as the initial mapping relationship when the target object re-enters the pre-trigger area of ​​the pedal.

7. The surgical robot pedal control system according to claim 1, characterized in that, The control device is also configured to, when connected to the master device of the surgical robot system, acquire the current mapping relationship of the master device and use the acquired mapping relationship as the initial mapping relationship.

8. A method for controlling the pedals of a surgical robot, characterized in that, The surgical robot pedal control system is applied to any one of claims 1 to 7; the surgical robot pedal control method includes: Acquire real-time three-dimensional images of the pedal and its adjacent area; Based on the acquired real-time 3D image, the pedal and the target object are identified; and based on the identified pedal and the target object, the real-time relative position information of the pedal and the target object in space is obtained. Based on the real-time relative position information, if the target object is within the pre-trigger area of ​​the pedal, then when the displacement of the target object along the preset direction reaches a threshold, the mapping relationship between the pedal and the slave device of the surgical robot system is switched.

9. A readable storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the steps of the surgical robot pedal control method according to claim 8.

10. A surgical robot system, characterized in that, The surgical robot pedal control system according to any one of claims 1 to 7 further includes a master device, the master device including a pedal, the preset direction being perpendicular to the triggering direction of the pedal, and the preset direction being perpendicular to the facing direction of the master device.

11. The surgical robot system according to claim 10, characterized in that, The master device includes at most two pedals.

Images