Oral surgical instrument automatic mounting system, method, medium, program product, and terminal

Abstract

The application provides an oral surgery instrument automatic installation system, method, medium, program product and terminal, which comprises a cooperative robot and a control module; the cooperative robot is connected with the control module; the cooperative robot comprises an image mechanical arm and an instrument mechanical arm; the front end of the instrument mechanical arm is used for clamping and fixing an oral surgery instrument; and the cooperative robot moves the oral surgery instrument to a target installation position for fixed installation according to a real-time adjusted path planning. The application can realize full-automatic, high-precision, safe and reliable positioning, installation and fixation of an oral surgery instrument (such as an endoscope, a special instrument channel pipe, a retractor and the like) required in oral surgery (such as oral thyroid surgery, oral throat surgery and the like) in the oral cavity of a patient.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of surgical robots, in particular to an automatic installation system and method for transoral surgical instruments, a medium, a program product and a terminal. BACKGROUND

[0002] Transoral surgery, such as transoral thyroid surgery, transoral laryngopharyngeal surgery, etc., as a minimally invasive and unique surgical method, has gradually gained attention in clinical application in recent years. This surgical method takes advantage of the natural cavity of the oral cavity to perform surgical operations, avoiding complications such as scarring and nerve damage caused by traditional neck incisions, and has the advantages of minimal trauma, rapid recovery, and high aesthetic value.

[0003] During transoral surgery, endoscopes, special instrument channel tubes, retractors and other core transoral surgical instruments play a crucial role. Endoscopes provide a clear surgical field for the operation, allowing the surgeon to accurately observe the anatomical structure and lesion of the surgical site; special instrument channel tubes provide a stable access channel for surgical instruments, ensuring the precision and continuity of surgical operations; retractors are used to spread the oral tissue and expand the surgical space, creating convenient conditions for the smooth operation of surgical instruments.

[0004] Robots have gradually entered the field of view of people due to their strong repeatability, high positioning accuracy, and resistance to external environmental influences. Medical robots, which integrate advanced and cutting-edge disciplines such as robotics, automation, computer science, and materials science, have become an international research hotspot in the field of robotics. Many advanced robotic technologies are currently being applied in medicine, such as preoperative planning, minimally invasive surgery, transoral surgery, and new treatment methods. This has not only promoted the development of medicine but also updated the technology of robotics.

[0005] Currently, the installation of transoral surgical instruments mainly relies on manual operation by doctors or operation of surgical robots. However, manual operation has certain limitations, as it is affected by factors such as the stability of the operator's hands, operating experience, and fatigue level. Existing surgical robots mostly rely on single-source sensor data installed at the end of the robot for positioning, resulting in poor positioning accuracy and low operating safety. In addition, the complexity of the surgical area environment and the complexity of the internal structure of the oral cavity need to be considered. Therefore, there is an urgent need for an automatic installation system and method for transoral surgical instruments for transoral surgery to provide high-precision real-time navigation and positioning for collaborative robots. SUMMARY

[0006] In view of the above-mentioned disadvantages of the prior art, the present application provides an oral surgery instrument automatic installation system, method, medium, program product and terminal, which is used to solve the problems of complexity of the operation area environment in oral surgery, complexity of the internal structure of the oral cavity, low positioning accuracy of manual or robotic installation of the oral surgery instrument, poor operation safety and the like.

[0007] To achieve the above-mentioned objects and other related objects, the first aspect of the present application provides an oral surgery instrument automatic installation system, comprising: a cooperative robot and a control module; the cooperative robot is connected with the control module; the cooperative robot comprises an image mechanical arm and an instrument mechanical arm; the front end of the instrument mechanical arm is used to clamp and fix an oral surgery instrument; wherein the control module comprises: a model construction unit, which is used to acquire preoperative oral scan image data of a patient based on a scanning device, and construct a three-dimensional model of the oral cavity of the patient according to the preoperative oral scan image data; a bracket customization unit, which is used to manufacture an oral customized bracket of the patient based on the three-dimensional model of the oral cavity, and set a reference mark and a target installation position of the oral surgery instrument on the oral customized bracket; a space registration unit, which is used to perform space registration based on the preoperative oral scan image data of the patient, multi-source sensor data and actual anatomical structure data of the patient, to obtain a space registration result; a navigation installation unit, which is used to perform path planning based on the space registration result in combination with the target installation position of the oral surgery instrument, and perform real-time detection and positioning of the cooperative robot in the installation process through an optical positioning device, and collect force feedback data in real time according to a force feedback device, and perform analysis based on the real-time detection and positioning result and the force feedback data to perform real-time adjustment of the path planning, so that the cooperative robot moves the oral surgery instrument to the target installation position for fixed installation according to the real-time adjusted path planning.

[0008] In some embodiments of the first aspect of the present application, the space registration unit comprises: a coordinate system construction subunit, which is used to establish a preoperative image coordinate system based on the preoperative oral scan image data of the patient; acquire multi-source sensor data of the cooperative robot and the oral customized bracket, and construct an intraoperative sensor coordinate system based on the multi-source sensor data; acquire actual anatomical structure data of the patient, and generate a real-time anatomical three-dimensional coordinate system based on the actual anatomical structure data; and a coordinate system registration subunit, which is used to perform space registration on the preoperative image coordinate system, the intraoperative sensor coordinate system and the real-time anatomical three-dimensional coordinate system, to obtain a space registration result.

[0009] In some embodiments of the first aspect of the present application, the process of real-time detection and positioning of the collaborative robot during installation by the optical positioning device comprises: setting a first positioning mark on the image mechanical arm and a second positioning mark on the instrument mechanical arm; the first positioning mark and the second positioning mark each comprise a plurality of mark points; detecting the three-dimensional coordinates of each mark point on the first positioning mark and the three-dimensional coordinates of each mark point on the second positioning mark by the optical positioning device; calculating the pose conversion matrix between the image mechanical arm and the instrument mechanical arm according to the three-dimensional coordinates of each mark point on the first positioning mark and the three-dimensional coordinates of each mark point on the second positioning mark; and calculating the relative pose relationship between the image mechanical arm and the instrument mechanical arm in real time according to the pose conversion matrix.

[0010] In some embodiments of the first aspect of the present application, the navigation installation unit further comprises: if the force feedback data collected by the force feedback device in real time is greater than a preset safety threshold, controlling the collaborative robot to slow down or pause.

[0011] In some embodiments of the first aspect of the present application, the system further comprises a human-computer interaction interface module; the human-computer interaction interface module comprises a display operation interface and a voice control unit; the display operation interface is used to display a three-dimensional model of a patient's oral cavity, multi-source sensor data, actual anatomical structure data of a patient, path planning, and force feedback data; and the voice control unit is used to receive voice control instructions to control the system.

[0012] In some embodiments of the first aspect of the present application, the control module further comprises a surgical instrument loading unit; the surgical instrument loading unit is used to fix a transoral surgical instrument on the front end of the instrument mechanical arm according to a loading instruction.

[0013] To achieve the above object and other related objects, the second aspect of the present application provides an automatic installation method of a transoral surgical instrument, applied to the automatic installation system of the transoral surgical instrument, and the method comprises the following steps: obtaining preoperative oral scanning image data of a patient based on a scanning device, and constructing a three-dimensional oral model of the patient according to the preoperative oral scanning image data; manufacturing a customized oral support of the patient based on the three-dimensional oral model, and setting a reference mark and a target installation position of the transoral surgical instrument on the customized oral support; performing spatial registration based on the preoperative oral scanning image data of the patient, multi-source sensor data and actual anatomical structure data of the patient to obtain a spatial registration result; performing path planning based on the spatial registration result in combination with the target installation position of the transoral surgical instrument, and performing real-time detection and positioning on a collaborative robot in an installation process through an optical positioning device, and collecting force feedback data in real time according to a force feedback device, and performing analysis based on the real-time detection and positioning result and the force feedback data to perform real-time adjustment on the path planning, so that the collaborative robot moves the transoral surgical instrument to the target installation position for fixed installation according to the real-time adjusted path planning.

[0014] To achieve the above object and other related objects, the third aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the automatic installation method of the transoral surgical instrument.

[0015] To achieve the above object and other related objects, the fourth aspect of the present application provides a computer program product, which comprises computer program codes, and when the computer program codes run on a computer, the computer program codes make the computer implement the automatic installation method of the transoral surgical instrument.

[0016] To achieve the above object and other related objects, the fifth aspect of the present application provides an electronic terminal, which comprises a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the automatic installation method of the transoral surgical instrument.

[0017] As described above, the automatic installation system, method, medium, program product and terminal of the transoral surgical instrument provided by the present application have the following beneficial effects: the present application can realize full-automatic, high-precision, safe and reliable positioning, installation and fixation of the transoral surgical instrument (such as an endoscope, a special instrument channel tube, a retractor, etc.) required for a transoral surgery (such as a transoral thyroid surgery, a transoral throat surgery, etc.) in the oral cavity of a patient. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 Fig. 1 shows a structure schematic diagram of an automatic installation system of a transoral surgical instrument according to an embodiment of the present application.

[0019] Figure 2 The diagram shown is a structural schematic of a spatial registration unit according to an embodiment of this application.

[0020] Figure 3 The diagram shown is another structural schematic of an automatic installation system for transoral surgical instruments according to one embodiment of this application.

[0021] Figure 4 The diagram shown is a flowchart illustrating an automatic installation method for transoral surgical instruments according to an embodiment of this application.

[0022] Figure 5 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0023] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0024] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1 This illustration shows a structural schematic of an automated transoral surgical instrument installation system according to an embodiment of the present invention. The system in this embodiment includes: a collaborative robot 100 and a control module 200; the collaborative robot 100 is connected to the control module 200; the collaborative robot 100 includes an imaging robotic arm and an instrument robotic arm; the front end of the instrument robotic arm is used to clamp and fix the transoral surgical instrument 300;

[0025] The control module 200 comprises: a model construction unit 210, configured to acquire preoperative oral scanning image data of a patient based on a scanning device, and construct a three-dimensional oral model of the patient according to the preoperative oral scanning image data; a bracket customization unit 220, configured to manufacture an oral customized bracket 400 of the patient based on the three-dimensional oral model, and set a reference mark and a target installation position of a transoral surgical instrument 300 on the oral customized bracket 400; a space registration unit 230, configured to perform space registration based on the preoperative oral scanning image data of the patient, multi-source sensor data and actual anatomical structure data of the patient, to obtain a space registration result; and a navigation installation unit 240, configured to perform path planning based on the space registration result in combination with the target installation position of the transoral surgical instrument 300, perform real-time detection and positioning on a collaborative robot 100 in an installation process through an optical positioning device, collect force feedback data in real time according to a force feedback device, analyze the real-time detection and positioning result and the force feedback data to perform real-time adjustment on the path planning, and move the transoral surgical instrument 300 to the target installation position for fixed installation according to the real-time adjusted path planning.

[0026] In this embodiment, the control module 200 can adopt one or more of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a neural network processor (NPU), a microcontroller unit (MCU), and an application-specific integrated circuit (ASIC).

[0027] In this embodiment, the scanning device comprises a digital oral scanner, a CBCT (cone beam computed tomography) device, a CT (computed tomography) scanner, an MRI (magnetic resonance imaging) device, etc., and is not limited in this embodiment.

[0028] In this embodiment, the preoperative oral scanning image data comprises, but is not limited to, scanning image data, CBCT scanning data, CT images, magnetic resonance images, etc. For example, the patient's oral cavity is segmented and scanned by a digital oral scanner to obtain multi-segment scanning image data in the form of point cloud data; the structure image data (CBCT scanning data) of the user's oral cavity can be obtained by a CBCT device, etc., and is not limited in this embodiment.

[0029] In this embodiment, a patient's oral cavity is customized based on a three-dimensional model of the oral cavity. The customized oral cavity is customized using a 3D printing or rapid prototyping technology, which is used to support the patient's oral cavity, ensuring that the patient's oral cavity remains stable, open and relatively fixed during surgery. A reference mark is provided on the oral cavity, which can be used for subsequent spatial registration operations. The reference mark includes a plurality of reference points, which have high contrast and no artifacts.

[0030] A position sensor can also be provided on the oral cavity, which is used to monitor the small displacement of the oral cavity relative to the patient's head in real time, for example, the sub-millimeter displacement of the oral cavity relative to the patient's head (such as 0.2-0.5mm offset caused by respiratory movement). The position sensor is a non-contact position sensor. The non-contact position sensor includes a Hall sensor and a non-contact inductive sensor. Non-contact sensors have a longer service life and higher stability. Therefore, the application can achieve sub-millimeter positioning accuracy and adapt to individual anatomical differences.

[0031] In this embodiment, the target installation position of the transoral surgical instrument can be provided on the oral cavity or near the tissue in the oral cavity, which is not limited in this embodiment according to the actual situation.

[0032] In this embodiment, the transoral surgical instrument is installed and fixed on the target installation position by the coordinated robot clamping the transoral surgical instrument into the patient's oral cavity during the automatic installation of the transoral surgical instrument. The transoral surgical instrument includes but is not limited to an endoscope, an instrument channel tube, a retractor, etc. The coordinated robot includes an image mechanical arm and an instrument mechanical arm. The front end of the instrument mechanical arm in the coordinated robot is provided with an instrument interface, which is suitable for automatic grabbing, releasing and locking of different types of transoral surgical instruments, so as to realize quick installation and disengagement of the transoral surgical instrument and the instrument mechanical arm, i.e. the instrument mechanical arm in the coordinated robot is used to clamp and fix the transoral surgical instrument. The instrument mechanical arm moves the transoral surgical instrument at its front end to the target installation position and installs and fixes it.

[0033] The coordinated robot also includes an image mechanical arm, which is used for real-time scanning and imaging during the movement of the instrument mechanical arm with the transoral surgical instrument. For example, a vision sensor or a miniature intracavity ultrasound probe can be provided at the front end of the image mechanical arm. The vision sensor can obtain local visual field image information during the installation process of the instrument mechanical arm, and the miniature intracavity ultrasound probe can provide real-time imaging of soft tissue.

[0034] In the embodiment, the spatial registration is one of the core technologies of the multi-modal imaging and navigation system, which provides accurate spatial positioning basis for surgical navigation (i.e. path planning) and other functions by high-precision alignment of preoperative oral scan image data, multi-source sensor data and actual anatomic structure data of the patient. The purpose of the spatial registration in the embodiment is to high-precision align three different coordinate systems.

[0035] In the embodiment, after constructing the three-dimensional model of the patient's oral cavity, the target installation position of the transoral surgical instrument is specified in the three-dimensional model of the oral cavity, and the corresponding preoperative ideal path is generated, at which time the preoperative ideal path is mapped in the virtual three-dimensional model of the oral cavity. After completing the above spatial registration operation, the path planning is performed based on the target installation position of the transoral surgical instrument provided on the oral customized support, that is, the preoperative ideal path is mapped to the actual coordinate system of the patient through spatial registration to obtain the actual moving path in the current oral cavity of the patient, and the transoral surgical instrument is moved and installed by the collaborative robot according to the actual moving path.

[0036] Further, in the process of installing the transoral surgical instrument by the collaborative robot based on the actual moving path, the collaborative robot during the installation process is detected and positioned in real time by the optical positioning device, and the force feedback data is collected in real time according to the force feedback device, and then the actual moving path is adjusted in real time based on the real-time detection and positioning results and the force feedback data, and the transoral surgical instrument is moved to the target installation position and fixedly installed by the collaborative robot according to the real-time adjusted path planning. According to the real-time detection and positioning of the collaborative robot and the force feedback, the actual moving path can be adjusted in real time, which can avoid the operation safety problem caused by abnormal situation. For example, the operation safety problem of damaging sensitive tissues such as oral cavity, teeth, throat and trachea can be avoided.

[0037] In the embodiment, the transoral surgical instrument is moved to the target installation position and fixedly installed by the collaborative robot according to the real-time adjusted path planning. The automatic alignment mechanism and the locking and fixing mechanism are also provided on the collaborative robot, and the automatic alignment mechanism and the locking and fixing mechanism can be provided on other mechanical arms of the collaborative robot. For example, the collaborative robot further includes an alignment mechanical arm and a fixing mechanical arm, and the automatic alignment mechanism is provided at the front end of the alignment mechanical arm, and the locking and fixing mechanism is provided at the front end of the fixing mechanical arm.

[0038] For example, in this embodiment, the target mounting position of the transoral surgical instrument is arranged on the oral customized support, and the automatic alignment mechanism is used to achieve the preliminary alignment of the transoral surgical instrument and the target mounting position through the conical surface, magnetic attraction or mechanical guide, and then the driving and locking fixing mechanism is used to achieve the final fixation of the transoral surgical instrument and the target mounting position through the mechanical lock (such as a threaded screw type mechanical lock, a buckle locking type mechanical lock, a shape memory alloy driven type mechanical lock) or a biological adhesive (such as a light-cured hydrogel).

[0039] In an embodiment, as shown in FIG. 1, the spatial registration unit 230 includes a coordinate system construction subunit 231 for establishing a preoperative image coordinate system based on preoperative oral scan image data of a patient, acquiring multi-source sensor data of the collaborative robot 100 and the oral customized support 400, and constructing an intraoperative sensor coordinate system based on the multi-source sensor data; and acquiring actual anatomic structure data of the patient and generating a real-time anatomic three-dimensional coordinate system based on the actual anatomic structure data; and a coordinate system registration subunit 232 for spatially registering the preoperative image coordinate system, the intraoperative sensor coordinate system, and the real-time anatomic three-dimensional coordinate system to obtain a spatial registration result. Figure 2 Specifically, the processing process in the coordinate system construction subunit 231 is as follows: after constructing a three-dimensional model of the patient's oral cavity according to the preoperative oral scan image data of the patient, the coordinate system of the three-dimensional model of the oral cavity is taken as the preoperative image coordinate system, which is a virtual image coordinate system. Then, the multi-source sensor data of the collaborative robot 100 and the oral customized support 400 are acquired. It should be noted that a plurality of sensors are arranged on the collaborative robot 100, and a plurality of sensors are also arranged on the oral customized support, and the plurality of sensors are position sensors. The multi-source sensor data is acquired according to the position sensors, and the positions of the robot and the oral customized support are acquired based on the multi-source sensor data, and then the intraoperative sensor coordinate system is constructed. The actual anatomic structure data of the patient is acquired in real time by a laser scanner or an optical coherence tomography device, and a real-time anatomic three-dimensional coordinate system, i.e., an actual coordinate system of the patient, is constructed.

[0040] After determining the three coordinate systems, i.e., the preoperative image coordinate system, the intraoperative sensor coordinate system, and the real-time anatomic three-dimensional coordinate system, spatial registration is performed. The essence of spatial registration is to establish a mapping relationship between the three coordinate systems through mathematical transformation, so as to unify the three coordinate systems, i.e., the preoperative image coordinate system, the intraoperative sensor coordinate system, and the real-time anatomic three-dimensional coordinate system, and obtain the final spatial registration result.

[0041] In an embodiment, the process of detecting and positioning the collaborative robot in the mounting process in real time by the optical positioning device includes:

[0042]

[0043] ​A first positioning marker is arranged on the image mechanical arm, and a second positioning marker is arranged on the instrument mechanical arm; the first positioning marker and the second positioning marker each include a plurality of marker points;

[0044] The three-dimensional coordinates of each marker point on the first positioning marker and the three-dimensional coordinates of each marker point on the second positioning marker are detected by an optical positioning device;

[0045] The three-dimensional coordinates of each marker point on the first positioning marker and the three-dimensional coordinates of each marker point on the second positioning marker are used for calculation to obtain a pose conversion matrix between the image mechanical arm and the instrument mechanical arm;

[0046] The pose conversion matrix is used for real-time calculation to obtain a relative pose relationship between the image mechanical arm and the instrument mechanical arm. It should be noted that the optical positioning device uses a planar array coded structured light technology to achieve high-precision three-dimensional positioning, and the optical positioning device is composed of a projector and a camera. The positioning principle of the optical positioning device is that a specific coded pattern (such as a Gray code + four-step phase shift combined coding) is projected onto the surface of the collaborative robot, and then the three-dimensional coordinates of the marker points on the collaborative robot are calculated and obtained by using the triangulation principle.

[0047] Specifically, the collaborative robot includes an image mechanical arm and an instrument mechanical arm, a first positioning marker is arranged on the image mechanical arm, the first positioning marker includes a plurality of marker points, and a second positioning marker is arranged on the instrument mechanical arm, the second positioning marker includes a plurality of marker points. An optical positioning device is used to measure the image mechanical arm and the instrument mechanical arm, to obtain the three-dimensional coordinates of all the marker points on the first positioning marker in a preset coordinate system of the optical positioning device, and the three-dimensional coordinates of all the marker points on the second positioning marker in the preset coordinate system of the optical positioning device. Calculation based on the three-dimensional coordinates of all the marker points on the first positioning marker in the preset coordinate system of the optical positioning device and the three-dimensional coordinates of all the marker points on the second positioning marker in the preset coordinate system of the optical positioning device can obtain the relative pose relationships between the optical positioning device and the image mechanical arm and the instrument mechanical arm respectively. Based on the relative pose relationships between the optical positioning device and the image mechanical arm and the instrument mechanical arm respectively, a pose conversion matrix between the image mechanical arm and the instrument mechanical arm is calculated, and then the relative pose relationship between the image mechanical arm and the instrument mechanical arm is obtained by real-time calculation, and the path planning is adjusted in real time according to the relative pose relationship between the image mechanical arm and the instrument mechanical arm.

[0048] In an embodiment, the navigation installation unit further includes: after real-time acquisition of force feedback data by the force feedback device, if the force feedback data is greater than a preset safety threshold, the collaborative robot is controlled to slow down or pause.

[0049] It should be noted that the force feedback devices are arranged on the image mechanical arm and the instrument mechanical arm respectively. During movement, if it is detected that the force feedback data is greater than a preset safety threshold, it indicates that sensitive tissues such as teeth and uvula are touched, at which time the collaborative robot needs to slow down or pause, so as to avoid damage to sensitive tissues such as the oral cavity, teeth, throat and trachea, and to re-adjust the path planning. Therefore, the system has perfect error detection and emergency handling mechanism.

[0050] In an embodiment, the system further comprises a human-computer interaction interface module; the human-computer interaction interface module comprises a display operation interface and a voice control unit; the display operation interface is used for displaying a three-dimensional model of the oral cavity of the patient, multi-source sensor data, actual anatomical structure data of the patient, path planning and force feedback data; the voice control unit is used for receiving voice control instructions to control the system. The human-computer interaction interface module further comprises a foot switch unit.

[0051] It should be noted that the display operation interface is a display screen, on which data and states during the entire installation process are displayed, and selection or switching operations can also be performed on the display screen, so as to realize human-computer interaction. During the operation, the voice control unit receives voice control instructions of the operator, and controls the start, pause and termination of the automatic installation process outside the sterile area of the operation according to the voice control instructions, or the foot switch unit controls the start, pause and termination of the automatic installation process outside the sterile area of the operation. Through the integration of the human-computer interaction interface module and the existing installation process of surgical instruments, the interface is intuitive and easy to operate, and conforms to the sterile operation specification of the operating room.

[0052] In an embodiment, as shown in Figure 3 The control module 200 further comprises a surgical instrument loading unit 250; the surgical instrument loading unit 250 is used for fixing the transoral surgical instrument on the front end of the instrument mechanical arm according to a loading instruction.

[0053] It should be noted that the surgical instrument loading unit 250 sends a loading instruction to the collaborative robot, and the collaborative robot drives the instrument mechanical arm according to the loading instruction, so that the instrument interface of the instrument mechanical arm is loaded into the corresponding transoral surgical instrument. When the transoral surgical instrument is fixedly installed on the target installation position, the surgical instrument loading unit 250 can also control the instrument mechanical arm to release the transoral surgical instrument.

[0054] In an embodiment, as shown in Figure 3 The control module 200 further comprises a state feedback unit 260, which is used for acquiring installation state information in real time, and judging whether the transoral surgical instrument is successfully installed according to the installation state information.

[0055] Integrating a state feedback sensor around the target installation position, the state feedback sensor including a micro switch, a pressure sensor, an optical encoder, etc., installation state information can be obtained through the state feedback sensor, and whether the transoral surgical instrument is successfully installed is determined according to the installation state information (such as confirming whether the instrument is installed in place, locked reliably, etc.). Meanwhile, the installation state information can also be displayed on the display operation interface.

[0056] In order to facilitate the understanding of the automatic installation system of the transoral surgical instrument of the present application, the following specific embodiments are provided for illustration.

[0057] Embodiment 1: Implementation process of an automatic installation system of a transoral surgical instrument.

[0058] Printing a patient's oral customized support, and the patient wearing the oral customized support;

[0059] The operator starts the automatic installation process through voice control, footswitch or display screen touch on the human-computer interaction interface module;

[0060] The collaborative robot starts working, and drives the instrument mechanical arm of the collaborative robot to load the transoral surgical instrument;

[0061] The collaborative robot moves the transoral surgical instrument according to the preoperative ideal path, and adjusts the path planning in real time according to the collaborative robot positioning and force feedback data during the movement;

[0062] When the transoral surgical instrument moves to the target installation position, the instrument and the target installation position are preliminarily coupled through the automatic alignment mechanism, and then the mechanical lock or biological adhesive is activated through the locking and fixing mechanism to realize the final fixation of the transoral surgical instrument and the target installation position;

[0063] The state feedback sensor confirms that the transoral surgical instrument is installed in place and fixed reliably, and displays the installation state on the display screen;

[0064] The collaborative robot releases the transoral surgical instrument and withdraws from the patient's oral cavity;

[0065] The operator performs transoral surgical operation and enters the operation stage.

[0066] It should be emphasized that the beneficial effects of the automatic installation system of the transoral surgical instrument of the present application are as follows.

[0067] (1) High-precision positioning: sub-millimeter positioning accuracy can be achieved, and individual anatomical differences can be adapted.

[0068] (2) Fast and efficient: through the automatic process, the operation preparation time can be significantly shortened.

[0069] (3) Operation safety: able to avoid damage to sensitive tissues such as the mouth, teeth, throat, trachea, etc.

[0070] (4) Sterile protection: in line with the sterile operation specifications of the operating room.

[0071] (5) User-friendly: integrated with existing surgical procedures, intuitive interface and easy to operate.

[0072] (6) Reliability: the system is stable and has a perfect error detection and emergency handling mechanism.

[0073] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish the same or similar items with basically the same function and effect, and do not limit the sequence. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the number and execution sequence, and the terms "first", "second", etc. also do not necessarily mean different.

[0074] It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" indicate an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of the words "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0075] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. The association relationship between the associated objects is described, which means that there can be three kinds of relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including single item or any combination of multiple items. For example, at least one of a, b or c, which can represent: a, b, c, a-b, a-c, b-c or a-b-c, where a, b and c can be single or multiple.

[0076] Figure 4 is a flowchart of an automatic installation method of a transoral surgical instrument provided by the embodiments of the present application. As shown in Figure 4 , the method comprises:

[0077] Step S41: obtaining preoperative oral scanning image data of a patient based on a scanning device, and constructing a three-dimensional model of the patient's oral cavity according to the preoperative oral scanning image data;

[0078] Step S42: manufacturing the oral cavity customized support of the patient based on the three-dimensional model of the oral cavity, and setting the reference mark and the target installation position of the transoral surgical instrument on the oral cavity customized support;

[0079] Step S43: performing spatial registration based on the preoperative oral cavity scanning image data of the patient, the multi-source sensor data and the actual anatomical structure data of the patient to obtain a spatial registration result;

[0080] Step S44: performing path planning based on the spatial registration result in combination with the target installation position of the transoral surgical instrument, performing real-time detection and positioning on the collaborative robot in the installation process through the optical positioning device, and collecting force feedback data in real time according to the force feedback device, performing analysis based on the real-time detection and positioning result and the force feedback data to perform real-time adjustment on the path planning, so that the collaborative robot moves the transoral surgical instrument to the target installation position for fixed installation according to the real-time adjusted path planning.

[0081] It should be understood that the specific process of the above-mentioned corresponding steps has been described in detail in the above-mentioned system embodiment, and for the sake of brevity, it will not be repeated here.

[0082] It should also be understood that the division of modules in the embodiments of the present application is illustrative, and is only a logical functional division. Actual implementation can have another division mode. In addition, each functional module in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of software functional module.

[0083] Figure 5 is a schematic block diagram of an electronic terminal provided by the embodiments of the present application. As shown in Figure 5 , the electronic terminal includes at least one processor 501, a memory 502, at least one network interface 503 and a user interface 505. Each component in the device is coupled together through a bus system 504. It can be understood that the bus system 504 is used to realize the connection and communication between these components. In addition to including a data bus, the bus system 504 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, all kinds of buses are marked as a bus system in Figure 5 .

[0084] Among them, the user interface 505 can include a display, a keyboard, a mouse, a trackball, a click gun, a key, a button, a touchpad or a touch screen, etc.

[0085] It is to be understood that the memory 502 can be volatile or nonvolatile memory, or both. The nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), which is used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present application is intended to include, but not be limited to, these and any other suitable type of memory.

[0086] The memory 502 in the embodiments of the present application is configured to store various types of data to support the operation of the electronic terminal 500. Examples of the data include any executable programs for operating on the electronic terminal 500, such as an operating system 5021 and an application program 5022. The operating system 5021 contains various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application program 5022 can contain various application programs, such as a media player, a browser, and the like, for implementing various application services. The method for automatically installing a transoral surgical instrument provided by the embodiments of the present application can be included in the application program 5022.

[0087] The method disclosed in the embodiments of the present application can be applied in the processor 501 or implemented by the processor 501. The processor 501 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 501. The processor 501 described above can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The processor 501 can implement or execute the disclosed methods, steps, and logic block diagrams in the embodiments of the present application. The general-purpose processor 501 can be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in conjunction with the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium, which is located in the memory. The processor reads the information in the memory and combines the hardware to complete the steps of the above method.

[0088] In an exemplary embodiment, the electronic terminal 500 can be implemented by one or more of an application specific integrated circuit (ASIC), a DSP, a programmable logic device (PLD), a complex programmable logic device (CPLD), and a complex programmable logic device (CPLD) for executing the aforementioned method.

[0089] According to the method provided by the embodiment of the present application, the present application further provides a computer program product, which comprises computer program code, and when the computer program code is run on a computer, the computer is caused to execute the automatic installation method of the transoral surgical instrument in any of the embodiments.

[0090] According to the method provided by the embodiment of the present application, the present application further provides a computer readable storage medium, which stores program code, and when the program code is run on a computer, the computer is caused to execute the automatic installation method of the transoral surgical instrument in any of the embodiments.

[0091] The terms "component," "module," "system," and the like used in the present specification are used to represent a computer-related entity, hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. By way of illustration, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and / or thread of execution, and a component can be localized on one computer and / or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components can communicate by way of local and / or remote processes such as in accordance with a signal having one or more data packets (e.g., data from programs, data included in a management information base, etc.), such as data in a signal provided through a network infrastructure, such as the Internet, a local area network, a wide area network, a wired network, a wireless network, etc. to another system.

[0092] Those of skill in the art would understand that the various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. The choice of hardware or software, or combinations of both, would be dependent on the specific application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0093] Those of skill in the art would understand that, for the purposes of description and enabling the claimed application, the specific process described above for the system, apparatus and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0094] In several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, for example, the division of units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0095] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0096] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present alone, or two or more units can be integrated into one unit.

[0097] In the above embodiments, the functions of the various functional units can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the functions can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, the whole or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a high-density digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD), etc.

[0098] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0099] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0100] In summary, the present application provides an oral surgery instrument automatic installation system, method, medium, program product and terminal, comprising: a collaborative robot and a control module; the collaborative robot is connected with the control module; the collaborative robot comprises an image mechanical arm and an instrument mechanical arm; the front end of the instrument mechanical arm is used for clamping and fixing an oral surgery instrument; wherein the control module comprises: a model construction unit, configured to obtain preoperative oral scan image data of a patient based on a scanning device, and construct a three-dimensional oral model of the patient according to the preoperative oral scan image data; a support customization unit, configured to manufacture an oral customized support of the patient based on the three-dimensional oral model, and set a reference mark and a target installation position of the oral surgery instrument on the oral customized support; a space registration unit, configured to perform space registration based on the preoperative oral scan image data of the patient, multi-source sensor data and actual anatomical structure data of the patient, to obtain a space registration result; a navigation installation unit, configured to perform path planning based on the space registration result combined with the target installation position of the oral surgery instrument, and perform real-time detection and positioning on the collaborative robot during installation through an optical positioning device, and collect force feedback data in real time according to a force feedback device, and perform analysis based on the real-time detection and positioning result and the force feedback data to perform real-time adjustment on the path planning, so that the collaborative robot moves the oral surgery instrument to the target installation position for fixed installation according to the real-time adjusted path planning. The present application can realize full-automatic, high-precision, safe and reliable positioning, installation and fixation of the oral surgery instrument (such as an endoscope, a special instrument channel pipe, a retractor, etc.) required for oral surgery (such as oral thyroid surgery, oral throat surgery, etc.) in the oral cavity of the patient. Therefore, the present application effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0101] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical idea of the present application should be covered by the claims of the present application.

Claims

1. An automatic instrument loading system for transoral surgical procedures, characterized in that, include: A collaborative robot and a control module; the collaborative robot is connected to the control module; the collaborative robot includes an imaging robotic arm and a mechanical robotic arm. The front end of the robotic arm is used to clamp and fix the surgical instruments used in oral surgery. The control module includes: The model building unit is used to acquire the patient's preoperative oral scan image data based on the scanning device, and to build a three-dimensional model of the patient's oral cavity based on the preoperative oral scan image data. The stent customization unit is used to create a custom oral stent for the patient based on a three-dimensional oral model, and to set reference marks and target installation positions for transoral surgical instruments on the custom oral stent; The spatial registration unit is used to perform spatial registration based on the patient's preoperative oral scan image data, multi-source sensor data, and the patient's actual anatomical structure data to obtain spatial registration results. The navigation and installation unit is used to plan a path based on the spatial registration results and the target installation position of the transoral surgical instrument. It also uses optical positioning equipment to detect and locate the collaborative robot in real time during the installation process, and uses force feedback equipment to collect force feedback data in real time. Based on the real-time detection and positioning results and force feedback data, the unit analyzes the data to adjust the path planning in real time, so that the collaborative robot can move the transoral surgical instrument to the target installation position for fixed installation according to the real-time adjusted path planning.

2. The automatic instrument loading system for transoral surgical instruments according to claim 1, characterized in that, The spatial registration unit includes: The coordinate system construction subunit is used to establish a preoperative image coordinate system based on the patient's preoperative oral scan image data; acquire multi-source sensor data from the collaborative robot and the customized oral support, and construct an intraoperative sensor coordinate system based on the multi-source sensor data; acquire the patient's actual anatomical structure data, and generate a real-time anatomical three-dimensional coordinate system based on the actual anatomical structure data. The coordinate system registration subunit is used to spatially register the preoperative image coordinate system, the intraoperative sensor coordinate system, and the real-time anatomical 3D coordinate system to obtain spatial registration results.

3. The automatic instrument loading system for transoral surgical instruments according to claim 1, characterized in that, The process of real-time detection and positioning of the collaborative robot during installation using optical positioning equipment includes: A first positioning mark is set on the imaging robotic arm, and a second positioning mark is set on the instrument robotic arm; the first positioning mark and the second positioning mark each include multiple marking points; The three-dimensional coordinates of each marker point on the first positioning mark and the three-dimensional coordinates of each marker point on the second positioning mark are obtained by detecting with optical positioning equipment. The pose transformation matrix between the imaging robotic arm and the instrument robotic arm is obtained by calculating the three-dimensional coordinates of each marker point on the first positioning mark and the three-dimensional coordinates of each marker point on the second positioning mark. The relative pose relationship between the imaging robotic arm and the instrument robotic arm is obtained by real-time calculation based on the pose transformation matrix.

4. The automatic instrument loading system for transoral surgical instruments according to claim 1, characterized in that, The navigation installation unit further includes: after collecting force feedback data in real time from the force feedback device, if the force feedback data is greater than a preset safety threshold, controlling the collaborative robot to decelerate or pause.

5. The automatic instrument loading system for transoral surgical instruments according to claim 1, characterized in that, The system also includes a human-computer interaction interface module; the human-computer interaction interface module includes a display operation interface and a voice control unit; the display operation interface is used to display the patient's oral cavity three-dimensional model, multi-source sensor data, the patient's actual anatomical structure data, path planning, and force feedback data; The voice control unit is used to receive voice control commands to control the system.

6. The automatic installation system for transoral surgical instruments according to claim 1, characterized in that, The control module also includes a surgical instrument loading unit; the surgical instrument loading unit is used to fix the oral surgical instruments to the front end of the surgical instrument arm according to the loading command.

7. An automatic installation method for transoral surgical instruments, characterized in that, The method, applied to the automatic instrument loading system for transoral surgical instruments as described in any one of claims 1 to 6, comprises: Preoperative oral scan images of patients are acquired using scanning equipment, and a three-dimensional model of the patient's oral cavity is constructed based on the preoperative oral scan images. A custom-made oral support for the patient is created based on a three-dimensional oral model, and reference marks and target installation positions for transoral surgical instruments are set on the custom-made oral support. Spatial registration is performed based on the patient's preoperative oral scan image data, multi-source sensor data, and the patient's actual anatomical structure data to obtain spatial registration results. Path planning is performed based on spatial registration results and the target installation position of the transoral surgical instruments. The collaborative robot is monitored and positioned in real time during the installation process using optical positioning equipment. Force feedback data is collected in real time using force feedback equipment. The path planning is adjusted in real time based on the real-time detection and positioning results and force feedback data. The collaborative robot then moves the transoral surgical instruments to the target installation position for fixed installation according to the adjusted path planning.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the automatic installation method for oral surgical instruments as described in claim 7.

9. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, enables the computer to implement the automatic installation method for transoral surgical instruments as described in claim 7.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the automatic installation method for transoral surgical instruments as described in claim 7.

Images