Surgical system and method for operating the same

Abstract

Embodiments herein provide method for operating a surgical system (100). The method includes choosing to execute a segmentation technique using a toggle if a surgeon is satisfied that a lesion is suspicious and needs to undergo a therapeutic procedure for resection, or removal of the lesion. In the event of underlying blood vessels, or deep tissue being detected by computer vision techniques, force feedback mechanism gets triggered, so as to avoid the damage to underlying structures and deep tissues. The method uses a damping technique to prevent excessive force application during the surgical procedure, while maintaining a steady and controlled operation throughout procedure. During the procedure, a force sensor (116) is capable of sensing the force applied to the tissue, and a position sensor (118) provides a spatiotemporal trajectory, and the master computational and control unit (160) executes commands to ensures surgical accuracy.

Description

SURGICAL SYSTEM AND METHOD FOR OPERATING THE SAMETECHNICAL FIELD

[0001] Embodiments disclosed herein relate to a surgical system, and more particularly to a system and a method for handling a function (or operation) of the surgical system.BACKGROUND

[0002] Gastrointestinal endoscopic procedures are the gold standard method for the detection and diagnosis of almost all gastrointestinal pathologies. These gastrointestinal endoscopic procedures are also used to perform certain minimally invasive therapeutic procedures, such as endoscopic mucosal resection and endoscopic submucosal dissection. These are surgical procedures conducted to remove precancerous, cancerous tissue from a gastrointestinal tract. Detection of gastrointestinal diseases via this modality of diagnosis is a notoriously difficult process, and statistics reveal that 30% (for example) of all lesions are missed during these gastrointestinal endoscopic procedures. The miss rate for gastric cancer is even more startling, with published statistics revealing a current miss rate of upto 25% (for example) for the debilitating disease.

[0003] Moreover, therapeutic procedures conducted via an endoscopic mode are extremely difficult to perform. There is only one biopsy port (i.e., instrument channel), through which a single instrument can be passed to perform these therapeutic procedures. As only a single instrument can be used at a time, a physician cannot retract the tissue, which is an essential principle of performing a surgical excision. The instrument cannot be manipulated freely, and has very limited spatial access to the tissue, measured in terms of its degrees of freedom. Triangulation, which is a prerequisite to the accuracy and success of any operation, cannot be achieved using a single instrument channel. Moreover, a very limited space is accessible to the physician. All of these constraints result in a very high learning curve, increased risk of bleeding and perforation, increased risk of retained tumor margins, and any complication (such as perforation, that requires suturing) has to be managed by immediately converting the endoscopic procedure into an open surgery. These procedures are complex to perform, and are prone to the constant sub -optimal over and under delivery of force, often leading to surgical excision of deep underlying tissue structures, rupture of underlying blood vessels, retention of tumour margins, and harmful and erratic patterns of delivery of force.

[0004] Hence, there is a need in the art for solutions which will overcome the above- mentioned drawback(s), among others.OBJECTS

[0005] The principal object of embodiments herein is to disclose a method for handling a function (or operation) of a surgical system.

[0006] Another object of embodiments herein is to switch ON (or triggering) a computer vision handling controller, so as to evaluate and analyze a constant video stream via a monitor.

[0007] Another object of embodiments herein is to detect a suspicious lesion by using the computer vision technique.

[0008] Another object of embodiments herein is to generate a bounding box around coordinates of the suspicious lesion.

[0009] Another object of embodiments herein is to display and highlight the lesion with a generated bounding box output in real time on the monitor.

[0010] Another object of embodiments herein is to choose to freeze a frame to view the lesion for a predefined time.

[0011] Another object of embodiments herein is to choose to execute a segmentation technique using at least one toggle if a surgeon is satisfied that the lesion is suspicious and needs to undergo a therapeutic procedure utilizing third space endoscopic principles or not. The therapeutic procedure can be, for example, but not limited to an Endoscopic Mucosal Resection (EMR), an Endoscopic Submucosal Dissection (ESD), a Polypectomy, an Endoscopic Full Thickness Resection (EFTR), a Hemostasis, Band Ligation, an Endoscopic Suturing, an Endoscopic Stenting, an Ablation Therapy (using at least one of: light, ultrasound, and radiofrequency), a Peroral Endoscopic Myotomy (POEM), an Endoscopic Retrograde Cholangiopancreatography (ERCP) with intervention, a Submucosal Tunneling Endoscopic Resection (STER), a Third Space Endoscopy or the like.

[0012] Another object of embodiments herein is to delineate the 2-dimensional coordinates of the lesion based on the computer vision technique and the segmentation technique.

[0013] Another object of embodiments herein is to initiate a depth estimation technique to generate 3 -dimensional coordinates of the lesion.

[0014] Another object of embodiments herein is to compute the output of the depth estimation technique and the segmentation technique to generate real time 3 -dimensional coordinates of the lesion in an optimized manner.

[0015] Another object of embodiments herein is to execute a collision detection technique to constantly monitor the 3D position of a surgical instrument along with the 3D position of the lesion.

[0016] Another object of embodiments herein is to generate an alert when the surgical instrument is outside the 3D position of the lesion.

[0017] Another object of embodiments herein is to trigger a force feedback mechanism, by issuing a stop command to an input current of an electrocautery instrument or by issuing a stop command to an end effector (in case a dual robotic arm is being used to perform a surgical procedure). In the event of underlying blood vessels, or deep tissue being detected by the computer vision techniques, the force feedback mechanism gets triggered, so as to avoid the damage to underlying structures and deep tissue if detected.

[0018] Another object of embodiments herein is to use a damping technique to prevent excessive force application during the surgical procedure, while maintaining a steady and controlled operation throughout the surgical procedure.

[0019] Another object of embodiments herein is to receive at least one continuous input to compute details regarding a current position of each joint segment and an arm, force being applied at various points of the arm as and an overall motion during a surgical procedure, wherein the at least one continuous input is obtained from at least one of a force sensor, a position sensor, and a motion sensor.

[0020] Another object of embodiments herein is to process and compute at least one continuous input to determine at least one operational parameter, wherein at least one operational parameter comprises at least one of: current position, desired position, current force, desired force, proximity to healthy tissue, proximity to threshold force of each individual segment as well as overall kinematics and dynamics of an arm.

[0021] Another object of embodiments herein is to determine whether the desired position is achieved using the current state, the exact motion to be undertaken to achieve desired state, as well as the current state of force, and a control mechanism to ensure steady output of an optimal force.

[0022] Another object of embodiments herein is to determine whether a desired position of a robotic arm can be achieved from the current state, spatial constraints and existing surgical environment.

[0023] Another object of embodiments herein is to determine an exact movement required to transition the robotic arm from the current state to a desired position based on the determination.

[0024] Another object of embodiments herein is to compute a necessary adjustment to align the current force with the desired levels by comparing a current force applied by a robotic arm with a desired force level.

[0025] Another object of embodiments herein is to provide a control mechanism that ensures a steady application of optimal force, so as to avoid sudden spikes that result in tissue damage, wherein the master computational and control unit is equipped with at least one damping mechanism designed to respond to sudden changes in the applied force during the surgical procedure.

[0026] Another object of embodiments herein is to provide a force sensor that is capable of sensing the force applied to the tissue, and is a part of a feedback loop to regulate and control the force to be applied to the tissue during the surgical procedure, wherein the force sensor continuously monitors the force applied to the tissues by end effectors, wherein the force sensor are included in a complex feedback loop, wherein the complex feedback loop comprises a visual and haptic feedback mechanism, wherein the complex feedback loop allows the surgical system to dynamically adjust the applied force, preventing tissue damage during the surgical procedure, wherein the visual and haptic feedback mechanism handles threshold force detection, force reduction and real time force adjustment to prevent tissue damage during the surgical procedure.

[0027] Another object of embodiments herein is to provide a control mechanism that ensures a steady application of optimal force, so as to avoid sudden spikes that result in tissue damage.

[0028] Another object of embodiments herein is to provide a force sensor that is capable of sensing the force applied to the tissue, and is a part of a feedback loop to regulate and control the force to be applied to the tissue during the surgical procedure, where the force sensor continuously monitors the force applied to the tissues by end effectors.

[0029] Another object of embodiments herein is to provide the force sensor that is included in a complex feedback loop, where the complex feedback loop includes a visual and haptic feedback mechanism, wherein the complex feedback loop allows the surgical system to dynamically adjust the applied force, preventing tissue damage during the surgical procedure, wherein the visual and haptic feedback mechanism handles threshold force detection, force reduction and real time force adjustment to prevent tissue damage during the surgical procedure.

[0030] Another object of embodiments herein is to provide a position sensor that provides a spatiotemporal trajectory to ensure surgical accuracy during the surgical procedure, where the position sensor is integrated with at least one visual technique, where the position sensor continually monitors angles, rotation, and displacement of each joint and segment of a robotic arm, and data associated with the angles, the rotation, and the displacement of each joint and segment of the robotic arm is transferred to the master computational and control unit.

[0031] Another object of embodiments herein is to use an adaptive force control mechanism for identifying a type of tissue in a surgical scene and a physical property of the tissue by using the computer vision technique.

[0032] Another object of embodiments herein is to embed the computer vision handling controller, the force sensor, the position sensor, and the motion sensor in dual robotic arms.

[0033] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating at least one embodiment and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the scope thereof, and the embodiments herein include all such modifications.BRIEF DESCRIPTION OF FIGURES

[0034] Embodiments herein are illustrated in the accompanying drawings, throughout which like reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood from the following description with reference to the following illustratory drawings. Embodiments herein are illustrated by way of examples in the accompanying drawings, and in which:

[0035] FIG. 1 shows various hardware components of a surgical system, according to embodiments as disclosed herein;

[0036] FIG. 2 A and FIG. 2B are flow charts illustrating a method for handling a function of the surgical system, according to embodiments as disclosed herein;

[0037] FIG. 3 is a flow chart illustrating a damping mechanism while handling the function of the surgical system, according to embodiments as disclosed herein;

[0038] FIG. 4 is an example environment in which an operation and a function of the surgical system is depicted, according to embodiments as disclosed herein; and

[0039] FIG. 5 is an example illustration depicting 3-dimensional coordinates of the lesion by delineating 2-dimensional coordinates of the lesion based on a computer vision technique and a segmentation technique.DETAILED DESCRIPTION

[0040] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

[0041] For the purposes of interpreting this specification, the definitions (as defined herein) will apply and whenever appropriate the terms used in singular will also include the plural and vice versa. It is to be understood that the terminology used herein is for the purposes of describing particular embodiments only and is not intended to be limiting. The terms “comprising”, “having” and “including” are to be construed as open-ended terms unless otherwise noted.

[0042] The words / phrases "exemplary", “example”, “illustration”, “in an instance”, “and the like”, “and so on”, “etc.”, “etcetera”, “e.g.,” , “i.e.,” are merely used herein to mean "serving as an example, instance, or illustration." Any embodiment or implementation of the present subject matter described herein using the words / phrases "exemplary", “example”,“illustration”, “in an instance”, “and the like”, “and so on”, “etc ”, “etcetera”, “e.g.,” , “i.e.,” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0043] Embodiments herein may be described and illustrated in terms of blocks which carry out a described function or functions. These blocks, which may be referred to herein as managers, units, modules, hardware components or the like, are physically implemented by analog and / or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits and the like, and may optionally be driven by a firmware. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like. The circuits constituting a block may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block. Each block of the embodiments may be physically separated into two or more interacting and discrete blocks without departing from the scope of the disclosure. Likewise, the blocks of the embodiments may be physically combined into more complex blocks without departing from the scope of the disclosure.

[0044] It should be noted that elements in the drawings are illustrated for the purposes of this description and ease of understanding and may not have necessarily been drawn to scale. For example, the flowcharts / sequence diagrams illustrate the method in terms of the steps required for the understanding of aspects of the embodiments as disclosed herein. Furthermore, in terms of the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding the present embodiments so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. Furthermore, in terms of the system, one or more components / modules which comprise the system may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding the present embodiments so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

[0045] The accompanying drawings are used to help easily understand various technical features and it should be understood that the embodiments presented herein are notlimited by the accompanying drawings. As such, the present disclosure should be construed to extend to any modifications, equivalents, and substitutes in addition to those which are particularly set out in the accompanying drawings and the corresponding description. Usage of words such as first, second, third etc., to describe components / elements / steps is for the purposes of this description and should not be construed as sequential ordering / placement / occurrence unless specified otherwise.

[0046] The embodiments herein achieve a surgical system including a modified surgical instrument, or a robotic arm with an end effector of a miniaturized surgical instrument, inserted through a GI orifice. The modified surgical instrument refers to any surgical instrument, which has been constructed to be equipped with sensors and communication mechanisms to receive and input signals or commands for control from the master computational and control unit, and has the ability to accept and execute these commands via output toggles responsible for mechanical movement (in case of motors) or stopping the inflow of current (in case of electrocautery or other electromechanical instruments)- and such modified surgical instruments are compatible and are an integral part of this surgical control workflow. The surgical instrument communicates with a monitor, and a computer vision handling controller can be switched on via the toggle (or any input unit), so as to analyze a constant video stream as viewed on the monitor. The toggle switches on a computer vision technique to perform processing and analysis of the real time endoscopic video feed to identify and detect suspicious lesions via specially trained neural networks, which form the fundamental base of the computer vision handling controller. The computer vision post-processing module generates a bounding box around coordinates of the suspicious lesion as detected by the neural network techniques. Hence, the computer vision handling controller displays and highlights the lesion with a generated bounding box output in real time on the monitor.

[0047] The proposed system pertains to the field of gastrointestinal endoscopy (e.g., flexible endoscopes and colonoscopes used to perform diagnosis of gastrointestinal diseases, and therapeutic procedures). Also, the proposed system pertains to the field of computer vision and force feedback loops during robotic endoscopic surgery for precise movement regulation during the surgical procedure, especially in events of sudden force transmission, or detection of force transmitted outside of the area of pathology (outside the mucosal / submucosal layers), or in the event of a blood vessel being detected in the layer of the submucosa.

[0048] The proposed system relates particularly to an endoscopic system designed for minimally invasive procedures. The system integrates advanced force, position, and visualfeedback loops to enhance surgical precision, safety, and overall outcomes. Traditional robotic systems lack integrated feedback mechanisms needed to dynamically adjust to real-time conditions, leading to potential risks such as excessive force application or inaccurate instrument positioning. Moreover, there is no way of regulating unsteady force, or immediately stopping operations in real-time to prevent damage to underlying blood vessels, deep tissue, and other vulnerable structures. The proposed system addresses these limitations by providing a sophisticated robotic or non-robotic instrumentation-based system that leverages real-time data from multiple sensors and advanced computational techniques to optimize surgical performance.

[0049] In the proposed endoscopic system, the computer vision techniques (act as a second scout, and automate identification and detection of lesions in real time), interact with a dual robotic arm with surgical instrument end effectors (which is used to support two instruments at a time, enables tissue retraction, triangulation, movement in multiple degrees of freedom, controlled and precise movements and force delivery, assistance in performing functions such as suturing), or a modified surgical instrument; as well as force feedback and position feedback systems, to improve the precision of surgical procedures including accurate delivery of force, precise manipulation of robotic arms interfacing with the surgeon's console, accurate interpretation of commands and ensuring that the robotic arms aid in the execution of precise and accurate surgical outcomes. The endoscopic system also assists in preventing excessive delivery of force, by damping force in case of an adverse event such as force manipulation beyond region of interest, presence of underlying blood vessels or deep structures / tissue as detected by the computer vision handling controller, and aiding in a constant delivery of force to prevent sudden injection of force transmission into the tissues.

[0050] The invention is a robotic endoscopic system comprising of dual robotic arms, or alternatively modified modular surgical instruments, each equipped with a network of sensors, including force, position, and motion sensors, which is integrated with a comprehensive integrated visual feedback system. A master computational and control unit serves as a central processing hub, receiving and interpreting data from these sensors and the surgeon's console. The surgeon’s console inputs commands relayed by the surgeon for the manipulation of the surgical instruments / robotic arm to perform the operation, which includes the force transmitted, position to be manipulated in, exact kinematics and dynamics in all translational and rotational degrees of freedom. These commands are relayed to the master computational and control unit for analysis, and corrective action (in case of unsteady orexcessive force transmission) and are relayed back to the end effectors of the modified surgical instrument / robotic arm.

[0051] The system continuously monitors the state of the robotic arms and the surgical environment, processing sensor data in real-time to compute essential parameters such as current and desired positions, applied and optimal forces, and proximity to healthy tissue. This data is used to dynamically adjust the robotic arms' movements and force application, ensuring precise execution of surgical tasks while preventing tissue damage.

[0052] Referring now to the drawings, and more particularly to FIGS. 1 through 5, where similar reference characters denote corresponding features consistently throughout the figures, there are shown embodiments.

[0053] FIG. 1 shows various hardware components a surgical system (100), according to embodiments as disclosed herein. The surgical system (100) includes a surgical instrument (110) (i.e., modified surgical instrument or enhanced surgical instrument), an endoscopy tube (140), an internal instrument port (140), an image processing system (150) (e.g., video processing system, or the like), a monitor (155) (e.g., video processing system, computer, server or the like), a master computational and control unit (160), a position sensing and control system (165), a camera (170) and a processor (175). The surgical instrument (110) can be, for example, but not limited to an endoscope, a colonoscope or the like. In an embodiment, a dual robotic arm (130) is attached to the endoscope or the colonoscope. In an embodiment, the surgical instrument (110) includes a mechanical arm unit (112), a motor (114), a force sensor (116), a position sensor (118), a toggle (120), a computer vision handling controller (122), a segmentation controller (124), a depth estimation controller (126), a force control unit (128) and a motion sensor (129). The motor (114) is responsible for the kinematic and dynamic controlled movement of joints of the robotic arm (130) / modified surgical instrument (110) and receives input from surgeon’s console, as well as master computational and control unit (160). In an embodiment, the endoscopy tube (140) includes a valve, a light guide connector, a light guide tube, a control body, and an insertion tube. In an implementation, the force sensor (116), the position sensor (118) and the motion sensor (129) are housed in the dual robotic arm (130). In another embodiment, the dual robotic arm (130) includes the force sensor (116), the position sensor (118), the toggle (120), the computer vision handling controller (122), the segmentation controller (124), the depth estimation controller (126), the force control unit (128) and the motion sensor (129). The valve and the insertion tube covers and maintains the integrity of the endoscope, as a sterile mechanical barrier to the gastrointestinal tissue. The light guide isresponsible for illuminating the gastrointestinal tract, to make tissues more prominent for capture on the real time video feed by the camera (170). The control body is the mechanical interface through which the gastroenterologist controls the motion of endoscope and enables advancing / retraction of endoscope into the gastrointestinal tract.

[0054] The surgical instrument (110) (i.e., enhanced surgical instrument) is inserted through a gastrointestinal (GI) orifice, where the surgical instrument (110) communicates with a monitor (155). The toggle (120) switches on a computer vision technique to start processing a real time endoscopic video feed to detect a suspicious lesion based on the computer vision technique by using a computer vision handling controller (122). The at least one toggle (120) can be an input unit or a control unit. The computer vision handling controller (122) generates a bounding box around coordinates of the suspicious lesion by using the computer vision technique. Further, the computer vision handling controller (122) displays and highlights the lesion with a generated bounding box output in real time on the monitor (155).

[0055] In an embodiment, the computer vision handling controller (122) chooses to freeze a frame to view the lesion for a predefined time. The predefined time is set by the surgeon (i.e., user of the surgical system (100)). In another embodiment, the computer vision handling controller (122) chooses to execute a segmentation technique using the toggle (120) if the surgeon is satisfied that the lesion is suspicious and needs to undergo a therapeutic procedure utilizing a third space endoscopic principle or not. The therapeutic procedure can be, for example, but not limited to an Endoscopic Mucosal Resection (EMR), an Endoscopic Submucosal Dissection (ESD), a Polypectomy, an Endoscopic Full Thickness Resection (EFTR), a Hemostasis, Band Ligation, an Endoscopic Suturing, an Endoscopic Stenting, an Ablation Therapy (using at least one of: light, ultrasound, and radiofrequency), a Peroral Endoscopic Myotomy (POEM), an Endoscopic Retrograde Cholangiopancreatography (ERCP) with intervention, a Submucosal Tunneling Endoscopic Resection (STER), a Third Space Endoscopy or the like.

[0056] Further, the segmentation controller (124) delineates 2-dimensional coordinates of the lesion based on the computer vision technique and the segmentation technique. Further, the depth estimation controller (126) initiates a depth estimation technique to generate 3- dimensional coordinates of the lesion (as shown in FIG. 5). A segmentation controller (124) and the depth estimation controller (126) computes an output of the depth estimation technique and the segmentation technique to generate a real time 3 -dimensional coordinates of the lesion in an optimized manner.

[0057] Further, the force control unit (128) executes a collision detection technique to constantly monitor the 3D position of a surgical instrument (110) along with the 3D position of the lesion. Further, the force control unit (128) generates an alert when the surgical instrument (110) transmits any force outside the 3D position of the lesion.

[0058] Further, the force control unit (128) triggers a force feedback mechanism, by issuing a stop command to an input current of an electrocautery instrument or by issuing a stop command to an end effector (135) in case the dual robotic arm (130) is being used to perform a surgical procedure. In an embodiment, the force feedback mechanism is triggered while detecting underlying blood vessels, or close proximity to deep tissue by using at least one of: the computer vision technique and the depth estimation technique.

[0059] In an embodiment, while the surgical procedure is happening, the force feedback mechanism simultaneously runs by continuously gauging the input from the force sensor (116) integrated with the surgical instrument (110) or the end effector (135) of the robotic arm (130). The force sensor (116) constantly measures the exact force at the point of control at the tip of the surgical instrument (110).

[0060] In an example, the surgical system (100) is an advanced robotic surgical platform that integrates the master computational and control unit (160) with the dual robotic arms (130) (or modified surgical instrument (HO)) during the surgery. The surgical system (100) receives real-time input from the computer vision handling controller (122), the force sensor (116), the position sensor (118), and the motion sensor (129) embedded in the dual robotic arms (130), as well as input data from the surgeon’s console which is then transmitted to the master computational and control unit (160). This data is processed to compute critical parameters like current, potential future and desired positions, applied and optimal forces, and proximity to the healthy tissue. Further, the surgical system (100) adjusts the robotic arms' movements and force application to align with surgical goals while preventing tissue damage, as the master computational and control unit (160) executes commands to the motors (114) of the surgical instrument (110) / end effectors (135) of the robotic arms (130) to hasten or stop movement and electrical connectivity. Further, the surgical system (100) also interfaces with the surgeon's console, interpreting commands and ensuring that the robotic arms execute precise, safe movements. Safety mechanisms, including damping techniques, prevent excessive force application, maintaining a steady and controlled operation throughout the surgical procedure.

[0061] The surgical system (100) can handle automated tasks such as needle pick up, suturing and knot tying, leading to enhanced precision and accuracy during diagnostic and therapeutic endoscopic procedures. The surgical system (100) is a closed-loop system that ensures that the robotic arms (130) operate with a high degree of precision, responding dynamically to the ever-changing conditions of the GI surgical environment. The integration of advanced computational techniques, real-time data processing, and safety mechanisms enables the surgical system (100) to support complex surgical procedures with minimal risk to the patient.

[0062] In an implementation, the surgical system (100) is an addition to an existing flexible endoscope and colonoscope device (that is an attachment which consists of multiple mechanical segments interconnected via various joints which constitute the robotic arm). The attachment can be connected to the endoscope via an over -tube structure (not shown). That is, the attachment can be placed on top of the diameter of an existing machine, or can be attached at a distal most end of the endoscope / colonoscope device in a fashion that may or may not include inflatable structures to support the distension of arms housed within the attachment.

[0063] In an implementation, the attachment can either house two robotic arms, i.e. a dual robotic arm (130), or a modified surgical instrument (110). The modified surgical instrument (110) is integrated with controls which can switch the electrical connectivity on or off according to input received by the surgical system (100), and is also equipped with the motors (114) to act dynamically according to kinematic output commands executed by the master computational and control unit (160) to accelerate or stop motion. Each robotic arm includes multiple proximal and distal segments which are connected via different joints to provide the necessary degrees of freedom. Miniaturized actuators (or micro actuators) (not shown) including electromechanical units (not shown) are components of each joint and enable movements of the joint. The distal most segment of the arm is connected to the end effectors (135), which are interchangeable surgical instruments. There is a joint connecting the end effector (135) with the distal most segment to enable motion and a variety of degrees of freedom of the end effector (135). There is a tool changing mechanism which enables each arm’s end effectors to change the instruments. These instruments include but are not limited to electrocautery devices, needles, etc. which are grasped by distal wrist through a joint which allows manipulation in multiple degrees of freedom. Each segment and joint contain sensors, which include position, force sensors, as well as input and output wiring to sense and control the force and position each segment and each arm as a whole to be manipulated in. Eachindividual segment is interconnected via wiring, and is connected with another segment through various joints which allow it to be manipulated across various degrees of freedom.

[0064] The robotic arm (130) comprises mechanical rotators (not shown) and pivots (electromechanical units) (not shown) which are controlled via input wires to move in a desired manner. Each segment of the robotic arm (130), including the end effector (135), are equipped with the position sensor (118), the motion sensor (129) and the force sensor (116). The actuators of the joints, electromechanical units, and sensors are interconnected via wires, and are capable of receiving and sending signals to the master computational and control unit (160). Each robotic arm (130) can be manipulated to provide translational as well as rotational degrees of freedom.

[0065] The robotic arms (130) together are capable of achieving triangulation, and are able to converge the end effectors (135), i.e. surgical instruments, to the desired field of the tissue, to perform the surgical procedure. The surgical instruments (110) are housed in a modular tool system with interchangeable distal ends. Manipulation of the surgical instruments (110) to various angles, and positions, is possible, to ensure the convergence of the instruments. The interchangeable tools allow for a variety of functions including retraction, electrocoagulation, cutting, of the tissue. Both arms have a dual control function that enables them to bring end effectors (135). The segments and the end effectors (135) of the dual robotic arm (130) are equipped with the position sensor (118), the motion sensor (129) and the force sensor (116).

[0066] The master computational and control unit (160) serves as a central hub for processing and managing intricate operations of the dual robotic arms (130) within the system (100). The master computational and control unit (160) is equipped to receive and interpret information from various components of the robotic arms (130), such as the joint actuators, segment electromechanical units, which include an extensive network of sensors such as position sensor (118), force sensor (116) and motion sensors (129). The input of the force sensor (116), the position sensor (118) and the motion sensors (129) present in the segments, joints and end effectors (135) of the dual robotic arms (130) are transmitted via the input wiring system to the monitor (155). The transmission occurs in real time, ensuring that the master computational and control unit (160) receives continuous input to compute details regarding the current position of each joint segment and arm, force being applied at various points of the arm as well as the overall motion. The master computational and control unit (160) is capable of receiving this input information, processing this information (via filtering, refining the rawdata) and subsequently computing in real time during the procedure to calculate or determine operational parameters. The parameters computed include the current position, desired position, current force, desired force, proximity to healthy tissue, proximity to threshold force of each individual segment as well as overall kinematics and dynamics of the robotic arm (130). This information is used to calculate whether the desired position can be achieved using the current state, exact motion to be undertaken to achieve desired state, as well as the current state of force, and control mechanisms to ensure steady output of optimal force. Using this information, the inbuilt technique can input and plot the exact position, force applied and motion of the instrument (as well as all segments and joints of the robotic arm) with respect to the tissue surroundings. The master computational and control unit (160) interfaces with the surgeon’ s master console, receiving input commands that reflect the surgeon's intended actions. These commands are interpreted by the system’s techniques to plot future and desired positions, angles, and motions of the robotic arms (130) and surgical instruments (110).

[0067] The force applied by the surgeon through the console is also factored into the control calculations. The master computational and control unit (160) determines how the input force should be transmitted through the robotic arms (130) to achieve the desired surgical outcomes, ensuring that the applied force is both effective and safe. This output is transmitted via the wires to the electromechanical units and joint actuators of the robotic arm (130). Using these computed parameters, the master control unit performs certain critical functions, such as position adjustment, force regulation and damping and safety mechanisms.

[0068] The master computational and control unit (160) determines (or calculates) whether the desired position of the robotic arms (130) can be achieved from the current state, considering the spatial constraints and the surgical environment. The master computational and control unit (160) determines the exact movements required, including joint angles, rotations, and linear displacements, to transition the arms from their current state to the desired position.

[0069] Further, the master computational and control unit (160) continuously compares the current force applied by the robotic arms (130) with the desired force levels. If discrepancies are detected, the master computational and control unit (160) computes the necessary adjustments to align the applied force with the desired levels. The master computational and control unit (160) includes control mechanisms that ensure the steady application of optimal force, avoiding sudden spikes that could result in tissue damage. In the event that the force exceeds or approaches the maximal threshold, the master computational and control unit (160)immediately intervenes by inhibiting additional force transmission. This is achieved through the generation of output signals that adjust the actuators to reduce the applied force.

[0070] The master computational and control unit (160) is equipped with damping mechanisms designed to respond to sudden changes in force during the procedure. If a rapid increase in force is detected, the master computational and control unit (160) activates the techniques (e.g., force control technique or the like) that dampen the force, reducing its impact on the tissue and preventing potential damage. These safety mechanisms ensure that the force applied to the tissue remains within safe, steady-state levels at all times. The force control techniques include but are not limited to mechanical control, by controlling via reducing or stopping signal to the motors ofjoints of robotic arm (130) / modified surgical instrument (110) to cease or resist mechanical motion, and electrical force control techniques for instruments such as electrocautery knives wherein stoppage of current via a current control mechanism of the modified surgical instrument (110) ceases operations, etc.

[0071] The master computational and control unit (160) can immediately compute certain threshold or warning states, such as when the maximal threshold of force exceeds or is about to exceed, and can subsequently inhibit the additional force which is transmitted using output signals. Additionally, the master computational and control unit (160) can compute sudden changes in force during the surgical procedure and can provide damping mechanisms via the force control technique to reduce sudden high impact forces that can be received to the tissue and ensure only a certain amount of force in a steady state is delivered to the tissue at all times.

[0072] The computed output parameters, including the precise position adjustments and force control signals, are transmitted back through the wiring system to the electromechanical units and joint actuators of the robotic arms (130). These components then execute the necessary adjustments in real time, aligning the robotic arms (130) with the calculated trajectories and force requirements.

[0073] In an example, the force sensor (116) is capable of sensing the force applied to the tissue, and is a part of a feedback loop to regulate and control the force to be applied. The force sensor (116), the master computational and control unit (160), the actuators, and as well as transmission for wiring form a force control system (not shown).

[0074] The force sensor (116) continuously monitors the force applied to the tissues by the end effectors (135). This data is transmitted in real time to the master computational andcontrol unit (160). The force sensor (116) is part of a complex feedback loop that includes visual and haptic feedback mechanisms. The complex feedback loop allows the surgical system (100) to dynamically adjust the applied force, preventing tissue damage such as bleeding or perforation. This data is transmitted via wiring to the master computational and control unit (160), where the data is translated, filtered and preprocessed. Raw data is computed to determine the exact spatio-temporal trajectory, desired state, and pattern of force application with respect to the surgical environment (by 3D reconstruction computer vision techniques which can scout deep underlying structures such as tissue or blood vessels). The processed data regarding current state (from joints) and future state (from surgeon’s console) is compared to predefined force levels and tissue-specific thresholds, as well as the desired force levels. The master computational and control unit (160) utilizes various techniques, such as error processing and adjusting techniques to measure the difference between the applied force and the desired force. If the threshold for the maximal force is exceeded, the error is corrected in magnitude. It is capable of predicting future errors, and hence the damping mechanism can control fluctuations due to the surgeon's movements. The control signals are sent to the actuators for control of the force transmission to the end effectors (135). Corrective control signals are sent to the actuators, which adjust the force transmitted to the end effectors (135) in real time. The actuators ensure that the force applied remains within safe limits, preventing excessive pressure on the tissue.

[0075] The feedback system handles threshold force detection, force reduction and real time force adjustment. Inbuilt mechanisms of the force feedback system include damping mechanisms which are triggered in certain events in order to prevent excess force liable to damage the tissue or cause complications such as bleeding and perforation. The force sensor (116) can detect and relay signals in real time, and are also part of visual as well as force feedback loops to modulate and regulate the force applied and transmitted to the tissue.

[0076] The position sensing and control system (165) ensures precise navigation and operation of the robotic arms (130) attached to the endoscope. The position sensor (118) is strategically placed at the joints of the robotic arms (130), continuously monitoring angles, rotations, and displacements in real time. They relay information about the current state of the robotic arm (130). This information is useful in calculations, such as determining paths to be followed by the robotic arm (130), once a visually desired state, as well as current and potential future state (surgeon’s console) is calculated. The position sensor (118) provides critical information for motion planning techniques. These techniques consider the current state,desired outcome, and potential future movements to map a spatiotemporal trajectory that ensures precision during procedures.

[0077] The position sensor (118) works in tandem with the computer vision handling controller (122), especially during tasks such as tissue incision after delineation in surgical procedures therapeutic procedure utilizing third space endoscopic principles or not. The system ensures that instruments do not move outside the delineated boundaries or penetrate deeper layers, by regulating force and direction. The position sensor (118) is integrated with visual techniques, particularly while executing operations such as cutting the tissue after delineation of boundaries and ensure the instruments are not traversed outside the boundaries of the delineated area of the pathology, as well as do not traverse the submucosal / mucosal layer, by regulating the force, by damping force if it exceeds threshold or suboptimal direction, which may lead to piercing the muscular tissue.

[0078] The position sensor (118) continually monitors the angles, rotation, and displacement of each joint and segment of the arm in real time, and this data is transferred to the master computational and control unit (160) in real time via wires. The sensor data, as well as data from the visual input field (via the endoscopic camera) using surgical instrument identification and computer vision techniques is used to determine the current state. The desired state is calculated by defining target points in accordance with the delineated lesion, as well as estimating the depth relative to lesion via 3D reconstruction techniques. The spatio temporal trajectory to achieve convergence and desired instrument control is calculated to achieve the desired state by the master console. The system can predict potential errors in the trajectory or force application, allowing for timely interventions. Position control is integrated with force control mechanisms to ensure boundary adherence and an accurate positioning and manoeuvring of instruments.

[0079] The endoscope / colonoscope device includes a camera (170), which displays the real time observations of the gastrointestinal tract, and displays them on the monitor (155). The endoscope / colonoscope consists of a processor (175) which is capable of hosting techniques which can interpret the video feed in real time during the procedure. The processor (175) is capable due to integration with GPU and availability of specified RAM to host the various techniques. The endoscopy workflow hosts predictive computer vision techniques which are capable of analyzing the video and breaking it down into frames. The computer vision techniques are trained to detect lesions in the video feed that may be present in the gastrointestinal tract. The lesions detected are highlighted via a bounding box in real timeduring the endoscopic procedure. This is displayed simultaneously on the monitor (155) which displays the same overlaying on the live video feed.

[0080] The techniques are capable of detecting lesions suspicious of gastrointestinal cancer (e.g., esophageal cancer, gastric cancer, colorectal cancer or the like) as well as associated precancerous states (e.g., Barrett's esophagus, early gastric cancer, colorectal polyps or the like). These detected lesions are highlighted via a bounding box and alerted to the performing endoscopist in real time. The endoscopist may pause the frame of the continuous video feed to determine the suspicious lesion. If found to be positive, they may additionally optically characterize the lesion (manually, via Narrow Band Imaging (NBI) mode or chromoendoscopy), as well as utilize the characterization techniques for assistance in determination. If found to be cancerous, the endoscopist may utilize the delineation techniques to visually mark the boundaries of the lesion. The delineation techniques are trained on segmented images of cancerous lesions and defines and highlights the boundaries of the cancerous lesion, inclusive of all margins.

[0081] The robotic arm (130) or the modified surgical instrument (110) and the computer vision handling controller (122)can interact with each other. One example of the same is once the delineation boundaries of the lesion are finalized, the surgeon may enter a control mode. In the control mode, the force transmitted to outside the boundaries of the lesion will not translate into movements by the end effectors (135). The delineation map is detected and transmitted to the master control system via the camera (170). The force sensor (116) and the position sensor (118)of the end effector (135) as well as the output via the master computational and control unit (160) to the force sensor (116) and the position sensor (118) are continuously transmitted to the master computational and control unit (160). The system is capable of calculating the force transmitted to the current state of the instrument, and the final state of the instrument. If force transmitted will lead to healthy tissue, or structures at risk being compromised by the instrument, such as the underlying blood vessels as detected by the computer vision techniques, the damping mechanism will be triggered disabling the transmission of the force to the end effector (135).

[0082] Adaptive force control mechanisms are also integrated, by computer vision techniques identifying the type of tissue in the surgical scene, as well as determining the physical properties of the tissue. The current force, as well as planned and desired trajectory are calculated via the master computational and control unit (160), and in case the exerted force is higher than that to achieve desired trajectory, a haptic feedback loop for force control is setin place, where the damping mechanisms reduce the force transmitted to prevent overshooting, and complications such as perforations.

[0083] The surgeon may choose to perform endoscopic submucosal dissection or endoscopic mucosal resection to remove the lesion endoscopically. The command for assembly of the dual robotic arm (130) is made, and the arms are automatically kept into position. The surgical instruments (110) are determined and selected via interchangeable tool mechanism, as the end effectors (135). The surgeon can operate both robotic arms (130) via a remote master console which transmits the force and position feedback to the end effectors (135). The end effectors (135) are capable of multiple degrees of freedom and can achieve triangulation. The master control compute and execution system has multiple operative modes. One such mode is the ESDZEMR mode. Each mode differs in the automated operations that can be performed during the procedure along with the end effectors (135) that are executed via the dual robotic arm (130). The ESD ZEMR mode can perform certain automations, via using an interaction between our computer vision techniques and force related techniques and the modular components of our robotic arm (130). In the ESD, EMR mode, delineation techniques automatically delineate the boundaries of the precancerous / cancerous lesions. The doctor can confirm or change the boundaries and has the ability to 'lock the boundaries' of delineation. This mode allows automated punctuation of the lesions using an electrocautery knife for physical delineation of the lesion. The 'cutting' mode of this lesion allows physicians to decide or automatically select the desired level of force, and trigger the force control automation mechanisms of our instrument. These force control automation mechanisms allow a steady output of force to cut the delineated lesion. This mechanism automatically detects force or position of instruments which could lead to cutting of tissue outside the delineated area which automatically triggers stop force techniques to immediately stop the movement of the instruments and prevents transmission of force. The force sensing feedback mechanism detects any variations from optimal force or sudden changes in force delivery and enables damping mechanisms or techniques to prevent sudden changes in force delivered to tissue. Any changes resulting in threshold level of force being achieved which has the potential to pierce healthy, non target tissue to be cut or vessels to be damaged and could potentially lead to perforation or bleed is automatically stopped from being transmitted using an emergency stop mechanism.

[0084] The surgical system (100) can be used during upper and lower gastrointestinal endoscopic procedures for the accurate and automated detection and delineation of gastrointestinal cancers. The surgical system (100) can also be used during the therapeuticprocedures via endoscopies, to control and ensure they are completed in an automated and highly precise manner.

[0085] The processor (175) may include one or a plurality of processors. At this time, one or a plurality of processors may be a general purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU).

[0086] One or a plurality of processors control the processing of the input data in accordance with a predefined operating rule or Al model stored in a non-volatile memory (not shown) and a volatile memory (not shown). The predefined operating rule or artificial intelligence model is provided through training or learning. Here, being provided through learning means that a predefined operating rule or Al model of a desired characteristic is made by applying a learning technique to a plurality of learning data.

[0087] The Al model may comprise of a plurality of neural network layers. Each layer has a plurality of weight values, and performs a layer operation through calculation of a previous layer and an operation of a plurality of weights. Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN) etc.

[0088] The learning technique is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction. Examples of learning techniques include, but are not limited to, supervised learning, unsupervised learning, semi -supervised learning, or reinforcement learning.

[0089] Although FIG. 1 shows various hardware components of the surgical system (100) but, it is to be understood that other embodiments are not limited thereon. In other embodiments, the surgical system (100) may include less or more number of components. Further, the labels or names of the components are used only for illustrative purposes and does not limit the scope of the invention. One or more components can be combined together to perform the same or substantially similar function in the surgical system (100).

[0090] FIG. 2 A and FIG. 2B are flow charts (S200) illustrating a method for handling a function of the surgical system (100), according to embodiments as disclosed herein.

[0091] At S202, the method includes detecting suspicious lesions in real time by using computer vision technique. At S204, the method includes generating the bounding box around the coordinates of the suspicious lesion.

[0092] At S206, the method includes displaying and highlighting the lesion with the generated bounding box output in real time on the monitor (155). At S208, the method includes choosing to freeze the frame to view the lesion for the predefined time or choosing to execute a segmentation technique using the at least one toggle (120) if a surgeon is satisfied that the lesion is suspicious and needs to undergo the therapeutic procedure. The therapeutic procedure can be, for example, but not limited to an Endoscopic Mucosal Resection (EMR), an Endoscopic Submucosal Dissection (ESD), a Polypectomy, an Endoscopic Full Thickness Resection (EFTR), a Hemostasis, Band Ligation, an Endoscopic Suturing, an Endoscopic Stenting, an Ablation Therapy (using at least one of: light, ultrasound, and radiofrequency), a Peroral Endoscopic Myotomy (POEM), an Endoscopic Retrograde Cholangiopancreatography (ERCP) with intervention, a Submucosal Tunneling Endoscopic Resection (STER), and a Third Space Endoscopy. The predefined time is set by the surgeon.

[0093] At S210, the method includes delineating the 2-dimensional coordinates of the lesion based on the computer vision technique and a segmentation technique. At S212, the method includes initiating a depth estimation technique to generate 3 -dimensional coordinates of the lesion.

[0094] At S214, the method includes computing an output of the depth estimation technique and the segmentation technique to generate a real time 3 -dimensional coordinates of the lesion in an optimized manner. At S216, the method includes executing a collision detection technique to constantly monitor the 3D position of the surgical instrument (110) along with the 3D position of the lesion. In an example, the collision detection technique and the boundary estimation technique ensure that the instrument does not damage tissue beyond the boundaries of lesion. In another example, an integrated with blood vessel detection and healthy tissue detection techniques to alert potential collision of instrument with these structures.

[0095] At S218, the method includes generating an alert when the surgical instrument (110) is outside the 3D position of the lesion. At S220, the method includes triggering a force feedback mechanism, by issuing a stop command to an input current of an electrocautery instrument or by issuing a stop command to an end effector (135) in case the dual robotic arm (130) is being used to perform a surgical procedure.

[0096] In an embodiment, the force feedback mechanism is triggered while detecting underlying blood vessels, or close proximity to deep tissue by using at least one of: the computer vision technique and the depth estimation technique.

[0097] In an embodiment, during the surgical procedure, the force feedback mechanism is simultaneously running by continuously analyzing the input from the force sensor (116) integrated with the surgical instrument (110), or the end effector (135) of the robotic arm (130). The force sensor (116) constantly transmits the exact force at the point of control. In case of fluctuations of force at the point of control, the damping force mechanism ensures the signal transmitted is smoothened to prevent erratic, haphazard transmission of force.

[0098] In case of potential harm to any of the underlying tissues, the force feedback mechanism delays the motion and force of the instrument or the end effector (135), by ceasing the connection to the electrocautery current in case of modified surgical instrument and in case of the robotic arm (130), the force feedback mechanism ceases the motion of the end effector (135) of the robotic arm (130).

[0099] Further, the method uses a force correction and damping module to detect excessive force fluctuations at the end effector (135) or end point of instrument, and point of physician’s contact, and diminish excess signals via a filtering technique to filter out force received at the end effector level.

[0100] Further, the method uses an advanced position sensing technique. The advanced position sensing technique takes output of force sensors (116), the vision techniques and the position sensors (118) to correct the real time 3D position; as per any deviation detected by the master control and computational system (170).

[0101] Further, the surgeon (e.g., gastroenterologist or the like) can switch off the computer handling controller (127)for lesion estimation, instrument estimation, force estimation, force feedback mechanism and depth sensing individually using at least one toggle (120) at a command system.

[0102] FIG. 3 is a flow chart (S300) illustrating the damping mechanism while handling the function of the surgical system (100), according to embodiments as disclosed herein.

[0103] At S302, the force transmitted by the surgeon' s console is input to the master console. At S304, the force sensor on the end effectors (135) transmits real time force data into the master console. At S306, the master console interprets the current state of force and adjusts the same to ensure surgeon' s desired force received.

[0104] At S308, constant monitoring of the force applied by the surgeon' s at the end effector (135), in case the transmitted force reaches the threshold or maximum state, a dampingmechanism is triggered. At S310, the surgeon' s input transmission is blocked and the force transmission to the end effector (135) is stopped. At S312, the warning on the monitor (155) is displayed that threshold force has been achieved and the damping mechanism is in place. At S314, the force transmission resumes and continues until the next instance of maximal force being achieved.

[0105] FIG. 4 is an example environment (400) in which an operation and function of the surgical system (100) is depicted, according to embodiments as disclosed herein. As shown in FIG. 4, the frames are input through the surgical system (100) to generate the 3- dimensional coordinates of the lesion. The monitor (155) displays the flagged lesion in real time and raises an alarm (if required). The frames are processed using a machine learning technique and an artificial intelligence technique.

[0106] The various actions in flow charts (S200 and S300) may be performed in the order presented, in a different order or simultaneously. Further, in some embodiments, some actions listed in FIG. 1, FIG. 2A and FIG. 2B may be omitted.

[0107] The embodiments disclosed herein can be implemented through at least one software program running on at least one hardware device and performing network management functions to control the network elements. The network elements shown in FIG. 1 include blocks which can be at least one of a hardware device, or a combination of hardware device and software module.

[0108] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and / or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of embodiments and examples, those skilled in the art will recognize that the embodiments and examples disclosed herein can be practiced with modification within the scope of the embodiments as described herein.

Claims

CLAIMSWe claim:

1. A method for handling a function of a surgical system (100), comprising: triggering a computer vision technique using the at least one toggle (120) to start process a real time endoscopic video feed; detecting a suspicious lesion by using the computer vision technique; generating a bounding box around coordinates of the suspicious lesion; displaying and highlighting the lesion with a generated bounding box output in real time on a monitor (155); and performing at least one of: choosing to freeze a frame to view the lesion for a predefined time, and choosing to execute a segmentation technique using the at least one toggle (120) if a user of a surgical system (100) is satisfied that the lesion is suspicious and needs to undergo a therapeutic procedure.

2. The method as claimed in claim 1, wherein the method comprises: delineating a 2-dimensional coordinate of the lesion based on the computer vision technique and the segmentation technique; initiating a depth estimation technique to generate 3 -dimensional coordinates of the lesion; and computing an output of the depth estimation technique and the segmentation technique to generate real time 3 -dimensional coordinates of the lesion in an optimized manner, wherein the computer vision technique, the segmentation technique and the depth estimation technique are handled by at least one of: a machine learning technique and an artificial intelligence technique.

3. The method as claimed in claim 2, wherein the method comprises: executing a collision detection technique to constantly monitor the 3D position of a surgical instrument (110) along with the 3D position of the lesion; and generating an alert when the surgical instrument (110) is outside the 3D position of the lesion.

4. The method as claimed in claim 2, wherein the method comprises: triggering a force feedback mechanism, by issuing a stop command to an input current of an electrocautery instrument or by issuing a stop command to an end effector (135) in case a dual robotic arm (130) is being used to perform a surgical procedure.

5. The method as claimed in claim 4, wherein the force feedback mechanism is triggered while detecting underlying blood vessels, or close proximity to deep tissue or a vulnerable structure by using at least one of: the computer vision technique and the depth estimation technique.

6. The method as claimed in claim 4, wherein the method uses a damping technique to prevent excessive force application during the surgical procedure, while maintaining a steady and controlled operation throughout the surgical procedure.

7. The method as claimed in claim 1, wherein the method comprises, by a master computational and control unit (160),: receiving at least one continuous input to compute details regarding a current position of each joint segment and an arm, force being applied at various points of the arm and an overall motion during a surgical procedure, wherein the at least one continuous input is obtained from at least one of: a force sensor (116), a position sensor (118), and a motion sensor (129); processing and computing the at least one continuous input to determine at least one operational parameter, wherein the at least one operational parameter comprises at least one of: current position, desired position, current force, desired force, proximity to healthy tissue, proximity to threshold force of each individual segment, and overall kinematics and dynamics of an arm; determining whether the desired position is achieved using a current state, the exact motion to be undertaken to achieve desired state, the current state of force, and a control mechanism to ensure steady output of an optimal force; inputting and plotting an exact position, force applied and motion of the surgical instrument (110) with respect to the tissue surrounding based on the at least one operational parameter; and receiving an input command that reflect an action intended by the user of the surgical system (100).

8. The method as claimed in claim 1, wherein the method comprises, by a master computational and control unit (160),: determining whether a desired position of a robotic arm (130) is achieved from a current state, spatial constraints and a surgical environment; and determines an exact movement required to transition the robotic arm (130) from the current state to a desired position based on the determination.

9. The method as claimed in claim 1, wherein the method comprises, by a master computational and control unit (160),: comparing a current force applied by a robotic arm (130) with a desired force level; and computing a necessary adjustment to align the current force with the desired levels based on the comparison, wherein the master computational and control unit (160) includes a control mechanism that ensures a steady application of optimal force, so as to avoid sudden spikes that result in tissue damage, and wherein the master computational and control unit (160) is equipped with at least one damping mechanism designed to respond to sudden changes in the applied force during the surgical procedure.

10. The method as claimed in claim 7, wherein the force sensor (116) is capable of sensing the force applied to the tissue, and is a part of a feedback loop to regulate and control the force to be applied to the tissue during the surgical procedure, wherein the force sensor (116) continuously monitors the force applied to the tissues by end effectors (135), wherein data associated with the force is transmitted in real time to the master computational and control unit (160).

11. The method as claimed in claim 7, wherein the force sensor (116) is included in a complex feedback loop, wherein the complex feedback loop comprises a visual and force feedback mechanism, wherein the complex feedback loop allows the surgical system (100) to dynamically adjust the applied force, and prevent the tissue damage during the surgical procedure, wherein the visual and feedback mechanism handles threshold force detection, force reduction and real time force adjustment to prevent tissue damage during the surgical procedure.

12. The method as claimed in claim 7, wherein a position sensor (118) provides a spatiotemporal trajectory that ensures a surgical accuracy during the surgical procedure, wherein the position sensor (118) is integrated with at least one visual technique, wherein the position sensor (118) continually monitors angles, rotation, and displacement of each joint and segment of the robotic arm (130), and data associated with the angles, the rotation, and the displacement of each joint and segment of the robotic arm (130) is transferred to the master computational and control unit (160).

13. The method as claimed in claim 1, wherein the surgical system (100) uses a visual technique for identifying a type of tissue in a surgical scene and a physical property of the tissue by using the computer vision technique, wherein the therapeutic procedure comprises at least one of: an Endoscopic Mucosal Resection (EMR), an Endoscopic Submucosal Dissection (ESD), a Polypectomy, an Endoscopic Full Thickness Resection (EFTR), a Hemostasis, Band Ligation, an Endoscopic Suturing, an Endoscopic Stenting, an Ablation Therapy (using at least one of: light, ultrasound, and radiofrequency), a Peroral Endoscopic Myotomy (POEM), an Endoscopic Retrograde Cholangiopancreatography (ERCP) with intervention, a Submucosal Tunneling Endoscopic Resection (STER), and a Third Space Endoscopy.

14. The method as claimed in claim 4, wherein while the surgical procedure is happening, the force feedback mechanism is simultaneously running by constantly taking an input from a force sensor (116) integrated with the surgical instrument (110) and an end effector (135) of the robotic arm (130).

15. The method as claimed in claim 14, wherein the force sensor (116) constantly transmits an exact force at the point of control and at a tip of the surgical instrument (110).

16. A surgical system (100), comprising: a computer vision handling controller (122) configured to: detect a suspicious lesion based on a computer vision technique; generate a bounding box around coordinates of a suspicious lesion by using on the computer vision technique; display and highlight the lesion with a generated bounding box output in real time on a monitor (155); and perform at least one of:choose to freeze a frame to view the lesion for a predefined time, and choose to execute a segmentation technique using the at least one toggle (120) if a user of the surgical system (100) is satisfied that the lesion is suspicious and needs to undergo a therapeutic procedure.

17. The surgical system (100) as claimed in claim 16, wherein the surgical system (100) comprises: a segmentation controller (124) configured to delineate 2-dimensional coordinates of the lesion based on the computer vision technique and the segmentation technique; a depth estimation controller (126) configured to initiate a depth estimation technique to generate 3 -dimensional coordinates of the lesion; and a segmentation controller (124) and the depth estimation controller (126) configured to compute an output of the depth estimation technique and the segmentation technique to generate a real time 3 -dimensional coordinates of the lesion in an optimized manner, and wherein the computer vision technique, the segmentation technique and the depth estimation technique are handled by using at least one of: a machine learning technique and an artificial intelligence technique.

18. The surgical system (100) as claimed in claim 17, wherein a force control unit (128) is configured to: execute a collision detection technique to constantly monitor the 3D position of the surgical instrument (110) along with the 3D position of the lesion; and generate an alert when the surgical instrument (110) transmits force outside the 3D position of the lesion.

19. The surgical system (100) as claimed in claim 17, wherein the force control unit (128) is configured to: trigger a force feedback mechanism, by issuing a stop command to an input current of an electrocautery instrument or by issuing a stop command to an end effector (135) in case a dual robotic arm (130) is being used to perform a surgical procedure.

20. The surgical system (100) as claimed in claim 19, wherein the force feedback mechanism is triggered while detecting underlying blood vessels, or close proximity to deep tissue ora vulnerable structure by using at least one of: the computer vision technique and the depth estimation technique.

21. The surgical system (100) as claimed in claim 19, wherein while the surgical procedure is happening, the force feedback mechanism is simultaneously running by constantly taking an input from a force sensor (116) integrated with the surgical instrument (110) or an end effector (135) of the robotic arm (130).

22. The surgical system (100) as claimed in claim 21, wherein the force sensor (116) constantly transmits an exact force at the point of control and at a tip of the surgical instrument (110).

23. The surgical system (100) as claimed in claim 16, wherein the master computational and control unit (160): receives at least one continuous input to compute details regarding a current position of each joint segment and arm, force being applied at various points of an arm and an overall motion during the surgical procedure, wherein the at least one continuous input is obtained from at least one of: a force sensor (116), a position sensor (118), and a motion sensor (129), processes and computes the at least one continuous input to determine at least one operational parameter, wherein the at least one operational parameters comprises at least one of: a current position, desired position, current force, desired force, proximity to healthy tissue, proximity to threshold force of each individual segment as well as overall kinematics and dynamics of the robotic arm (130); determines whether the desired position is achieved using the current state, exact motion to be undertaken to achieve desired state, the current state of force, and control mechanisms to ensure steady output of optimal force; inputs and plots an exact position, force applied and motion of the instrument with respect to the tissue surroundings based on the at least one operational parameter; and interfaces with the surgeon’s master console, receiving input commands that reflect an action intended by the user of the surgical system (100).

24. The surgical system (100) as claimed in claim 16, wherein the master computational and control unit (160): determines whether a desired position of a robotic arms (130) is achieved from a current state, a spatial constraint and a surgical environment; anddetermines exact movements required to transition the robotic arm (130) from the current state to a desired position based on the determination.

25. The surgical system (100) as claimed in claim 16, wherein the master computational and control unit (160): compares a current force applied by the robotic arm (130) with a desired force level; and computes a necessary adjustment to align the current force with the desired levels based on the comparison, wherein the master computational and control unit (160) includes a control mechanism that ensures the steady application of optimal force, avoiding sudden spikes that result in tissue damage, and wherein the master computational and control unit (160) is equipped with at least one damping mechanism designed to respond to sudden changes in the applied force during the surgical procedure.

26. The surgical system (100) as claimed in claim 16, wherein a force sensor (116) is capable of sensing the force applied to the tissue, and is a part of a feedback loop to regulate and control the force to be applied to the tissue during the surgical procedure, wherein the force sensor (116) continuously monitors the force applied to the tissues by an end effector (135), wherein data associated with the force is transmitted in real time to the master computational and control unit (160).

27. The surgical system (100) as claimed in claim 16, wherein a force sensor (116) is included in a complex feedback loop, wherein the complex feedback loop comprises a visual and force feedback mechanism, wherein the complex feedback loop allows the surgical system (100) to dynamically adjust the applied force, preventing tissue damage during the surgical procedure, wherein the visual and force feedback mechanism handles threshold force detection, force reduction and real time force adjustment.

28. The surgical system (100) as claimed in claim 16, wherein a position sensor (118) provides a spatiotemporal trajectory that ensures a surgical accuracy during the surgical procedure, wherein the position sensor (118) is integrated with at least one visual technique, wherein the position sensor (118) continually monitors angles, rotation, and displacement of each joint and segment of the arm, and data associated with the angles, the rotation, and thedisplacement of each joint and segment of the arm is transferred to the master computational and control unit (160).

29. The surgical system (100) as claimed in claim 16, wherein the surgical system (100) uses a visual technique mechanism for identifying a type of tissue in a surgical scene and a physical property of the tissue by using the computer vision technique, wherein the therapeutic procedure comprises at least one of: an Endoscopic Mucosal Resection (EMR), an Endoscopic Submucosal Dissection (ESD), a Polypectomy, an Endoscopic Full Thickness Resection (EFTR), a Hemostasis, Band Ligation, an Endoscopic Suturing, an Endoscopic Stenting, an Ablation Therapy (using at least one of: light, ultrasound, and radiofrequency), a Peroral Endoscopic Myotomy (POEM), an Endoscopic Retrograde Cholangiopancreatography (ERCP) with intervention, a Submucosal Tunneling Endoscopic Resection (STER), and a Third Space Endoscopy.

30. The surgical system (100) as claimed in claim 16, wherein the computer vision handling controller (122), the force sensor (116), the position sensor (118), and the motion sensor (129) are embedded in dual robotic arms (130).

31. The surgical system (100) as claimed in claim 16, wherein the surgical system (100) adjusts movement of the robotic arm (130) and force application to align with surgical goals while preventing tissue damage, as the master computational and control unit (160) executes at least one command to a motor (114) of at least one of: the surgical instrument and end effectors (135) of the robotic arms (130) to hasten or stop movement and electrical connectivity.

32. The surgical system (100) as claimed in claim 16, wherein the surgical instrument (110) is integrated with controls that switches electrical connectivity on or off according to input received by the surgical system (100), and is also equipped with motors (114) to act dynamically according to kinematic output commands executed by the master computational and control unit (160) to accelerate or stop motion.

Images