Auxiliary control devices, methods, and electrosurgical systems for energy equipment

By using image recognition technology to identify the spatial position and structure of the energy device and the tissue, and dynamically adjusting the energy output mode, the problem of dependence on energy output adjustment in sliding cutting scenarios is solved, and more precise and safer energy control is achieved.

CN122461026BActive Publication Date: 2026-11-10HANG AN MEDTECH (HANGZHOU) CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202610944366.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2025-08-22
Filing Date
2026-06-29
Publication Date
2026-11-10
Estimated Expiration
2046-06-29

AI Technical Summary

Technical Problem

In minimally invasive surgery, it is difficult to achieve dynamic and adaptive energy output adjustment in sliding cutting scenarios, relying on the operator's manual experience, which can lead to insufficient or excessive energy output, affecting hemostasis and causing tissue damage.

Method used

By using image acquisition and recognition technology, the spatial location and structure of energy devices and tissues can be identified, and the energy output mode can be dynamically adjusted, including the switching between cutting and coagulation modes. The image recognition unit can identify target objects and generate energy adjustment instructions, reducing the reliance on operator experience.

Benefits of technology

It enables precise, safe, and easy-to-operate energy output adjustment in sliding cutting scenarios, reducing reliance on operator experience and improving surgical safety and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122461026B_ABST
    Figure CN122461026B_ABST
Patent Text Reader

Abstract

This application relates to an auxiliary control device, method, and electrosurgical system for an energy device. The auxiliary control device includes: an image acquisition unit for acquiring image data including captured images; an image recognition unit for recognizing a target object in the captured images; the target object includes an object that reflects the operating state of the energy device; and a processing unit configured to determine the spatial position structure of the target object based on the object reflecting the operating state of the energy device, and to determine energy adjustment indication information of the energy generator of the energy device based on the spatial position structure; the spatial position structure of the target object at least indicates the proximity of the energy device to the sensitive treatment content of the mucosal tissue; the energy adjustment indication information includes at least a first energy mode and a second energy mode; in the first energy mode, the cutting energy output parameter is higher than the coagulation energy output parameter; and in the second energy mode, the coagulation energy output parameter is higher than the cutting energy output parameter.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application claims priority to Chinese patent application filed on August 22, 2025, with application number 202511185783X, entitled “Auxiliary Control Device and Method for Energy Devices, Endoscopic Device and Electrosurgical Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of minimally invasive surgical instruments, and in particular to energy-assisted control devices, methods, and electrosurgical systems. Background Technology

[0003] Currently, with the development of minimally invasive surgical techniques, endoscopic energy devices, due to their ability to achieve precise cutting and reliable hemostasis under high-frequency current and their good compatibility with minimally invasive instruments such as endoscopes, have been widely used in surgical scenarios involving the digestive tract, thoracic and abdominal cavities. Energy devices generate high-frequency, high-voltage or ultrasonic energy through an energy generator, which is transmitted to the device's tip via a connecting cable. The device then applies the energy to the patient's tissue. Adjustments to the cutting and coagulation parameters of the energy device are often made manually by the operator using a foot switch or knob. Therefore, this method is highly dependent on the operator's experience and can easily lead to incomplete hemostasis due to insufficient energy output, or tissue carbonization and thermal damage due to excessive energy output.

[0004] In related technologies, for surgical scenarios involving clamping and fixation, the energy applied to the clamped tissue is adjusted based on factors such as the type of tissue being clamped, the amount of tissue held, or the tissue tension. For example, when the clamped tissue is identified as a thin membrane that is easily cut, the energy output is reduced; when the clamped tissue is identified as a tissue that is difficult to cut, the energy output is increased. These technologies focus on adjusting the output energy in clamping and fixation surgical scenarios. In clamping and cutting scenarios, the instrument and the clamped tissue maintain a relatively stable clamping state, thus allowing energy adjustment based on the static parameters of the tissue. However, in sliding cutting scenarios, because the energy instrument tip slides continuously along the tissue surface, the tissue in contact with or near the energy instrument tip is constantly changing, making it difficult to obtain static parameters such as tissue type and tension. These static parameters also make it difficult to support energy adjustment in the dynamic scenarios of sliding cutting.

[0005] Therefore, the energy output parameters in sliding cutting scenarios still rely on the operator's manual experience and lack dynamic, adaptive energy output adjustment, which needs to be addressed. Summary of the Invention

[0006] This embodiment provides an auxiliary control device, method, and electrosurgical system for energy equipment to address the problems in related technologies that rely on operator experience and lack dynamic, adaptive energy output regulation.

[0007] In a first aspect, this embodiment provides an auxiliary control device for an energy device, used to assist the energy device in controlling the operation of an energy instrument under endoscopy; the energy instrument includes a connector electrically connected to the energy device and a tissue-processing tip, the tip of the energy instrument being capable of cutting and processing the mucosal tissue while sliding along the surface of the mucosal tissue; the auxiliary control device for the energy device includes:

[0008] An image acquisition unit is configured to acquire image data, including captured images;

[0009] An image recognition unit is configured to recognize a target object in the captured image; the target object includes an object that can reflect the operational state of the energy device.

[0010] The processing unit is configured to determine the spatial position structure of the target object based on the object that reflects the operating state of the energy device, and to determine the energy adjustment indication information of the energy generator of the energy device based on the spatial position structure; the spatial position structure of the target object is at least used to indicate the proximity of the energy device to the sensitive treatment content of the mucosal tissue; the proximity includes the energy device contacting or cutting the sensitive treatment content; the energy adjustment indication information includes at least a first energy mode and a second energy mode; the second energy mode is used when the energy device contacts or cuts the sensitive treatment content; in the first energy mode, the cutting energy output parameter is higher than the coagulation energy output parameter; in the second energy mode, the coagulation energy output parameter is higher than the cutting energy output parameter.

[0011] In some of these embodiments, the first energy mode includes: turning off coagulation energy output, setting the cutting energy output voltage amplitude to a reference cutting voltage value, and setting the cutting time percentage to 100%.

[0012] The second energy mode includes: setting the coagulation energy output voltage amplitude to be higher than the cutting energy output voltage amplitude, and setting the coagulation time percentage to be higher than the cutting time percentage.

[0013] In some embodiments, the energy adjustment indication information further includes a third energy mode; the third energy mode includes: setting the cutting energy output voltage amplitude to be equal to the coagulation energy voltage amplitude, and setting the cutting time percentage to be equal to the coagulation time percentage.

[0014] In some embodiments, the sensitive content of the mucosal tissue includes blood vessels; the proximity of the energy device to the sensitive content of the mucosal tissue includes the proximity of the energy device to the blood vessels; the proximity of the energy device to the blood vessels includes the distance between the tip of the energy device and the blood vessels in the captured image.

[0015] In some embodiments, the distance between the head of the energy device and the blood vessel in the captured image is determined based on the pixel distance between the energy device region and the blood vessel region in the captured image.

[0016] In some embodiments, the sensitive content of the mucosal tissue includes bleeding tissue; the proximity of the energy device to the sensitive content of the mucosal tissue includes the proximity of the energy device to the bleeding tissue.

[0017] In some embodiments, the image recognition unit identifies image features of the energy device from the captured image and locates the energy device in the captured image based on the image features.

[0018] In some embodiments, the image features include key points; the image recognition unit identifies key points of the energy device from the captured image based on a trained key point detection model, and determines the position information of the energy device based on the key points.

[0019] In some embodiments, the objects that can represent the working state of the energy device include the energy device posture and the working tissue;

[0020] The processing unit is further configured to determine the spatial position structure of the energy device posture and the working tissue based on the image data; send a prompting message to the prompting unit based on the spatial position structure of the energy device posture and the working tissue, or determine the contamination status of the energy device as a prompting message based on the color space detection result of the energy device in the spatial position structure of the energy device posture and the working tissue, so that the prompting unit can prompt the user based on the prompting message.

[0021] In some embodiments, the spatial location structure of the target object includes a geometric feature image of the energy device; the processing unit compares the geometric feature image of the energy device with the device structure diagram in the device structure diagram library to determine the type of the energy device, and determines the energy adjustment instruction information according to the output parameter configuration information corresponding to the type of the energy device.

[0022] Secondly, this embodiment provides an auxiliary control device for energy equipment, including:

[0023] An image acquisition unit is configured to acquire image data including captured images, wherein the captured images are images including at least one target object, and the target object is an object capable of displaying the state of the energy device; the object capable of displaying the state of the energy device is the location of the energy device's action and the tissue it acts upon.

[0024] The processing unit is configured to determine, based on the spatial position of the energy device and the tissue it acts on, whether the distance between the energy device and the nearest intersecting blood vessel on the predicted trajectory of the energy device is less than a preset safe distance. If so, the energy adjustment mode of the energy generator of the energy device is to trigger the coagulation mode and output high-frequency pulse energy; otherwise, the energy adjustment mode of the energy generator of the energy device is to keep the current energy output mode unchanged.

[0025] Thirdly, this embodiment provides an auxiliary control method for energy devices, applied to the auxiliary control device for energy devices described in the first aspect above; the method includes:

[0026] Acquire image data, including captured images;

[0027] Identify the target object in the captured image; the target object includes objects that can reflect the operational state of the energy device.

[0028] Based on the object that can reflect the working state of the energy device, the spatial position structure of the target object is determined, and the energy adjustment indication information of the energy generator of the energy device is determined based on the spatial position structure; the spatial position structure of the target object is at least used to indicate the proximity of the energy device to the sensitive treatment content of the mucosal tissue; the energy adjustment indication information includes at least a first energy mode and a second energy mode; in the first energy mode, the cutting energy output parameter is higher than the coagulation energy output parameter; in the second energy mode, the coagulation energy output parameter is higher than the cutting energy output parameter.

[0029] Fourthly, this embodiment provides an electrosurgical system, including an energy device, an endoscope system, and an auxiliary control device for the energy device as described in the first aspect; the auxiliary control device is connected to the energy device and the endoscope respectively.

[0030] Compared with related technologies, this embodiment provides an energy device auxiliary control device, method, and electrosurgical system. By identifying target objects that can reflect the working state of energy instruments in a sliding cutting scenario and introducing a dynamic spatial position structure as the core control variable, the problem of static parameter failure in a sliding cutting scenario is solved, the dependence on the operator's manual experience is reduced, and more accurate, safe, and easier-to-operate dynamic and adaptive energy output adjustment is achieved.

[0031] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0032] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0033] Figure 1 This is a schematic diagram of the structure of the energy equipment auxiliary control device according to an embodiment of this application;

[0034] Figure 2 This is a schematic diagram of an application scenario of the energy equipment auxiliary control device according to an embodiment of this application;

[0035] Figure 3a This is a system schematic diagram of an application scenario of the energy equipment auxiliary control device in this embodiment;

[0036] Figure 3b This is a schematic diagram showing the state where the energy device is in good contact with the target tissue in this embodiment;

[0037] Figure 3c This is a schematic diagram illustrating the state of poor contact between the energy device and the target tissue in this embodiment;

[0038] Figure 4a This is a schematic diagram of the predicted trajectory of the energy device according to an embodiment of this application;

[0039] Figure 4b This is a schematic diagram of the cutting and peeling operation of the head end of an energy device on tissue;

[0040] Figure 5a This is a waveform diagram showing the ratio of cutting time to clotting time as 5:5;

[0041] Figure 5b This is a waveform diagram showing the ratio of cutting time to clotting time as 8:2;

[0042] Figure 6 This is a schematic diagram of the data flow of an auxiliary control device for an energy equipment according to some embodiments of this application;

[0043] Figure 7 This is a schematic diagram illustrating the switching of energy output modes in some embodiments of this application;

[0044] Figure 8 This is a flowchart of an energy device auxiliary control method according to an embodiment of this application. Detailed Implementation

[0045] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0046] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by a person skilled in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, or B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0047] This embodiment provides an auxiliary control device for an energy device, used to assist the energy device in controlling the operation of an energy instrument under an endoscope; the energy instrument includes a connector that can be electrically connected to the energy device and a head end for processing tissue, the head end of the energy instrument being able to cut and dispose of mucosal tissue while sliding along the surface of mucosal tissue; Figure 1 This is a schematic diagram of the structure of the energy equipment auxiliary control device 100 in this embodiment. Figure 2 This is a schematic diagram of an application scenario of the energy equipment auxiliary control device in this embodiment, combined with... Figure 1 and Figure 2The energy device auxiliary control device 100 includes an image acquisition unit 102 configured to acquire image data including captured images; an image recognition unit 104 configured to recognize a target object in the captured images; the target object includes an object that can reflect the working state of the energy device; and a processing unit 106 configured to determine the spatial position structure of the target object based on the object that can reflect the working state of the energy device, and to determine energy adjustment instruction information of the energy generator of the energy device based on the spatial position structure; the spatial position structure of the target object is at least used to indicate the proximity of the energy device to the sensitive treatment content of the mucosal tissue; the proximity includes the energy device contacting or cutting the sensitive treatment content; the energy adjustment instruction information includes at least a first energy mode and a second energy mode; the second energy mode is used when the energy device contacts or cuts the sensitive treatment content; in the first energy mode, the cutting energy output parameter is higher than the coagulation energy output parameter; in the second energy mode, the coagulation energy output parameter is higher than the cutting energy output parameter.

[0048] The energy equipment auxiliary control device 100, serving as the decision-making end in this embodiment, can be composed of electronic devices such as computers, server devices, and terminals that perform data computation and information processing, or it can be implemented in the cloud. In specific hardware implementation, the energy equipment auxiliary control device 100 may include at least one processor as hardware. This processor may be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (GPU), or a programmable gate circuit (FPGA), etc. The energy equipment auxiliary control device 100 may also include an input / output unit 108, through which the energy equipment auxiliary control device 100 can acquire images from external sources, such as through... Figure 2 The endoscope system 200 provides a surgical field of view and acquires real-time image data of the surgical area. The input / output unit 108 can be a connector that connects to a cable supporting image signal transmission; alternatively, the input / output unit 108 can be an interface circuit that connects to the aforementioned connector and communicates with external systems such as the endoscope system 200. The input / output unit 108 can interact with the endoscope system 200, the energy generator 300 of the energy device, etc., for data exchange and command transmission. Furthermore, one or more processors of the energy device auxiliary control device 100 execute stored programs to implement the functions of the image acquisition unit 102, the image recognition unit 104, and the processing unit 106.

[0049] The image acquisition unit 102 receives image data containing captured images from the input / output unit 108 and transmits it to the image recognition unit 104. For example... Figure 2Optionally, the image acquisition unit 102 can also acquire real-time captured images via the imaging device 400 on the energy device. The image data acquired by the image acquisition unit 102 can be video stream data. The image recognition unit 104 can identify target objects from the captured images based on a configured machine learning model. These target objects include objects that can reflect the operating state of the energy device. Before identifying the target object, the captured images can be preprocessed. The processing unit 106 then determines the spatial position structure of the target object based on the object that reflects the operating state of the energy device, and then generates energy adjustment instruction information for the energy generator 300 of the energy device based on this spatial position structure. Understandably, when the image recognition unit 104 fails to identify a target object from the captured image, it can maintain the current energy output mode and continue energy output, or adjust the energy output based on signals transmitted from the head end of the energy device.

[0050] In addition, the energy device auxiliary control device 100 sends energy adjustment instruction information to the energy generator 300 of the energy device. Structurally, the energy device auxiliary control device 100 can be installed separately from the energy generator 300 or integrated with it. This energy adjustment instruction information can be specific adjustment values ​​for various energy parameters, allowing the energy generator 300 to directly adjust the energy output based on these values; or it can be a biased energy adjustment suggestion or strategy (e.g., a suggestion to adjust to cutting mode, a suggestion to adjust to coagulation mode, etc.), allowing the energy generator 300 to generate specific energy parameters based on the energy adjustment suggestion or strategy, combined with information such as tissue impedance. The adjusted energy parameters can be electrical or temporal characteristic quantities output by the energy generator to the energy instrument, used to change the thermal and mechanical effects when energy acts on biological tissue. These energy parameters include, but are not limited to, voltage amplitude, pulse width modulation (PWM) duty cycle, and cutting / coagulation time ratio.

[0051] The endoscope system 200, connected to the energy device auxiliary control device 100, can consist of an endoscope, a main unit, and a display. The endoscope can be a rigid surgical endoscope, its insertion section including a rigid tube, objective lens optics, camera element, illumination optics, transmission cables, and light guide; its tip is flexible. The operating section includes buttons and an angle control lever. The main unit can include an image processing processor and a light source for generating illumination light. The display is used to show endoscopic images.

[0052] The energy device involved in this embodiment specifically outputs energy via high-frequency electricity or ultrasound. As an execution terminal, it is an electrosurgical instrument used for cutting, coagulating, sealing, and hemostasis of biological tissue. The energy device includes a head and a connector. The head outputs energy and acts on the biological tissue; the connector connects to an energy device. The energy device can specifically be a device that supplies energy to the energy device, controls the energy output, and obtains feedback such as tissue impedance information. The energy device includes an energy generator 300. For example, as shown... Figure 2 The energy generator 300 can be composed of a DC power supply, an inverter / resonant circuit, an output transformer, a voltage / current sampling circuit, and a control circuit. The DC power supply provides the energy input for the entire electrosurgical unit. The inverter / resonant circuit converts the DC power generated by the DC power supply into high-frequency AC power to provide the electrosurgical energy required for cutting and coagulation. The output transformer is used to regulate the output voltage / current to match surgical requirements. The voltage / current sampling circuit is used to collect the output signal in real time and feed it back to the control circuit to prevent power overload or abnormalities. The control circuit receives energy adjustment instructions from the energy device auxiliary control device 100, dynamically adjusts the output mode of the energy instrument, and simultaneously receives the sampling signal for safe closed-loop control, achieving precise and safe tissue cutting and hemostasis.

[0053] Figure 3a This is a system schematic diagram of an application scenario of the energy equipment auxiliary control device in this embodiment; Figure 3b This is a schematic diagram showing the state where the energy device is in good contact with the target tissue in this embodiment; Figure 3c This is a schematic diagram illustrating the state of poor contact between the energy device and the target tissue in this embodiment. The following will be combined with... Figure 3a , Figure 3b as well as Figure 3c The sliding cutting operation of the energy device is explained. Specifically, such as... Figure 3aAs shown, the energy device may include a handle 311, a sheath 312, and a head end 313. Both the sheath 312 and the head end 313 are connected to the handle 311. Specifically, the proximal end of the sheath 312 is connected to the handle 311, and the head end 313 is connected to the handle 311 via an operating line. The sheath 312 at least partially confines the operating line and the head end 313 within it. The handle 311 can operate the head end 313 axially relative to the sheath 312 via the operating line. When the head end 313 extends out of the sheath, it can cut tissue. The operator can also axially operate the sheath 312 to achieve synchronous movement of the sheath 312 and the head end 313, thereby adjusting the position of the head end 313 within the human body and achieving sliding cutting of human tissue. Specifically, the head end 313 of this energy device can enter through the forceps port 201 of an endoscope and extend from the insertion part 202 of the endoscope to act on human tissue. Based on communication with the energy generator 300, the energy device auxiliary control device 100 sends energy adjustment instruction information to the energy generator 300 so that the energy generator 300 controls the head end 313 of the energy device to cut and / or coagulate the tissue when it slides on the human tissue, and controls the parameters of the specific output energy. Figure 3b This shows the state of good contact between the head end 313 of the energy device and the human tissue when the device moves on the tissue. Figure 3c This shows the state of poor contact between the head end 313 of the energy device and the human tissue when the device moves on the tissue.

[0054] In surgical scenarios involving clamping and fixation, it is often necessary to control the instruments to avoid blood vessels or related organs to prevent damage. In other words, these technologies must ensure that the instruments do not come into contact with blood vessels or related organs. Therefore, these technologies only provide energy output control while avoiding blood vessels and other tissues. In conjunction with the above... Figures 3a to 3c In the sliding cutting scenario addressed in this embodiment, the tip 313 of the energy instrument may come into contact with or sever blood vessels when cutting mucosal tissue, inevitably causing bleeding. Therefore, this embodiment requires energy output control to address this situation. When bleeding is anticipated or has already occurred, the energy mode can be switched to increase the coagulation energy output parameter to reduce bleeding. Conversely, when there is no bleeding, the cutting energy output parameter can be increased to improve cutting efficiency and reduce thermal damage. Thus, this embodiment is a solution proposed by those skilled in the art to address the limitations of existing technical solutions disclosed in related technologies, which fail to achieve energy control in sliding cutting scenarios where contact with or severing of blood vessels is unavoidable.

[0055] Therefore, the energy device auxiliary control device 100 of this embodiment can achieve energy device auxiliary control in sliding cutting scenarios. It does not rely on static tissue type or tension parameter analysis, but utilizes the dynamic spatial position structure of the target object in continuous frame images. Based on the proximity of the energy device to the sensitive processing content of the mucosal tissue indicated by the spatial position structure, it judges in real time the state of the energy device acting on the tissue in a sliding cutting scenario, thereby dynamically controlling the switching of cutting and coagulation modes or adjusting cutting and coagulation parameters. Therefore, compared to the static adjustment of energy level for clamped and fixed surgical scenarios in related technologies, this embodiment fills the technical gap of lacking automatic energy regulation in sliding cutting scenarios. It can respond to the constantly changing tissue environment and operational dynamics during sliding, achieving predictive safety control. This embodiment is applicable to dynamic scenarios where the energy device continuously slides and cuts along the tissue surface, expanding the clinical application scope of energy adaptive control technology. In summary, this embodiment solves the problem of static parameter failure in sliding cutting scenarios by identifying the target object and introducing spatial position structure as the core control variable, reduces the reliance on operator experience, and achieves more accurate, safe, and easier-to-operate dynamic and adaptive energy output adjustment.

[0056] also, Figure 3a The endoscope's operating section 203 is held and manipulated by the operator to control the rotation of the curved section at the tip of the endoscope insertion section 202; the endoscope's cable 204 transmits endoscope image signals to the processor 206 of the endoscope system 200 via connector 205. The endoscope system 200's display 207 is electrically connected to the processor 206 to display the endoscope images.

[0057] The following section provides a more detailed explanation of how the energy equipment auxiliary control device 100 achieves image data recognition and generates energy adjustment indication information.

[0058] The aforementioned target object can specifically be a set of one or more visual objects that can be identified through captured images and used to characterize the current operational state of the energy device. This operational state includes, but is not limited to: whether the energy device is activated, whether it is in contact with tissue, whether it is performing cutting or coagulation operations, and whether it is causing or controlling bleeding. This operational state can include various different types. Specifically, to avoid excessive tissue bleeding and incomplete hemostasis due to the movement and cutting of the energy device tip during sliding cutting scenarios, and to consider reducing bleeding based on coagulation function when predicting impending bleeding, the target object can include the energy device (e.g., the identified tip of the energy device) and the sensitive processing content of the mucosal tissue. Specifically, the sensitive processing content of the mucosal tissue can be specific tissues or states that require special attention or adjustment of energy output during the sliding cutting process along the mucosal tissue surface, such as blood vessels or bleeding tissue. Furthermore, the aforementioned target object can also include other visual objects (e.g., diseased tissue, nerve tissue, etc.). For ease of description, the following mainly uses energy devices and blood vessels / bleeding tissue as examples.

[0059] The identification of energy devices can be achieved based on key point detection. Specifically, the image recognition unit 104 can use a deep learning-based key point detection model to extract key points from the received captured image and determine the two-dimensional coordinate information of key points containing the position of the head end of the energy device, thereby constructing the spatial positioning information of the energy device.

[0060] Additionally, blood vessels can include arteries, veins, and their branches. Blood vessels can be identified by analyzing features such as morphology, color, and pulsatility within the region of interest (ROI). Alternatively, a pre-trained semantic segmentation model can be used to classify each pixel within the ROI as a blood vessel, thus achieving semantic segmentation and outputting a binary mask or probability distribution map of the blood vessel region. Optionally, this semantic segmentation model can employ an encoder-decoder structure, using multi-scale feature fusion to identify the blood vessel region. This semantic segmentation model can also be based on other machine learning models suitable for image segmentation; this embodiment does not impose any limitations on this. Furthermore, a pre-trained machine learning model can be used to identify active bleeding areas or bleeding points through semantic segmentation or keypoint extraction, thereby completing the identification of bleeding tissue. The ROI can be a pre-defined image region centered on the head of the energy device for vascular structure analysis. The specific size of the ROI can be set according to the scale of the energy device or the image resolution; this embodiment does not specify the exact size. When the energy device and such blood vessels are identified from the captured image, the state of contact, proximity, or interaction between the energy device and the blood vessel can be characterized based on the identified objects.

[0061] In addition, it can be understood that the image recognition unit 104 can also identify and segment key structures such as lesions, blood vessels and nerves in the image, and can also determine the type and state of the tissue acted upon by the head of the energy device (e.g., whether there is bleeding, tissue tension).

[0062] The spatial location structure is used at least to indicate the proximity of the energy device to the sensitive processing content of the mucosal tissue. Specifically, the spatial location structure can be a set of quantitative parameters determined based on the identified target object, used to describe the relative geometric relationship between different types of target objects, or a set of quantitative parameters representing the motion state of a target object itself. For example, it can be the distance between the energy device (specifically, the head end of the energy device) and the blood vessel, or the relative motion parameters of the energy device (such as the speed of movement, the predicted trajectory), or the angle, orientation, and posture of the energy device relative to the blood vessel. In some embodiments, the image recognition unit 104 first identifies the energy device from the captured image based on key point detection, and then performs local region modeling centered on the head end of the energy device to obtain the region of interest. From the region of interest, semantic segmentation is used to extract the distribution information of the blood vessel structure and the location of the bleeding tissue. Furthermore, based on the extracted image attributes of the energy device and the blood vessel / bleeding tissue (such as location, size, shape, etc.), the spatial location structure of the energy device and the blood vessel is further constructed to realize the spatial relationship analysis and temporal behavior modeling of the energy device and the blood vessel.

[0063] The spatial positioning structure can also indicate the cutting type when the energy instrument processes mucosal tissue. The cutting type refers to the type of cutting method the energy instrument uses during surgery to cut the mucosal tissue. Specifically, this can include fine dissection and cutting of the mucosal tissue, conventional cutting, and rapid separation. By changing the movement speed of the energy instrument, the cutting type can be adjusted. Different cutting types can be matched with different energy outputs, thus allowing for the determination of energy adjustment instructions based on the cutting type of the mucosal tissue.

[0064] It should also be noted that before target object recognition, the image recognition unit 104 can perform usability determination on the captured image of the current frame acquired by the image acquisition unit 102. Specifically, an image quality assessment model can be pre-built to comprehensively evaluate factors such as image clarity, bloodstain occlusion, highlight interference, and smoke that affect image quality and interfere with the surgical field of view in the received captured image of the current frame, in order to achieve usability determination. If, based on usability determination, it is determined that the usability of the captured image is insufficient to support subsequent accurate target object recognition, the energy control strategy driven by image recognition can be paused to maintain the current energy output, or switched to a preset safe energy output mode to avoid misjudgment caused by low-quality image analysis. When the image usability determination indicates that the quality and field of view of the captured image can support subsequent target object recognition, subsequent target object recognition and spatial location structure construction can be performed.

[0065] In this embodiment, image usability is assessed in real time by evaluating the clarity, occlusion level, and lighting interference of the captured images, thus resolving the problem of control failure caused by visual algorithm misjudgment under interference from smoke, bloodstains, or bright light. Through target object recognition and spatial positional relationship construction, the location of blood vessels and bleeding status can be predicted. Furthermore, an energy adaptive adjustment strategy based on the spatial positional structure of the target object is provided, enabling automatic switching between cutting and coagulation modes at different risk levels, improving the protection of tissues, especially blood vessels, within the surgical field. Thus, through the coordinated efforts of target object recognition and spatial positional structure construction, precise visual perception and risk quantification of the energy instrument operation area can be achieved.

[0066] Specifically, this embodiment establishes a core logic of using image vision as the perceptual eye, algorithms as the decision-making brain, and energy as the execution hand, breaking through the limitations of open-loop or simple threshold control in traditional sliding cutting scenarios. Through temporal analysis of the sequence of captured images, the tissue structure and motion characteristics of the target object (such as the trajectory, speed, or distance of the energy device) are extracted and fused to generate a dynamic spatial position structure. Based on this spatial position structure, energy adjustment suggestions are generated to dynamically reconstruct the cutting waveform and cut-coagulation ratio during the sliding cutting process, achieving on-demand energy output allocation and pre-intervention. Therefore, this embodiment achieves a leap from manual experience-based parameter tuning to data-driven adaptive control. Energy output is achieved by accurately constructing the dynamic spatial position structure of the target object in the sliding cutting scenario. For example, when the energy device is detected approaching bleeding tissue or blood vessels, the coagulation ratio and low-frequency energy output are automatically increased to shorten hemostasis time; by predicting the trajectory of the energy device, the cutting voltage is reduced and the waveform optimized before the energy device contacts the blood vessel, reducing unexpected vascular damage and postoperative bleeding rates. Therefore, it can also reduce the learning difficulty of operating energy devices and performing endoscopic surgery, shorten the training cycle, reduce the use of intraoperative hemostasis consumables, reduce medical costs, and improve the standardization of surgery and the safety of patient prognosis.

[0067] The energy adjustment indication information includes at least a first energy mode and a second energy mode; in the first energy mode, the cutting energy output parameter is higher than the coagulation energy output parameter; in the second energy mode, the coagulation energy output parameter is higher than the cutting energy output parameter.

[0068] The first energy mode is an energy output method that achieves efficient and stable sliding cutting of non-bleeding tissue while minimizing thermal damage. This can be a pure cutting mode or a hybrid cutting-coagulation mode where the cutting energy output parameters are higher than the coagulation energy output parameters. The second energy mode is an energy output method that automatically increases the coagulation energy parameters, adjusts the output voltage and duty cycle, and targets bleeding points or vascular ends to stop bleeding and prevent excessive carbonization. This can be a pure coagulation mode or a hybrid cutting-coagulation mode where the coagulation energy output parameters are higher than the cutting energy output parameters. Cutting energy output parameters are electrical and temporal parameters that enable tissue separation and cutting, specifically including the output voltage amplitude of the cutting energy, the PWM duty cycle, and the proportion of cutting time. Coagulation energy output parameters are electrical and temporal parameters that promote tissue protein denaturation, vascular closure, and hemostasis, specifically including the output voltage amplitude of the coagulation energy, the PWM duty cycle, and the proportion of coagulation time.

[0069] In one embodiment, the first energy mode includes: turning off the coagulation energy output, setting the cutting energy output voltage amplitude to a reference cutting voltage value, and setting the cutting time percentage to 100%. The second energy mode includes: setting the coagulation energy output voltage amplitude to be higher than the cutting energy output voltage amplitude, and setting the coagulation time percentage to be higher than the cutting time percentage. Furthermore, in one embodiment, the energy adjustment indication information also includes a third energy mode; the third energy mode includes: setting the cutting energy output voltage amplitude to be equal to the coagulation energy voltage amplitude, and setting the cutting time percentage to be equal to the coagulation time percentage.

[0070] In this embodiment, by setting different energy modes, the cutting energy output parameters and coagulation energy output parameters can be adjusted. It can also adaptively perform cutting and coagulation functions in sliding cutting scenarios according to the operation rhythm and state, taking into account both cutting efficiency and hemostasis.

[0071] Optionally, in one embodiment, the energy adjustment indication information may further involve at least one of output mode adjustment, output voltage adjustment, and output pulse configuration parameter adjustment, so that the energy device directly outputs energy according to the energy adjustment indication information; the output mode includes at least two of coagulation mode, cutting mode, and coagulation-cutting mixed mode; the output voltage adjustment is either an increase in output voltage or a decrease in output voltage; and the output pulse configuration parameters include output pulse width, output pulse interval, and output pulse effect.

[0072] That is, the output mode adjustment can be switching between output modes such as coagulation mode, cutting mode, and coagulation-cutting hybrid mode; the output power adjustment can be adjusting the output power by increasing the preset power, decreasing the preset power, or keeping the current output power unchanged; the output pulse configuration parameter adjustment can be increasing the preset width of the output pulse, decreasing the preset width; or increasing the preset interval of the output pulse, or decreasing the preset interval; or adjusting one or more of various output pulse effects. Therefore, the energy adjustment indication information in this embodiment can cover the adjustment of different types of energy parameters, thereby realizing fine adjustment of output energy in sliding cutting scenarios.

[0073] Specifically, in one embodiment, the sensitive content of the mucosal tissue includes blood vessels; the proximity of the energy device to the sensitive processing content of the mucosal tissue includes the proximity of the energy device to the blood vessels; the proximity of the energy device to the blood vessels includes the distance between the tip of the energy device and the blood vessels in the captured image. In a more specific embodiment, the distance between the tip of the energy device and the blood vessels in the captured image is determined based on the pixel distance between the energy device region and the blood vessel region in the captured image.

[0074] In this case, the processing unit 106 determines whether the distance between the energy device and the blood vessel is less than the preset safe distance. If so, the energy adjustment instruction is: increase the coagulation energy output parameter and decrease the cutting energy output parameter; otherwise, the energy adjustment instruction is: keep the current energy parameter unchanged.

[0075] For example, the spatial proximity between the energy device region and the blood vessel region in the captured image can be calculated using the minimum Euclidean distance. Alternatively, other distance calculation methods can be used to quantify spatial proximity, such as the distance between the geometric centers of the energy device region and the blood vessel region, or the boundary distance between their respective bounding rectangles. Furthermore, the spatial proximity between the energy device and the blood vessel can also be quantified based on the calculated projection distance along the direction of energy device movement or the direction of the blood vessel normal; or, the spatial proximity can be quantified based on the intersection-union ratio (IUU) of the energy device region and the blood vessel region in the captured image.

[0076] More specifically, in some embodiments, the image acquisition unit 102 captures the endoscopic video stream in real time as a captured image, and the image recognition unit 104 extracts key points of the energy device from the captured image, locates the position of the energy device, and then constructs a region of interest (ROI) centered on the head end of the energy device. The size of the ROI can be adaptively adjusted according to factors such as the scale of the energy device itself or the image resolution. Compared with the global image processing of related technologies, this embodiment, after identifying the position of the head end of the energy device, performs local region modeling around the energy device, which can effectively reduce the amount of data processing and computational complexity, enhance the ability to focus on key surgical operation areas, and thus improve the robustness of subsequent spatial location structure modeling. Then, semantic segmentation is performed on the image data of the ROI to identify the distribution characteristics and diameter characteristics of the main trunk or branches of the blood vessel. The processing unit 106 constructs a local two-dimensional spatial location structure based on the identified energy device and blood vessel, and determines the real-time distance between the blood vessel and the energy device. When this distance is less than a preset safety distance, the energy adjustment instruction information is determined to increase the coagulation energy output parameter and decrease the cutting energy output parameter. The processing unit 106 sends the generated energy adjustment instruction information to the energy generator, and the energy generator outputs an intelligent pulse waveform based on the energy adjustment instruction information.

[0077] For example, the safety distance can be set to 2mm. When the calculated distance between the energy device and the blood vessel is within the range of 0mm to 2mm, it is determined that the energy device is in a state of approaching or contacting the blood vessel. At this time, it is necessary to increase the coagulation energy output parameter and decrease the cutting energy output parameter. For example, when it is determined that the distance is less than the preset safety distance, the ratio of coagulation energy to cutting energy can be adjusted to 7:3. For example, the coagulation energy output voltage amplitude can be stabilized at 1050 volts (V), the cutting energy output voltage amplitude can be reduced to 450V, the PWM duty cycle corresponding to the coagulation mode can be 40%, the PWM duty cycle corresponding to the cutting mode can be 20%, the coagulation time can account for 70%, and the cutting time can account for 30%. The specific setting of the energy parameters can be either directly output to the energy generator 300 by the processing unit 106, or the processing unit 106 can generate an abstract directional signal, which is then generated by the energy generator 300 based on the received signal.

[0078] When the distance between the energy device and the blood vessel is greater than or equal to the aforementioned safe distance (i.e., greater than or equal to 2mm), the energy device can be considered to be within the safe distance range. At this point, the cutting mode can be maintained or resumed. The cutting energy output voltage amplitude can be maintained at 1200V, corresponding to a PWM duty cycle of 50% and a cutting time percentage of 100%. Furthermore, the coagulation energy output should be turned off to improve cutting efficiency while avoiding damage to surrounding tissues from excess energy.

[0079] Therefore, this embodiment quantifies the spatial proximity of the energy device and the blood vessel based on the distance calculation between them, thereby determining the contact risk between the energy device and the blood vessel. When the energy device approaches or contacts the blood vessel, the cutting energy output parameter is reduced and the coagulation energy output parameter is increased. This enables the generation of energy adjustment indication information based on contact risk. It can timely and accurately switch between cutting and coagulation and control parameters in the dynamic scenario of sliding cutting, and complete the on-demand allocation and pre-intervention of energy output in complex surgical fields, thereby improving the robustness of energy control and operational safety.

[0080] Furthermore, in one embodiment, the sensitive content of the mucosal tissue includes bleeding tissue; the proximity of the energy device to the sensitive processing content of the mucosal tissue includes the proximity of the energy device to the bleeding tissue. This proximity reflects whether bleeding tissue exists within a certain range of action of the energy device; this range of action can correspond to the region of interest in the captured image. First, the image acquisition unit 102 captures the endoscopic video stream in real time as the captured image, and the image recognition unit 104 extracts key points of the energy device from the captured image to locate the position of the energy device; after recognizing the head position of the energy device, the image recognition unit 104 crops the captured image with the head position as the center according to a preset window size to obtain the region of interest. Then, through a pre-trained machine learning model, by performing semantic segmentation or key point extraction on the region of interest, active bleeding areas or bleeding points can be identified, thereby completing the identification of the tissue bleeding state and determining whether bleeding tissue exists within the preset range of action of the energy device. When the processing unit 106 determines that there is bleeding tissue within the preset range of the energy device, it determines the energy adjustment instruction information as follows: reduce the cutting energy output parameter and increase the coagulation energy output parameter (specifically, it can adjust the output voltage to a safe coagulation threshold and increase the low-frequency pulse duty cycle). The energy generator 300 adjusts the output waveform based on the energy adjustment instruction information, and the frequency and pulse width of the output waveform can be dynamically mapped according to the severity of bleeding in the bleeding tissue.

[0081] This embodiment can dynamically adjust the cutting energy output parameters and coagulation energy output parameters by identifying bleeding tissue, thereby directly replacing the manual pedal adjustment mode in the traditional solution and realizing quantitative closed-loop control of hemostasis effect.

[0082] Furthermore, in some embodiments, the processing unit 106 generates energy adjustment instruction information based on the predicted trajectory of the energy device and the distribution information of blood vessels within the region of interest. More specifically, in one embodiment, the processing unit determines whether the energy device will come into contact with a blood vessel along the predicted trajectory based on the predicted trajectory of the energy device and the distribution information of blood vessels within the region of interest; if so, the energy adjustment instruction information is: during a preset first time period before the energy device comes into contact with the blood vessel, increase the coagulation energy output parameter and decrease the cutting energy output parameter.

[0083] Specifically, considering the operational characteristics of the energy device sliding and cutting under endoscopy, the motion trajectory of the energy device in the sliding and cutting state can be predicted. Specifically, the image acquisition unit 102 obtains several frames of captured images through the video stream acquired by the endoscope; the image recognition unit 104 can use the image sequence optical flow method and the temporal neural network to predict the motion trajectory of the head end of the energy device, and obtain the predicted trajectory of the energy device, which represents the future sliding path of the energy device. Figure 4aThis is a schematic diagram of the predicted trajectory of the energy device in this embodiment, as shown below. Figure 4a As shown, the sliding path (trajectory) of the tip of the energy device 401 before contacting the blood vessel 402 can be predicted and generated. Figure 4a The dashed line segment in the diagram indicates the predicted trajectory 403 of the energy device 401. Then, based on the identified spatial distribution of blood vessels, the probability of the energy device 401 intersecting with blood vessel 402 as it slides along the predicted trajectory 403 is calculated, and the contact time (TTC) between the energy device 401 and blood vessel 402 is predicted, for example... Figure 4a The predicted trajectory 403 of the medium-energy device intersects with blood vessel 402 at point A. Figure 4a (The part marked in red in the middle). Based on the prediction results, the processing unit 106 instructs the energy generator 300 to perform energy presetting within a preset first time period (e.g., 300 milliseconds (ms) to 500 ms) before the energy device contacts the blood vessel. Specifically, within this first time period, the output waveform can be smoothly transitioned from the waveform corresponding to the sharp cutting mode to the waveform corresponding to the soft cutting mode or the coagulation transition mode. The sharp cutting mode is an energy output mode with tissue separation as its primary goal; in the sharp cutting mode, the energy output can have a waveform with a higher voltage amplitude, a lower PWM duty cycle, and a smaller proportion of coagulation time. The soft cutting mode or the coagulation transition mode is an energy output mode that balances cutting efficiency and mild coagulation effect; compared to the sharp cutting mode, the soft cutting mode uses a medium voltage amplitude, a medium PWM duty cycle, and a moderately increased proportion of coagulation time. In addition, the voltage amplitude and PWM duty cycle of the energy output can be gradually adjusted.

[0084] The processing unit 106 can also continuously detect the actual sliding trajectory of the energy device. When it detects that the actual sliding trajectory deviates from the predicted trajectory and the degree of deviation exceeds the safety tolerance, it can immediately trigger preset protection logic. For example, it can suspend image-based energy output adjustment, switch to a preset safety mode, or downgrade to basic coagulation output until the trajectory is restored.

[0085] This embodiment is based on the construction of a spatial position structure of the predicted trajectory of the energy device and the distribution of blood vessels, and the energy adjustment realized based on this spatial position structure can achieve fine and real-time dynamic adjustment of energy parameters in the sliding cutting scenario. This makes up for the inapplicability of the control logic of clamping cutting in related technologies in the sliding cutting scenario, and realizes the non-intrusive protection of the energy device before contact with blood vessels.

[0086] Figure 4b This is a schematic diagram of the energy device's tip performing tissue cutting and peeling operations, such as... Figure 4bThe head 313 of the energy device makes a precise incision around the lesion along the annular markings 405, and gradually dissects the submucosa over a large area during the cutting process. For example... Figure 4b As shown by the dashed arrow, when the energy device performs a cutting operation, it can cut open to both sides at the incision site, thereby achieving the peeling of the submucosal layer. Combined with... Figure 4a and Figure 4b The head of the energy device moves along a certain trajectory on human tissue under the control of the operator, changing position as needed. Figure 4a The predicted trajectory 403 of the medium-energy device. Furthermore, tissue can be cut and dissected during movement. In this sliding cutting state, to achieve further image recognition processing of the area where the energy device tip is located, the image recognition unit 104 can set and mark a certain area as a region of interest, for example... Figure 4a The area enclosed by the solid circle in the middle, Figure 4b The area enclosed by the square dashed line.

[0087] In another embodiment, the processing unit 106 generates energy adjustment instruction information based on the movement speed of the energy device in the sliding cutting state. In one embodiment, the processing unit 106 determines whether the movement speed is within a preset cutting speed range; if so, the energy adjustment instruction information is: increase the cutting energy output parameter; otherwise, the energy adjustment instruction information is: make the cutting energy output parameter consistent with the coagulation energy output parameter.

[0088] To adapt to the operational characteristics of the energy device in a sliding cutting scenario, this embodiment constructs a spatial position structure including the movement velocity of the energy device. Specifically, the image acquisition unit 102 acquires consecutive image frames of the captured image, and the image recognition unit 104 can calculate a pixel-level optical flow field for the region of interest based on the consecutive image frames. Based on the optical flow field and under the mask constraint of the energy device, the motion information of the energy device is extracted. Specifically, the motion velocity value and motion direction of the energy device can be statistically aggregated based on the optical flow vector, thereby achieving robust estimation of the motion state of the energy device in the sliding state. In addition to the motion estimation based on optical flow calculation, the motion information of the energy device in the sliding state can also be determined based on feature point tracking, inter-frame difference of key points of the energy device, and the centroid motion of the energy device region.

[0089] Then, the processing unit 106 determines the energy adjustment instruction information based on the movement speed of the energy device, so that the energy release of the energy device matches the dynamic operating rhythm of the energy device in the sliding cutting state. It can adjust the voltage amplitude, PWM duty cycle, cutting time percentage, and coagulation time percentage of the energy output according to the real-time movement speed of the energy device, and drive the energy generator 300 to output corresponding waveforms so that the energy device acts on the tissue. This achieves real-time closed-loop control from visual perception to energy execution, ensuring that the energy output of the energy device is always adapted to different operating rhythms, avoiding insufficient energy output leading to incomplete cutting when the speed is too fast, and excessive energy causing tissue burns when the speed is too slow.

[0090] For example, the correspondence between the movement speed of the energy device and the adjustment of energy parameters can be shown in Table 1:

[0091] Table 1

[0092]

[0093] Figure 5a The waveform diagram shows the ratio of cutting time to clotting time in a 5:5 ratio. Figure 5b This is a waveform diagram showing the ratio of cutting time to clotting time in an 8:2 ratio. Figure 5a and Figure 5b In the waveform diagram, the horizontal axis represents time, and the vertical axis represents waveform amplitude. Within a complete waveform segment (corresponding to...),... Figure 5a The waveform of the complete working cycle of the "interval" is as follows: the first part of the waveform is a cutting waveform (characterized by continuous sine pulses with high output power), and the second part is a coagulation waveform (characterized by discontinuous sine pulses with lower output power); combined with the table... Figure 5a , Figure 5bBy setting different cutting and coagulation time ratios, both cutting and coagulation effects can be achieved. The aforementioned preset cutting speed range can be the set medium or high speed range of the energy device. More specifically, when the energy device's movement speed is in the range of 0mm / s to 1mm / s, it is suitable for fine dissection and cutting of tissue. Therefore, this speed requires reducing the energy output intensity to avoid energy accumulation and burning of tissue. At this time, it is necessary to reduce the voltage amplitude of cutting and coagulation, and the voltage amplitudes of the two should be consistent. The corresponding PWM duty cycle should be controlled at 30%, and the cutting and coagulation time ratio should be adjusted to 5:5. In this way, it can adapt to the fine operation of the energy device, reduce damage to surrounding normal tissue, and balance the gentleness of cutting and coagulation. When the energy instrument's movement speed is between 1 mm / s and 2 mm / s, it is in a medium-speed sliding state, suitable for routine tissue cutting. Therefore, this speed requires maintaining a standard energy output intensity, balancing cutting efficiency and safety. In this case, the voltage amplitude corresponding to cutting can be set to 1200V, and the voltage amplitude corresponding to coagulation to 1050V. The voltage amplitude of the cutting energy output is slightly higher than that of the coagulation voltage amplitude. Furthermore, the corresponding PWM duty cycle is controlled at 50%, and the time ratio for cutting and coagulation is adjusted to 7:3. This meets the operational requirements of routine surgery, achieving high cutting efficiency while managing minor bleeding risks. When the energy instrument's movement speed is between 3 mm / s and 5 mm / s, it is in a high-speed sliding state, suitable for rapid tissue separation. Therefore, this speed requires increasing the energy output intensity to improve cutting efficiency, achieving rapid and thorough cutting, and avoiding tissue dragging. At this point, the voltage amplitude corresponding to cutting can be set to 1350V, and the voltage amplitude corresponding to coagulation can be set to 1200V. The voltage amplitude of the cutting energy output is slightly higher than that of the coagulation voltage amplitude. Furthermore, the corresponding PWM duty cycle is controlled to be 60%, and the time ratio of cutting to coagulation is adjusted to 8:2. In this way, it can meet the rapid cutting operation requirements of energy instruments, achieve timely energy release, and reduce damage and bleeding caused by tissue dragging.

[0094] Therefore, in this embodiment, the output waveform is dynamically adjusted based on the movement speed of the energy device, which can achieve precise matching between the energy output and the actual operating rhythm and operating state of the energy device. This allows for both cutting effect and safety in sliding cutting scenarios, overcoming the problem that energy control in related technologies cannot match the operating rhythm.

[0095] Furthermore, in one embodiment, the image recognition unit identifies image features of the energy device from the captured image and locates the energy device in the captured image based on the image features. These image features include, but are not limited to, contour features, texture features, and color features of the energy device. The image features of the energy device can be extracted from the captured image based on a pre-built machine learning-based feature extraction model. More specifically, in one embodiment, the image features include key points; the image recognition unit identifies the key points of the energy device from the captured image based on a trained key point detection model and determines the location information of the energy device based on the key points.

[0096] Specifically, a pre-trained deep learning-based keypoint detection model can be used to extract keypoint features from the input captured image, outputting two-dimensional information containing the coordinates of keypoints at the head of the energy device. This embodiment uses keypoint detection to locate the energy device from the captured image, achieving high efficiency and low computational complexity, thus enabling accurate perception of critical surgical areas.

[0097] In another embodiment, the processing unit 106 is further configured to determine the visual credibility of the image data; when the visual credibility is lower than a preset credibility threshold, it generates energy adjustment indication information as follows: keep the current energy parameters unchanged; or switch to a preset safe energy output mode.

[0098] Specifically, this safe energy output mode can be a mode that uses more conservative and safer energy output parameters to minimize uncontrollable thermal damage or accidental cutting. In this embodiment, a visual credibility constraint mechanism can also be introduced. By evaluating the visual credibility of image data, when the visual information is determined to be unstable based on visual credibility, or when the recognition result of the target object fluctuates significantly, the influence weight of visual control on energy regulation can be reduced, or a more conservative energy control strategy can be switched to improve the stability and safety of energy control in complex surgical field environments. Specifically, the visual credibility of the current captured image can be determined based on a pre-built visual credibility detection model, taking into account factors such as whether there is significant bleeding in the captured image, whether the edges of the identified blood vessel region are blurred, and image quality, thus achieving objective quantification of visual information. Subsequently, the energy output is dynamically adjusted based on the level of visual credibility.

[0099] For example, a visual confidence threshold can be set to 80%. When visual confidence is detected to be less than 80% (e.g., there is a large area of ​​bleeding in the surgical field, or the identified blood vessel edges are blurry, causing recognition fluctuations with a fluctuation frequency greater than 5 times / second), the visual control weight can be reduced from 70% to 30% (to make the energy generator rely more on other non-visual information, such as tissue impedance, for energy control). At the same time, the energy output strategy is switched to a pre-set safe output mode. For example, regardless of the distance between the energy device and the blood vessel, the coagulation energy ratio can be increased to 50%, the cutting energy amplitude reduced to 1200V, the corresponding PWM duty cycle controlled at 35%, and the movement speed of the energy device limited to below 2mm / s. In addition, if visual confidence remains below 60% for more than 3 seconds, energy output can be paused and an audible warning tone indicating that visual confidence remains low can be issued to the operator until visual confidence stabilizes (e.g., it remains above 80% for a period of time), after which it automatically returns to the normal energy control mode.

[0100] This embodiment can establish an energy control fault-tolerant mechanism based on visual credibility. When the surgical field is severely obscured or the image quality deteriorates due to factors such as smoke, obstruction, or bleeding, it switches to a safe degraded energy output mode, which can avoid thermal damage caused by misjudgment based on visual information, thereby improving the safety in sliding cutting scenarios.

[0101] Furthermore, in one embodiment, closed-loop control of the energy generator 300 can be established. Specifically, the processing unit 106 can send energy adjustment instruction information to the energy generator 300. Based on the received energy adjustment instruction, the energy generator 300 adjusts the energy output and outputs adaptive pulse energy to the energy device head. Physical feedback signals such as tissue impedance changes and hemostasis effects generated after the energy device head acts on the tissue are collected. These physical feedback signals, along with the aforementioned visual reliability, are input into the closed-loop feedback channel of the energy generator 300, enabling the energy generator 300 to integrate and judge these two signals. The received physical feedback signals are then sent back to the processing unit 106 to adjust the aforementioned safety thresholds regarding distance, speed, etc., to form a complete closed-loop control of energy output.

[0102] This embodiment enables the coordinated design of energy output and energy circuit, optimizes visual blind spots, and continuously optimizes judgment and control strategies through the decision and learning model deployed in the processing unit 106, thereby improving the smoothness of the overall surgical operation and the degree of adaptation between energy control and operation rhythm.

[0103] Furthermore, in one embodiment, an endoscopic auxiliary hand can be used structurally to replace the separate external circuit electrode of the energy instrument. Specifically, this endoscopic auxiliary hand can be an auxiliary mechanical device used in endoscopic surgery. Integrating the circuit electrode of the energy instrument into the gripping arm surface of the endoscopic auxiliary hand avoids occupying additional operating channels and also avoids mechanical interference between the separate circuit electrode and the main operating instrument, as well as obstruction of the endoscopic field of view. In this way, the endoscopic auxiliary hand can simultaneously perform the functions of tissue clamping and traction to expose the surgical field, achieving structural reuse, streamlined channels, and unobstructed vision, thus expanding the operating space and safety of endoscopic surgery.

[0104] Optionally, in one embodiment, the objects that can represent the working state of the energy device also include the posture of the energy device and the working tissue; the processing unit 106 is further configured to determine a segmentation mask image including the segmentation mask of the energy device and the segmentation mask of the working tissue based on the image data, as the spatial position structure of the energy device posture and the working tissue; and send a prompting message to the prompting unit based on the spatial position structure of the energy device posture and the working tissue, or determine the contamination status of the energy device based on the color space detection result of the energy device in the spatial position structure of the energy device posture and the working tissue, as the prompting message, so that the prompting unit prompts the user based on the prompting message.

[0105] Specifically, the captured image can be input into a pre-trained image segmentation mask model in the image recognition unit 104. The image segmentation mask model generates a segmentation mask image containing a segmentation mask for the energy device and a segmentation mask for the target tissue, and sends the segmentation mask image to the processing unit 106. The segmentation mask for the energy device can distinguish various parts of the energy device, such as the head and handle, while the segmentation mask for the target tissue can distinguish tissue types such as mucosal protrusions and the submucosa. The processing unit 106 extracts the minimum distance information between the energy device head and the target tissue, the minimum distance information between the energy device head and the target tissue, and the pixel change rate information of the contact area between the energy device and the target tissue from the segmentation mask image. Based on one or more of these information, it determines the contact information between the energy device and the target tissue and sends it as a prompt to the prompting unit to alert the user. This prompting unit can be a display that synchronously displays endoscopic images, and it can also have a sound output component. This embodiment can provide intuitive prompts on the contact status of the energy instrument with the tissue under sliding cutting conditions, thereby assisting operators in completing the sliding operation of the energy instrument and improving operational efficiency.

[0106] In addition, by detecting the color space of the energy device surface, the contamination status of the energy device during operation can be analyzed, thereby reminding users of the surface dirt of the energy device and reducing the problem of increased tissue impedance due to surface dirt affecting the surgical process.

[0107] In addition, in one embodiment, the spatial location structure of the target object also includes a geometric feature image of the energy device; the processing unit compares the geometric feature image of the energy device with the device structure diagram in the device structure diagram library to determine the type of the energy device, and generates energy adjustment instruction information according to the output parameter configuration information corresponding to the type of energy device.

[0108] Specifically, the geometric feature image can be a feature image representing the structural attributes or surface attachments of an energy device, and at least includes lines that can identify the type of energy device. The geometric feature image of the energy device can be compared with pre-stored device structure diagrams of various energy device structures to identify the type of energy device relatively accurately. The inherent characteristics of the energy device are identified through its structural attributes or surface attachments, thereby determining the type of energy device.

[0109] Then, based on the determined type of energy device, the configuration information of the energy output parameters corresponding to that device type is retrieved from a pre-built energy parameter library, and this information is sent to the energy generator 300 as energy adjustment instruction information, so that the energy generator 300 can output energy based on the output parameter configuration information corresponding to that type of energy device. Understandably, different energy device types can correspond to different energy output parameter configuration information, which may include information such as energy output power, energy output mode, pulse width, and pulse interval. Optionally, if the geometric feature image of the energy device fails to match the device structure diagram in the device structure diagram library, a matching failure alarm can be sent to the prompting unit, so that the prompting unit provides an audible and visual alarm to the user.

[0110] This embodiment identifies the type of energy device used in the current energy equipment by recognizing the geometric features of the energy device, and then determines the energy adjustment instruction information based on the energy device type. This solves the problem of cumbersome device replacement procedures caused by doctors having to manually adjust the output energy output parameter configuration information when changing energy devices, thereby improving surgical efficiency.

[0111] Figure 6 This is a schematic diagram of the data flow of an auxiliary control device for energy equipment in some embodiments, such as... Figure 6As shown, the image acquisition unit 102 first acquires real-time video stream data, which includes several frames of captured images. Then, the visual recognition and key point extraction module in the image recognition unit 104 identifies the target object from the captured images. Specifically, it can extract key point coordinates or tissue classification features to identify the energy device, and then construct a region of interest based on the head position of the energy device, extracting the region of blood vessels / bleeding tissue through semantic segmentation. Next, the planar modeling and trajectory prediction module in the processing unit 106 can determine the distance between the energy device and the blood vessel, or the predicted trajectory and speed of the energy device, based on the key point coordinates and tissue classification features. Then, the intelligent energy decision and control module in the processing unit 106 generates energy adjustment instruction information and sends it to the energy generator 300. The energy generator 300 outputs adaptive pulse energy based on the energy adjustment instruction information, which acts on the tissue through the head of the energy device, and collects physical feedback signals such as tissue impedance changes and hemostasis effects. Through a closed-loop feedback channel, this type of physical feedback signal, combined with the visual credibility determined by the image recognition unit 104, is sent back to the processing unit 106 to adjust various thresholds when generating energy adjustment instruction information.

[0112] Figure 7 This is a schematic diagram illustrating the switching of energy output modes in some embodiments, such as... Figure 7 As shown, the energy output modes can include manual mode, intelligent cutting mode, intelligent coagulation mode, and safety degradation mode. When the energy device is initialized or the operator takes over control, the energy output mode is in manual mode, controlled by the operator. When non-bleeding tissue is identified based on the captured image or the visual reliability of the image data is high, the manual mode can be switched to intelligent cutting mode. This intelligent cutting mode aims to achieve efficient and smooth sliding cutting of non-bleeding tissue while minimizing thermal damage. When the visual reliability is detected to be lower than a preset reliability threshold, or the actual sliding trajectory of the energy device deviates from the predicted trajectory, the intelligent cutting mode can be switched to safety degradation mode. Safety degradation mode is a conservative strategy for energy output, which reduces energy output power, increases the proportion of coagulation time, and limits the maximum voltage or PWM duty cycle to avoid uncontrollable thermal damage or accidental cutting as much as possible. After switching to safety degradation mode, if the field of view is restored or the visual reliability recovers, the device can switch back to intelligent cutting mode.

[0113] Furthermore, while in intelligent cutting mode, if the distance between the energy device and the blood vessel is detected to be less than the safe distance, or if active bleeding is detected, or if the predicted trajectory of the energy device intersects with the blood vessel, the mode can switch from intelligent cutting mode to intelligent coagulation mode. Intelligent coagulation mode automatically increases the coagulation energy ratio, adjusts the output voltage and duty cycle, and targets bleeding points or blood vessel ends to stop bleeding and prevent excessive carbonization. After switching to intelligent coagulation mode, if bleeding stops and the energy device returns to the safe distance from the blood vessel, the mode can switch back to intelligent cutting mode.

[0114] When in intelligent coagulation mode, if the visual confidence level is detected to be lower than the preset confidence level threshold, or if the actual sliding trajectory of the energy device deviates from the predicted trajectory, the mode can be switched to safety degradation mode. Additionally, when in intelligent cutting mode, intelligent coagulation mode, or safety degradation mode, if operator intervention or foot control is detected, the mode switches to manual mode.

[0115] This embodiment provides an auxiliary control device for an energy device, comprising: an image acquisition unit configured to acquire image data including captured images, wherein the captured images are images including at least one target object, and the target object is an object capable of displaying the state of the energy device; the object capable of displaying the state of the energy device is the location of the energy device and the tissue it acts upon; and a processing unit configured to, based on the spatial position structure of the location of the energy device and the tissue it acts upon, determine whether the distance between the energy device and the nearest intersecting blood vessel on the predicted trajectory of the energy device is less than a preset safety distance; if so, the energy adjustment mode of the energy generator of the energy device is to trigger a coagulation mode and output high-frequency pulse energy; otherwise, the energy adjustment mode of the energy generator of the energy device is to maintain the current energy output mode unchanged.

[0116] In this embodiment, the captured images in the acquired image data must include at least one target object, which is an object capable of displaying the state of the energy device. Further, the object capable of displaying the state of the energy device can be the energy device's operating location and its corresponding tissue; that is, the captured image must be able to display the energy device's operating location and corresponding tissue. The identification of the energy device's operating location can be achieved by identifying the energy device's operating head. Therefore, the captured image containing the energy device's operating location and its corresponding tissue must at least include the energy device's operating head and the corresponding tissue.

[0117] The trajectory processing unit in the specific processing section determines the distance between the energy device and the nearest intersecting blood vessel on the current predicted trajectory of the energy device, based on the predicted trajectory of the energy device and the spatial structure of the tissue it affects, and the distribution of blood vessels. It then checks if this distance is less than a preset safety distance. If so, the trajectory processing unit sends an energy adjustment signal to the energy generator of the energy device, triggering a coagulation mode and outputting high-frequency pulse energy. This causes the energy generator to immediately trigger the "coagulation mode" upon receiving the energy adjustment signal and switch the energy device's output to high-frequency pulse energy (e.g., a 100kHz intermittent waveform) for rapid hemostasis. Conversely, if the distance between the energy device and the nearest intersecting blood vessel on the current predicted trajectory is greater than or equal to the preset safety distance, the trajectory processing unit either sends an energy adjustment signal to the energy generator of the energy device to maintain the current energy output mode, or it does not send an energy adjustment signal, thus ensuring that the energy generator of the energy device maintains its current energy output mode. The preset safety distance can be set to a preset value or a value that is dynamically adjusted based on the current inertia of the energy device and the diameter of the involved blood vessels.

[0118] This embodiment eliminates the need for doctors to manually adjust the energy output in existing energy equipment usage scenarios, greatly simplifying the doctor's operation process and significantly improving the convenience of using energy equipment. It also reduces the requirements for the accuracy of doctors' parameter adjustments, allowing doctors to perform surgeries smoothly without manually adjusting the energy output of the energy equipment, thus achieving the goal of simplifying doctors' operations.

[0119] This embodiment provides an auxiliary control method for energy devices, which can be applied to the auxiliary control device for energy devices in any of the above embodiments. Figure 8 This is a flowchart of the energy device auxiliary control method in this embodiment, such as... Figure 8 As shown, the process includes the following steps:

[0120] Step S801: Acquire image data including the captured image;

[0121] Step S802: Identify the target object in the captured image; the target object includes an object that can display the operating status of the energy device;

[0122] Step S803: Determine the spatial position structure of the target object based on the object that can reflect the working state of the energy device, and determine the energy adjustment indication information of the energy generator of the energy device based on the spatial position structure; the spatial position structure of the target object is at least used to indicate the proximity of the energy device to the sensitive treatment content of the mucosal tissue; the energy adjustment indication information includes at least a first energy mode and a second energy mode; in the first energy mode, the cutting energy output parameter is higher than the coagulation energy output parameter; in the second energy mode, the coagulation energy output parameter is higher than the cutting energy output parameter.

[0123] Through the above steps S801 to S803, it can identify objects that can reflect the working state of the energy device and introduce spatial position structure as the core control variable, thus solving the problem of static parameter failure in sliding cutting scenarios, reducing the dependence on the operator's manual experience, and realizing more accurate, safe, and easier-to-operate dynamic and adaptive energy output adjustment.

[0124] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0125] Furthermore, this embodiment also provides an electrosurgical system, such as Figure 2 As shown, the electrosurgical system may specifically include an energy device, an endoscope system 200, and an energy device auxiliary control device 100 provided in any of the above embodiments; the energy device auxiliary control device 100 is connected to the energy device and the endoscope system 200 respectively; wherein, the energy device includes an energy generator 300.

[0126] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0128] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0129] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An auxiliary control device for energy equipment, characterized in that, This device is used to assist in controlling the operation of an energy instrument under endoscopy; the energy instrument includes a connector electrically connected to the energy device and a tissue-processing tip, the tip of which is capable of cutting and dissecting the mucosal tissue while sliding along the surface of the mucosal tissue; The energy equipment auxiliary control device includes: An image acquisition unit is configured to acquire image data, including captured images; An image recognition unit is configured to recognize a target object in the captured image; the target object includes an object that can reflect the operational state of the energy device. The processing unit is configured to determine the spatial position structure of the target object based on the object that reflects the operating state of the energy device, and to determine the energy adjustment indication information of the energy generator of the energy device based on the spatial position structure. The spatial position structure of the target object is at least used to indicate the proximity of the energy device to the sensitive treatment content of the mucosal tissue. The proximity includes the energy device contacting or cutting the sensitive treatment content. The spatial position structure also indicates the cutting type when the energy device treats the mucosal tissue. The energy adjustment indication information includes at least a first energy mode and a second energy mode. The first energy mode is used when the cutting type is conventional cutting or rapid separation of the mucosal tissue. The second energy mode is used when the energy device contacts or cuts the sensitive treatment content. In the first energy mode, the cutting energy output parameter is higher than the coagulation energy output parameter. In the second energy mode, the coagulation energy output parameter is higher than the cutting energy output parameter.

2. The energy equipment auxiliary control device according to claim 1, characterized in that, The first energy mode includes: turning off coagulation energy output, setting the cutting energy output voltage amplitude to the reference cutting voltage value, and setting the cutting time percentage to 100%; The second energy mode includes: setting the coagulation energy output voltage amplitude to be higher than the cutting energy output voltage amplitude, and setting the coagulation time percentage to be higher than the cutting time percentage.

3. The energy equipment auxiliary control device according to claim 1, characterized in that, The energy adjustment indication information also includes a third energy mode; The third energy mode includes: setting the output voltage amplitude of cutting energy to be equal to the voltage amplitude of coagulation energy, and setting the cutting time percentage to be equal to the coagulation time percentage.

4. The energy equipment auxiliary control device according to claim 1, characterized in that, The sensitive content of the mucosal tissue includes blood vessels; the proximity of the energy device to the sensitive content of the mucosal tissue includes the proximity of the energy device to the blood vessels; the proximity of the energy device to the blood vessels includes the distance between the head of the energy device and the blood vessels in the captured image.

5. The energy equipment auxiliary control device according to claim 4, characterized in that, The distance between the head of the energy device and the blood vessel in the captured image is determined based on the pixel distance between the energy device region and the blood vessel region in the captured image.

6. The energy equipment auxiliary control device according to claim 1, characterized in that, The sensitive contents of the mucosal tissue include bleeding tissue; the proximity of the energy device to the sensitive contents of the mucosal tissue includes the proximity of the energy device to the bleeding tissue.

7. The energy equipment auxiliary control device according to claim 1, characterized in that, The image recognition unit identifies image features of the energy device from the captured image and locates the energy device in the captured image based on the image features.

8. The energy equipment auxiliary control device according to claim 7, characterized in that, The image features include key points; the image recognition unit identifies the key points of the energy device from the captured image based on the trained key point detection model, and determines the position information of the energy device based on the key points.

9. The auxiliary control device for energy equipment according to claim 1, characterized in that, The objects that can reflect the working state of the energy device include the energy device's posture and the tissue in which it operates; The processing unit is further configured to determine the spatial position structure of the energy device posture and the working tissue based on the image data; send a prompting message to the prompting unit based on the spatial position structure of the energy device posture and the working tissue, or determine the contamination status of the energy device as a prompting message based on the color space detection result of the energy device in the spatial position structure of the energy device posture and the working tissue, so that the prompting unit can prompt the user based on the prompting message.

10. The auxiliary control device for energy equipment according to claim 1, characterized in that, The spatial location structure of the target object includes the geometric feature image of the energy device; the processing unit compares the geometric feature image of the energy device with the device structure diagram in the device structure diagram library to determine the type of the energy device, and determines the energy adjustment instruction information according to the output parameter configuration information corresponding to the type of the energy device.

11. An energy device auxiliary control method applied to the energy device auxiliary control device according to any one of claims 1 to 10, characterized in that, include: Acquire image data, including captured images; Identify the target object in the captured image; The target object includes objects that can reflect the operational state of the energy device; Based on the object that can reflect the working state of the energy device, determine the spatial position structure of the target object, and determine the energy adjustment indication information of the energy generator of the energy device based on the spatial position structure; The spatial location structure of the target object is at least used to indicate the proximity of the energy device to the sensitive treatment content of the mucosal tissue; The energy adjustment indication information includes at least a first energy mode and a second energy mode; in the first energy mode, the cutting energy output parameter is higher than the coagulation energy output parameter; in the second energy mode, the coagulation energy output parameter is higher than the cutting energy output parameter.

12. An electrosurgical system, characterized in that, It includes an energy device, an endoscope system, and an auxiliary control device for the energy device as described in any one of claims 1 to 10; the auxiliary control device for the energy device is connected to the energy device and the endoscope, respectively.

Citation Information

Patent Citations

  • Computer vision based control of an energy generator

    US20230284870A1