Treatment system, control device and control method

Abstract

In order to detect various anomalies of the end effector and improve convenience, the treatment system (1) includes: a treatment device (41) that applies treatment energy from the end effector (415) to biological tissue based on supplied power, thereby treating the biological tissue; a camera device (21) that generates a camera image of the state in which treatment energy is applied from the end effector (415) to the biological tissue; and a control device (22) having a processor (225) for controlling the operation of the treatment device, wherein the processor is configured to: acquire the camera image; determine, based on the camera image, whether an anomaly has occurred in the end effector; and, if the anomaly is determined to have occurred, stop supplying power to the treatment device.

Description

Technical Field

[0001] This invention relates to a treatment system, a control device, and a control method. Background Technology

[0002] Previously, a treatment system was known, comprising: a treatment device that applies treatment energy from an end effector to biological tissue based on supplied power, thereby treating the biological tissue; and a treatment energy generator that supplies power to the treatment device (see, for example, Patent Document 1).

[0003] In the treatment device described in Patent Document 1, ultrasonic energy is used as the treatment energy. Specifically, the treatment device includes an ultrasonic transducer that generates ultrasonic vibrations based on electricity supplied from a treatment energy generator. Furthermore, the end effector of the treatment device is composed of a vibration transmission member that transmits the ultrasonic vibrations, and applies the transmitted ultrasonic vibrations to biological tissue. In addition, in the treatment energy generator described in Patent Document 1, abnormalities of the end effector (bending, breakage, or detachment) are detected, and the occurrence of such abnormalities is displayed on a display unit.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent No. 4813097 Summary of the Invention

[0007] The problem the invention aims to solve

[0008] In addition to the aforementioned bending, breakage, and detachment, other abnormalities of the end effector may also include excessive burning. The treatment system described in Patent Document 1 is not capable of detecting these multiple abnormalities.

[0009] Therefore, a technology that can detect these multiple anomalies and improve convenience is desired.

[0010] The present invention was made in view of the above circumstances, and its object is to provide a processing system, control device and control method that can improve convenience.

[0011] Solution for solving the problem

[0012] To address the aforementioned problems and achieve the objective, the treatment system of the present invention comprises: a treatment apparatus that applies treatment energy from an end effector to biological tissue based on supplied power, thereby treating the biological tissue; a camera device that generates a camera image of the state in which the treatment energy is applied to the biological tissue from the end effector; and a control device having a processor that controls the operation of the treatment apparatus, wherein the processor performs the following processes: acquiring the camera image; determining, based on the camera image, whether the end effector has malfunctioned; and, if it is determined that the end effector has malfunctioned, stopping the supply of power to the treatment apparatus.

[0013] The control device of the present invention includes a processor for controlling the operation of a treatment apparatus, wherein the treatment apparatus applies treatment energy from an end effector to biological tissue based on supplied power, thereby treating the biological tissue. The processor performs the following processing: acquiring a camera image of the state in which the treatment energy is applied to the biological tissue from the end effector; determining, based on the camera image, whether the end effector has malfunctioned; and stopping the supply of power to the treatment apparatus if the end effector is determined to be malfunctioning.

[0014] The control method involved in this invention is a control method executed by the processor of a control device, wherein a camera image is acquired showing the state of treatment energy being applied to biological tissue from an end effector; based on the camera image, it is determined whether the end effector has malfunctioned; and if it is determined that the end effector has malfunctioned, the power supply to the treatment device is stopped.

[0015] The effects of the invention

[0016] The processing system, control device, and control method according to the present invention can improve convenience. Attached Figure Description

[0017] Figure 1 This is a diagram illustrating the processing system involved in Embodiment 1.

[0018] Figure 2 This is a flowchart illustrating the control method executed by the first processor.

[0019] Figure 3 This is a diagram illustrating the device identification process (step S2).

[0020] Figure 4 This is a diagram illustrating the first decision-making process (step S6).

[0021] Figure 5This is a diagram illustrating the first decision-making process (step S6).

[0022] Figure 6 This is a diagram illustrating the first decision-making process (step S6).

[0023] Figure 7 This is a flowchart illustrating the control method involved in Embodiment 2.

[0024] Figure 8 This is a diagram illustrating the second decision-making process (step S6A).

[0025] Figure 9 This is a diagram illustrating the second decision-making process (step S6A).

[0026] Figure 10 This is a diagram illustrating the second decision-making process (step S6A).

[0027] Figure 11 This is a diagram showing the structure of the gripping part according to Embodiment 3.

[0028] Figure 12 This is a flowchart illustrating the control method involved in Embodiment 3.

[0029] Figure 13 This is a diagram illustrating the third decision-making process (step S6B).

[0030] Figure 14 This is a diagram illustrating the third decision-making process (step S6B).

[0031] Figure 15 This is a diagram illustrating the third decision-making process (step S6B). Detailed Implementation

[0032] Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below. Furthermore, in the accompanying drawings, the same reference numerals are used to denote the same parts.

[0033] (Implementation Method 1)

[0034] [Overview Structure of the Disposal System]

[0035] Figure 1 This is a diagram showing the processing system 1 according to Embodiment 1.

[0036] Treatment System 1 is a system that simultaneously observes an organism while treating the organism's tissues (hereinafter referred to as the target sites). For example... Figure 1 As shown, the treatment system 1 includes an endoscope 2, a display device 3, and a treatment device 4.

[0037] The structures of the endoscope device 2 and the treatment device 4 will be described in turn below.

[0038] [Structure of the endoscopic device]

[0039] Endoscopic device 2 is a device for observing the inside of a living organism. For example... Figure 1 As shown, the endoscope device 2 includes an endoscope body 21 and a control device 22.

[0040] The endoscope 21 corresponds to the imaging device involved in this invention. This endoscope 21 is inserted into a living organism to take images within that organism. In this embodiment 1, the endoscope 21 is constructed from a flexible, elongated endoscope that is inserted into the living organism. Furthermore, the endoscope 21 is detachably connected to the control device 22 via a connector (not shown). Figure 1 As shown, the mirror body 21 includes an illumination lens 211, an objective lens 212, an imaging unit 213, and an operation unit 214.

[0041] Illumination lens 211 and light guide 23 ( Figure 1 The light-emitting ends of the light guide 23 are positioned facing each other at the front end of the mirror body 21. Furthermore, the light emitted from the light guide 23 passes through the illumination lens 211 and then illuminates the organism.

[0042] Objective lens 212 is disposed at the front end of lens body 21. Furthermore, objective lens 212 captures light (subject image) that is incident on the organism from illumination lens 211 and reflected within the organism, and images it onto the light-receiving surface of imaging unit 213.

[0043] Under the control of the control device 22, the camera unit 213 generates a video image by capturing the image of the subject imaged by the objective lens 212. Furthermore, the camera unit 213 outputs the generated video image to the control device 22.

[0044] The operation unit 214 is equipped with various switches (not shown) that accept user operations performed by users such as doctors. Furthermore, the operation unit 214 outputs an operation signal corresponding to the operation to the control device 22.

[0045] The control device 22 is configured to include a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), etc., to comprehensively control the operation of the mirror 21 and the display device 3. For example... Figure 1 As shown, the control device 22 includes an analog processing unit 221, an A / D conversion unit 222, an image processing unit 223, an image output I / F unit 224, a first processor 225, a memory 226, and a light source device 227.

[0046] The analog processing unit 221 receives the camera image (analog signal) from the mirror body 21 and performs analog processing such as clamping processing and noise removal processing (CDS (correlated double sampling)) on the camera image.

[0047] The A / D conversion unit 222 performs A / D conversion on the camera image (analog signal) that has undergone analog processing, and outputs the converted camera image (digital signal).

[0048] Under the control of the first processor 225, the image processing unit 223 performs various image processing operations on the input camera image while using various image processing parameters stored in the memory 226. Examples of these various image processing operations include optical black subtraction, white balance (WB) adjustment, demosaicing, color matrix operation, gamma correction, color reproduction, and edge enhancement.

[0049] The image output I / F unit 224 consists of a DAC (Digital Analog Converter), an encoder, etc., and generates an image signal for display based on the captured image (digital signal) that has undergone various image processing by the image processing unit 223. Furthermore, the image output I / F unit 224 outputs the image signal for display to the display device 3.

[0050] The display device 3 is composed of a display using liquid crystal or organic EL (Electro Luminescence). Furthermore, the display device 3 receives display image signals from the image output I / F unit 224 and displays captured images based on the display image signals.

[0051] like Figure 1 As shown, the light source device 227 includes a light source 228 and a light source driver 229. Furthermore, in this embodiment 1, the light source device 227 is configured to be integrated into the control device 22, but it is not limited to this and may also be configured to be independent of the control device 22.

[0052] The light source 228 is composed of, for example, a white LED (Light Emitting Diode), which emits light according to the supplied power. Moreover, the light emitted from the light source 228 passes through the light guide 23 and the illumination lens 211 and then irradiates the living organism.

[0053] The light source driver 229 supplies power to the light source 228 under the control of the first processor 225.

[0054] The first processor 225 corresponds to the processor involved in this invention. This first processor 225, for example, is composed of a CPU, FPGA, etc., and controls the movement of the mirror 21, the display device 3, and the overall movement of the control device 22. Furthermore, the control device 22 and the treatment energy generator 43 constituting the treatment device 4 ( Figure 1 ) via the third cable C3 ( Figure 1 They can be easily connected to each other. Furthermore, the first processor 225 controls the operation of the energy generator 43 via the third cable C3. In addition, detailed functions of the first processor 225 will be explained later in the section on "Control Method Executed by the First Processor".

[0055] The memory 226 stores the program executed by the first processor 225, the information required for the processing by the first processor 225, and various parameters used for image processing.

[0056] [Structure of the treatment device]

[0057] The treatment device 4 treats the target area by applying treatment energy to it. Examples of this treatment include coagulation and cutting of the target area. Furthermore, examples of this treatment energy include at least one of ultrasonic energy, high-frequency energy, and thermal energy. Applying ultrasonic energy to the target area means applying ultrasonic vibration to the target area. Applying high-frequency energy to the target area means flowing a high-frequency current through the target area. Applying thermal energy to the target area means transferring heat generated by a heater or the like to the target area. Figure 1 As shown, the treatment device 4 includes a first treatment device 41, a second treatment device 42, and a treatment energy generator 43.

[0058] Furthermore, regarding the treatment device 4, it is possible to select, based on the treatment method, the type of the target body part, etc., to transmit the treatment via the first cable C1 ( Figure 1 The first treatment device 41 is used in a state where it is connected to the treatment energy generator 43, and in a state where it is used via the second cable C2 ( Figure 1 The second treatment device 42 is used in a state where it is connected to the treatment energy generator 43.

[0059] The first treatment device 41 corresponds to the treatment apparatus involved in this invention, and is a clamp-type treatment device that treats the object part while holding it. For example... Figure 1 As shown, the first processing device 41 includes a first holding housing 411, an operating handle 412, a first switch 413, a shaft 414, and a gripping part 415.

[0060] The first retaining housing 411 supports the entire first treatment device 41. For example... Figure 1As shown, the first retaining housing 411 has a central axis Ax1 located on the long handle 414. Figure 1 The retaining housing body 416 on the retaining housing body 416 and the direction from the retaining housing body 416 to Figure 1 A fixed handle 417 extends from the lower side of the middle and is held by the surgical operator.

[0061] The operating handle 412 is axially supported so that it can rotate relative to the first retaining housing 411 to receive opening and closing operations by the surgical operator.

[0062] The first switch 413 is set to be exposed to the outside from the first retaining housing 411, and is subject to being pressed by the surgical operator (hereinafter referred to as the first treatment start operation). Furthermore, the first switch 413 outputs an operation signal corresponding to the first treatment start operation to the treatment energy generator 43 via the first cable C1.

[0063] The long handle 414 has a cylindrical shape, with the base end side ( Figure 1 The right end of the long handle 414 is connected to the housing body 416. Additionally, the front end of the long handle 414 ( Figure 1 A gripping part 415 is installed at the left end of the handle 414. Furthermore, an opening and closing mechanism (not shown) is provided inside the handle 414, which opens and closes the first gripping member 418 and the second gripping member 419 constituting the gripping part 415 according to the operator's operation of the operating handle 412.

[0064] The gripping part 415 corresponds to the end effector according to the present invention. This gripping part 415 is the part that performs manipulation on the object part while gripping it. For example... Figure 1 As shown, the gripping part 415 includes a first gripping member 418 and a second gripping member 419.

[0065] The first gripping member 418 and the second gripping member 419 correspond to a pair of gripping members involved in this invention, and are positioned according to the operator's opening and closing operation of the operating handle 412, as indicated by arrow R1 ( Figure 1 The device can be opened and closed in the direction of the object, thereby enabling it to hold the object. Furthermore, at least one of the first holding member 418 and the second holding member 419 applies processing energy to the held object under the control of the processing energy generator 43. Thus, the object is processed.

[0066] The second treatment device 42 is equivalent to the treatment instrument involved in this invention. It is a non-clamping type treatment device that cannot hold the object part, but treats the object part while in contact with it. For example Figure 1 As shown, the second processing device 42 includes a second holding housing 421, a second switch 422, and an end effector 423.

[0067] The second retaining housing 421 has a generally cylindrical shape extending along the central axis Ax2 and supports the end effector 423.

[0068] The second switch 422 is set to protrude from the second retaining housing 421 to the outside, and is subject to being pressed by the surgical operator (hereinafter referred to as the second treatment start operation). Furthermore, the second switch 422 outputs an operation signal corresponding to the second treatment start operation to the treatment energy generator 43 via the second cable C2.

[0069] The end effector 423 has an elongated shape extending along the central axis Ax2, with the front end side ( Figure 1 The end of the actuator 423 (left side) is installed inside the second retaining housing 421 with its end exposed to the outside. Furthermore, with its front end in contact with the object part, the end effector 423 applies processing energy to the object part from that end under the control of the processing energy generator 43. Thus, the object part is processed.

[0070] The energy generator 43 is configured to include a CPU, FPGA, etc., and to comprehensively control the operation of the first processing device 41 connected via the first cable C1, or the operation of the second processing device 42 connected via the second cable C2. Figure 1 As shown, the processing energy generator 43 includes a processing energy output unit 431 and a second processor 432.

[0071] Under the control of the second processor 432, the processing energy output unit 431 supplies the power required to apply processing energy to the target area to the first processing device 41 connected via the first cable C1 or the second processing device 42 connected via the second cable C2.

[0072] For example, in the case where the first treatment device 41 is a structure that applies ultrasonic energy to a target area, the first treatment device 41 includes an ultrasonic transducer that generates ultrasonic vibrations based on the power supplied from the treatment energy output unit 431. Furthermore, one of the first holding member 418 and the second holding member 419 is composed of a vibration transmission member that transmits the ultrasonic vibrations, and the transmitted ultrasonic vibrations are applied to the target area held between the first holding member 418 and the second holding member 419.

[0073] Furthermore, when the second treatment device 42 is a structure that applies ultrasonic energy to a target area, the second treatment device 42 includes an ultrasonic transducer that generates ultrasonic vibrations based on the power supplied from the treatment energy output unit 431. Moreover, the end effector 423 is composed of a vibration transmission member that transmits the ultrasonic vibrations, and applies the transmitted ultrasonic vibrations to the target area.

[0074] Furthermore, for example, if the first processing device 41 is a structure that applies high-frequency energy to the target area, the first gripping member 418 and the second gripping member 419 each have electrodes that are powered by the processing energy output unit 431. Moreover, by supplying power to the pair of electrodes, a high-frequency current flows through the target area gripped between the pair of electrodes.

[0075] Furthermore, in the case where the second treatment device 42 is a structure that applies high-frequency energy to the target area, the end effector 423 has an electrode, and power is supplied from the treatment energy output unit 431 between the electrode and a counter plate mounted on the surface of the subject. Moreover, by supplying this power between the electrode and the counter plate, a high-frequency current flows through the target area located between the end effector 423 and the counter plate.

[0076] Furthermore, for example, in the case where the first processing device 41 is a structure that applies heat energy to the target area, at least one of the first gripping member 418 and the second gripping member 419 has a heater that generates heat based on the electricity supplied from the processing energy output unit 431. Moreover, at least one of the first gripping member 418 and the second gripping member 419 transfers the heat from the heater to the target area held between the first gripping member 418 and the second gripping member 419 based on the supply of electricity.

[0077] Furthermore, in the case where the second processing device 42 is a structure that applies heat energy to the target area, the end effector 423 includes a heater that generates heat based on the electricity supplied from the processing energy output unit 431. Moreover, the end effector 423 transfers the heat from the heater to the target area based on the supply of this electricity.

[0078] The second processor 432 is, for example, composed of a CPU, an FPGA, etc. Furthermore, the second processor 432 performs treatment control based on the surgical operator's first treatment start operation on the first switch 413 or a second treatment start operation on the second switch 422. This treatment control refers to controlling the treatment energy output unit 431 to supply power to the first treatment device 41 or the second treatment device 42 to apply treatment energy to the target area, thereby treating the target area.

[0079] [Control method executed by the first processor]

[0080] Next, the control method executed by the first processor 225 will be explained.

[0081] Figure 2 This is a flowchart illustrating the control method executed by the first processor 225.

[0082] Furthermore, for ease of explanation, the first treatment device 41 and the second treatment device 42 will be referred to together as treatment device 40 below.

[0083] In addition, the first processor 225 sequentially acquires, frame by frame, the images (digital signals) captured by the camera unit 213 inside the organism and processed by the analog processing unit 221, the A / D conversion unit 222 and the image processing unit 223 (step S1).

[0084] Furthermore, in this embodiment 1, in step S1, the first processor 225 acquires a photographic image taken by the camera unit 213 inside the organism and processed by the analog processing unit 221, the A / D conversion unit 222, and the image processing unit 223, but is not limited thereto. For example, in step S1, the first processor 225 may acquire a photographic image taken by the camera unit 213 inside the organism and processed by the analog processing unit 221 and the A / D conversion unit 222 (the photographic image before the image processing unit 223 performs various image processing).

[0085] After step S1, the first processor 225 performs the device identification process shown below (step S2).

[0086] Figure 3 This is a diagram illustrating the device identification process (step S2). Specifically, Figure 3 The image F1 acquired in step S1 is shown. Furthermore, in Figure 3 The example shown illustrates the use of the second processing device 42 of the first processing device 41 and the second processing device 42.

[0087] First, the first processor 225, such as Figure 3 As shown, region Ar1 in the camera image F1 is extracted. Region Ar1 is composed of pixels with pixel values ​​of a specific color (e.g., silver) inherent to the processing device 40.

[0088] Next, the first processor 225 determines whether the extracted region Ar1 extends in a straight line from the end of the camera image F1.

[0089] Then, if the first processor 225 determines that the region Ar1 extends in a straight line from the end of the captured image F1, it identifies the processing device 40 (in the image F1) as part of the image captured by the captured image F1. Figure 3 In the case of a second treatment device 42).

[0090] In addition, the first processor 225 performs the device identification process described above (step S2) sequentially on a frame-by-frame basis for the camera images acquired in step S1.

[0091] After step S2, the first processor 225 performs the category discrimination process shown below (step S3).

[0092] Here, in memory 226, as information required for processing by the first processor 225, a first threshold for a clamping type, a first threshold for a non-clamping type whose value is different from the first threshold for the clamping type, and the learning model shown below are stored.

[0093] The learning model is set up for each category of the processing device 40 (the category of the first processing device 41: clamp type, the category of the second processing device 42: non-clamp type), and is a model obtained by performing machine learning (e.g., deep learning) on ​​the features of the processing device 40 based on photographic images (teacher images) obtained by photographing the processing device 40. Moreover, the first processor 225 is able to determine the category of the processing device 40 in the captured image (the category of the first processing device 41: clamp type, the category of the second processing device 42: non-clamp type) by using image recognition (image recognition using AI (Artificial Intelligence)) of this learning model.

[0094] Then, the first processor 225 performs a category discrimination process (step S3), which is to use the image recognition of the learning model stored in the memory 226 to determine the category of the processing device 40 in the camera image acquired in step S1 (the category of the first processing device: clamp type, the category of the second processing device 42: non-clamp type).

[0095] After step S3, the first processor 225 reads from memory 226 a first threshold corresponding to the category of the processing device 40 determined in step S3 (step S4). That is, if the first processor 225 determines that the category of the processing device 40 is clamp type, it reads the first threshold for clamp type from memory 226; if the category of the processing device 40 is non-clamp type, it reads the first threshold for non-clamp type from memory 226.

[0096] After step S4, the first processor 225 continuously monitors whether a processing start operation (in this embodiment 1, a first processing start operation or a second processing start operation) exists (step S5). Furthermore, if a first processing start operation or a second processing start operation exists, the second processor 432 begins processing control. That is, it begins moving the device from the gripping part 415 or the end effector 423 towards the target part LT (…). Figure 3 The second processor 432 applies processing energy. Additionally, the second processor 432 outputs a signal to the first processor 225 via the third cable C3, indicating that the processing operation has begun. Based on this signal, the first processor 225 determines that the processing operation has begun.

[0097] If it is determined that a processing start operation exists (step S5: "Yes"), the first processor 225 executes the first determination process shown below (step S6).

[0098] Figures 4 to 6 This is a diagram illustrating the first decision-making process (step S6). Specifically, Figure 4 and Figure 5 The camera images F2 and F3 acquired in step S1 are shown. Furthermore, in Figure 4 and Figure 5 In, with Figure 3 Similarly, the case of using the second processing device 42 of the first processing device 41 and the second processing device 42 is illustrated. Furthermore, Figure 4 The image F2 shown is a picture obtained when the treatment energy is first applied to the target area LT. Figure 5 The image F3 shown is an image obtained after the end effector 423 broke off after the treatment energy was applied to the target part LT. Figure 6 This is a diagram showing the consistency of the external shape of the end effector 423.

[0099] First, the first processor 225 extracts a frame Ar2 representing the outline of the shape of the end effector 423 (or gripping part 415 in the case of using the first processing device 41) based on the camera image F2 at the start of applying processing energy to the target part LT. Figure 4 ).

[0100] Specifically, the first processor 225 determines the end effector 423 located within the region Ar1 (processing device 40) extracted in step S2 based on pixel values ​​and shape features, thereby extracting the aforementioned border Ar2. Then, the first processor 225 stores the extracted border Ar2 into the memory 226.

[0101] Next, the first processor 225 calculates the consistency rate of the shape of the end effector 423 (the gripping part 415 in the case of using the first processing device 41) in the current frame (hereinafter referred to as the current frame) image (e.g., image F3) with respect to a predetermined shape (the border Ar2 stored in the memory 226).

[0102] Specifically, in the current frame of the captured image, the first processor 225 determines the second holding housing 421 (or long handle 414 in the case of using the first processing device 41) located within the region Ar1 (processing device 40) extracted in step S2 based on pixel values ​​and shape features. Furthermore, in the current frame of the captured image, while the processing device 40 maintains a straight line, the first processor 225 allocates the border Ar2 stored in memory 226 to the front end portion of the determined second holding housing 421. Then, in the current frame of the captured image, the first processor 225 calculates the ratio of the number of pixels with pixel values ​​inherent to the end effector 423 (or gripping part 415 in the case of using the first processing device 41) within the border Ar2 to the area (number of pixels) of the border Ar2, using this ratio as the shape consistency rate. Moreover, the first processor 225 sequentially performs this shape consistency rate calculation frame by frame for the captured images acquired sequentially in step S1. This shape consistency rate corresponds to the determination value involved in this invention.

[0103] Next, the first processor 225 compares the calculated shape consistency rate with the first threshold Th1 read in step S4. Figure 6 The comparisons are performed sequentially. Then, if the consistency rate of the external shape is below the first threshold Th1, the first processor 225 determines that the end effector 423 (or gripping part 415 in the case of using the first processing device 41) has malfunctioned (malfunction detected).

[0104] Furthermore, in the case where the end effector 423 (or gripping part 415 in the case of using the first processing device 41) detaches after bending or breaking, the consistency rate of the aforementioned external shape is low and falls below the first threshold Th1. That is, in this embodiment 1, detachment after bending or breaking is determined to be an abnormality of the end effector 423 (or gripping part 415 in the case of using the first processing device 41).

[0105] Then, if an anomaly is detected (step S7: "Yes"), the first processor 225 instructs the processing energy generator 43 (second processor 432) to stop output via the third cable C3 (step S8). As a result, the second processor 432 stops supplying power to the processing device 40 (stops the operation of the processing energy output unit 431).

[0106] According to the above-described Embodiment 1, the following effects are achieved.

[0107] The control device 22 of this embodiment determines whether an abnormality has occurred in the gripping unit 415 or the end effector 423 based on the captured image (e.g., image F1 to F3) obtained when the handling energy is applied from the gripping unit 415 or the end effector 423 to the target part LT. If the abnormality is determined to have occurred, the power supply to the handling device 40 is stopped.

[0108] That is, by performing anomaly determination based on camera images, it is possible to detect various anomalies in the gripping part 415 or the end effector 423 (in this embodiment 1, these are bending, breakage, and detachment), thereby improving convenience.

[0109] Furthermore, the control device 22 described in Embodiment 1 uses the aforementioned consistency rate of external shape in the anomaly determination (first determination process (step S6)), thus enabling it to detect with high precision any bending, breakage, or detachment in the gripping part 415 or the end effector 423.

[0110] Furthermore, since the control device 22 of this embodiment performs the above-described category discrimination process (step S3), it is possible to use an appropriate first threshold corresponding to the category of the disposal device 40 to detect abnormalities of the gripping part 415 or the end effector 423 with high precision.

[0111] (Implementation Method 2)

[0112] Next, this embodiment 2 will be described.

[0113] In the following description, the same reference numerals are used to refer to the same structures as in Embodiment 1 described above, and detailed descriptions are omitted or simplified.

[0114] Figure 7 This is a flowchart illustrating the control method involved in Embodiment 2.

[0115] In this embodiment 2, as Figure 7 As shown, the control method executed by the first processor 225 is different from that of Embodiment 1 described above.

[0116] In the control method involved in this embodiment 2, such as Figure 7 As shown, compared to the control method described in Embodiment 1 above ( Figure 2 Steps S4A, S6A, and S7A are used instead of steps S4, S6, and S7. The following mainly explains steps S4A, S6A, and S7A.

[0117] Step S4A is performed after step S3.

[0118] In this embodiment 2, a second threshold is stored in memory 226. Additionally, memory 226 stores a third threshold for a clamp-type clamp and a third threshold for a non-clamp-type clamp with a value different from the third threshold for the clamp-type clamp, replacing the first threshold for the clamp-type clamp and the first threshold for the non-clamp-type clamp.

[0119] Then, the first processor 225 reads from the memory 226 the third threshold corresponding to the category of the processing device 40 determined in step S3 (step S4A). That is, if the first processor 225 determines that the category of the processing device 40 is clamp type, it reads the third threshold for clamp type from the memory 226; if the category of the processing device 40 is non-clamp type, it reads the third threshold for non-clamp type from the memory 226.

[0120] Then, the first processor 225 proceeds to step S5.

[0121] Step S6A is executed when it is determined that a processing start operation exists (step S5: "Yes").

[0122] Specifically, the first processor 225 performs the second determination process shown below in step S6A.

[0123] Figures 8 to 10 This is a diagram illustrating the second decision-making process (step S6A). Specifically, Figure 8 and Figure 9 Images F4 and F5, acquired in step S1, are shown. Furthermore, in Figure 8 and Figure 9 The example shown illustrates the use of the first processing device 41 in the first processing device 41 and the second processing device 42. Additionally, Figure 8 The image F4 shown is a picture taken at the moment when treatment energy is first applied to the target area LT. Figure 9 The image F5 shown is an image obtained after the application of treatment energy to the target area LT, showing the state in which the first gripping member 418 is charred. Furthermore, in Figure 9 In the image, black dots represent the charred portion. Figure 10 This is a diagram showing the proportion of charred material.

[0124] First, the first processor 225 determines the gripping part 415 (or end effector 423 in the case of using the second processing device 42) located within the region Ar1 (processing device 40) extracted in step S2 based on pixel values ​​and shape features. Then, the first processor 225 sets a region Ar3 within the determined gripping part 415. Figure 8 , Figure 9In this second embodiment, the first processor 225 sets the first gripping member 418 as a region Ar3. This region Ar3 corresponds to the first region and the defined region involved in this invention. Furthermore, the first processor 225 sequentially sets this region Ar3 frame by frame for the camera images acquired sequentially in step S1.

[0125] Next, the first processor 225 calculates the initial area (number of pixels) of region Ar3 based on the image F4 captured when treatment energy is first applied to the target site LT. Then, the first processor 225 stores the initial area (number of pixels) in memory 226.

[0126] Next, the first processor 225 extracts pixels (hereinafter referred to as burnt pixels) within region Ar3 that have pixel values ​​below the second threshold stored in memory 226. Figure 9 The image is represented by a black dot, and the number of pixels of the burned pixel is calculated. Furthermore, the first processor 225 performs the calculation of the number of pixels of the burned pixel sequentially, frame by frame, for the camera images acquired sequentially in step S1.

[0127] Next, the first processor 225 calculates the ratio (burnt ratio) of the calculated number of burnt pixels to the initial area (number of pixels) stored in the memory 226. Furthermore, the first processor 225 performs this burnt ratio calculation frame by frame for each of the captured images acquired sequentially in step S1. This burnt ratio corresponds to the determination value involved in this invention.

[0128] Next, the first processor 225 compares the calculated burn ratio with the third threshold Th3 read in step S4A. Figure 10 The comparisons are performed sequentially. Then, if the scorch ratio is above the third threshold Th3, the first processor 225 determines that the gripping part 415 (or end effector 423 when using the second processing device 42) has malfunctioned (malfunction detected). That is, in this embodiment 2, excessive scorch of the gripping part 415 (or end effector 423 when using the second processing device 42) is determined to be "malfunction".

[0129] Then, if an anomaly is detected (step S7A: "Yes"), the first processor 225 proceeds to step S8.

[0130] According to the above-described Embodiment 2, in addition to the effects of Embodiment 1, the following effects are also achieved.

[0131] The control device 22 according to this embodiment 2 uses the above-mentioned scorching ratio in the abnormality determination (second determination process (step S6A)), so it can detect excessive scorching in the gripping part 415 or the end effector 423 with high accuracy.

[0132] (Implementation Method 3)

[0133] Next, this embodiment 3 will be described.

[0134] In the following description, the same reference numerals are used to refer to the same structures as in Embodiment 1 described above, and detailed descriptions are omitted or simplified.

[0135] Figure 11 This diagram illustrates the structure of the gripping part 415 according to Embodiment 3. Specifically, Figure 11 Used with central axis Ax1 ( Figure 1 A cross-sectional view obtained by cutting the holding part 415 with an orthogonal plane. Figure 12 This is a flowchart illustrating the control method involved in Embodiment 3.

[0136] The first processor 225 detection involved in this embodiment 3 Figure 11 The gripping part 415 in the clamp-type first processing device 41 shown may malfunction. That is, in this embodiment 3, the non-clamp-type second processing device 42 is not considered for malfunction. Moreover, at the same time, in this embodiment 3, as... Figure 12 As shown, the control method executed by the first processor 225 is also different from the above-described embodiment 1.

[0137] The first processing device 41 according to this embodiment 3 is a structure that applies ultrasonic energy from the first holding member 418 to the target area LT based on the power supplied from the processing energy output unit 431. That is, the first processing device 41 has an ultrasonic transducer that generates ultrasonic vibrations based on the power supplied from the processing energy output unit 431. Furthermore, the first holding member 418 is composed of a vibration transmission member that transmits the ultrasonic vibrations. Additionally, as... Figure 11 As shown, the second gripping member 419 has a jaw 419a and a resin pad 419b.

[0138] The jaws 419a open and close relative to the first gripping member 418 according to the surgeon's operation of opening and closing the operating handle 412.

[0139] The resin pad 419b is made of white resin and is mounted on the side of the jaws 419a facing the first gripping member 418. The resin pad 419b has the following function: even when the cutting of the target part LT is completed and the second gripping member 419 is in contact with the first gripping member 418, it prevents the first gripping member 418, which is undergoing ultrasonic vibration, from being damaged by impacting the jaws 419a.

[0140] In the control method described in Embodiment 3, such as Figure 12 As shown, compared to the control method described in Embodiment 1 above ( Figure 2 Steps S3 and S4 are omitted, and steps S6B and S7B are used instead of steps S6 and S7. Steps S6B and S7B will be explained in detail below. Furthermore, in this embodiment 3, the first processor 225 proceeds to step S5 after step S2.

[0141] In this embodiment 3, a fourth threshold is stored in memory 226. Additionally, a fifth threshold is stored in memory 226 to replace both the first threshold for the clamp type and the first threshold for the non-clamp type.

[0142] Step S6B is executed when it is determined that a processing start operation exists (step S5: "Yes").

[0143] Specifically, the first processor 225 performs the third determination process shown below in step S6B.

[0144] Figures 13 to 15 This is a diagram illustrating the third decision-making process (step S6B). Specifically, Figure 13 and Figure 14 Images F6 and F7, acquired in step S1, are shown. Additionally, Figure 13 The image shown, F6, is a picture taken at the moment when treatment energy is first applied to the target area LT. Figure 14 The image F7 shown is an image obtained after the treatment energy is applied to the target area LT, in the state where the resin pad 419b is exposed. Figure 15 This is a diagram showing the area of ​​the pad region.

[0145] First, the first processor 225 determines the region Ar4 (which contains the region Ar1 (first processing device 41) extracted in step S2). Figure 13 , Figure 14Furthermore, the first processor 225 sequentially determines the region Ar4 frame by frame for each of the camera images acquired sequentially in step S1. Each region Ar4 determined in each camera image (e.g., camera images F6, F7) is located in the same positional relationship as each region Ar1 (first processing device 41) extracted from that camera image. This region Ar4 corresponds to the first region involved in this invention.

[0146] Next, the first processor 225 extracts the pad region Ar5 (which is composed of pixels having pixel values ​​that are above the fourth threshold stored in memory 226) within region Ar4. Figure 14 The first processor 225 calculates the area of ​​the pad region Ar5 on a frame-by-frame basis for the camera images acquired sequentially in step S1.

[0147] Next, the first processor 225 compares the calculated area of ​​the pad region Ar5 with the fifth threshold Th5 stored in the memory 226. Figure 15 The comparisons are performed sequentially. Then, if the area of ​​the pad region Ar5 is greater than or equal to the fifth threshold Th5, the first processor 225 determines that an abnormality has occurred in the gripping part (an abnormality is detected). That is, in this embodiment 3, excessive exposure of the resin pad 419b is determined to be "abnormal".

[0148] Then, if an anomaly is detected (step S7B: "Yes"), the first processor 225 proceeds to step S8.

[0149] According to the above description, this embodiment 3, in addition to having the same effects as the above-described embodiment 1, also has the following effects.

[0150] The control device 22 described in this embodiment 3 uses the area of ​​the pad region Ar5 described above in the anomaly determination (third determination process (step S6B)), thus enabling it to detect excessive exposure of the resin pad 419b with high precision.

[0151] (Other implementation methods)

[0152] The methods for implementing the present invention have been described above, but the present invention should not be limited to the embodiments 1 to 3 described above.

[0153] In the above-described embodiments 1 and 2, a common treatment energy generator 43 is used for the first treatment device 41 and the second treatment device 42, but it is not limited to this. Treatment energy generators may also be provided for the first treatment device 41 and the second treatment device 42 respectively.

[0154] In embodiments 1 to 3 described above, the endoscope 21 is made of a flexible endoscope, but is not limited to this. A structure combining a rigid endoscope and a camera can also be used instead of the endoscope 21.

[0155] In the embodiments 1 to 3 described above, as the processing device according to the present invention, a robot processing device may also be adopted, which has multiple arms, joints that connect the multiple arms in a way that allows them to move relative to each other, and a drive mechanism that operates the joints to drive the arms (for example, see Japanese Patent No. 4960112).

[0156] In the above-described embodiments 1 to 3, the device identification process (step S2) can also be performed by using image recognition with a learning model (image recognition using AI), similar to the category discrimination process (step S3).

[0157] In embodiments 1 to 3 described above, the second processor 432 may also be executed. Figure 2 , Figure 7 as well as Figure 12 The control method shown. That is, the second processor 432, instead of the first processor 225, can also function as the processor involved in this invention.

[0158] In the above-described embodiments 1 to 3, multiple processors may also be used to execute. Figure 2 , Figure 7 as well as Figure 12 The control method shown is as follows. That is, the processor involved in this invention is not limited to one, but can also be composed of multiple processors.

[0159] The first determination process to the third determination process (steps S6, S6A, S6B) described in embodiments 1 to 3 above can also be appropriately combined.

[0160] In embodiments 1 to 3 described above, if an anomaly is detected (steps S7, S7A, S7B: "Yes"), the display device 3 may display information indicating that the anomaly has been detected. Alternatively, in addition to displaying the information on the display device 3, the information may be output from a speaker, or it may be configured to output the information by lighting or flashing an LED or similar device provided in the control device 22.

[0161] In the above embodiments 1 to 3, it is also possible that when an abnormality is detected (steps S7, S7A, S7B: "Yes"), the memory (not shown) provided in the processing device 40 stores information indicating the type of the abnormality (bending of the gripping part 415, bending of the end effector 423, breakage and detachment, excessive burning, excessive exposure of the resin pad 419b).

[0162] In this configuration, the remanufacturing of the treatment equipment 40 has the following effects. Furthermore, the remanufacturing of the treatment equipment 40 refers to the process of disassembling, cleaning, replacing parts, reassembling, sterilizing, etc., of the treatment equipment 40 after use at the treatment target area LT, and confirming that it has the required performance, so as to make the treatment equipment 40 usable again.

[0163] That is, the remanufacturer can identify the abnormal part by checking the information stored in the memory of the processing device 40, and determine whether to replace the part or keep it as is.

[0164] Explanation of reference numerals in the attached figures

[0165] 1: Treatment system; 2: Endoscopic device; 3: Display device; 4: Treatment device; 21: Endoscope body; 22: Control device; 23: Light guide; 40: Treatment equipment; 41: First treatment equipment; 42: Second treatment equipment; 43: Treatment energy generator; 211: Illumination lens; 212: Objective lens; 213: Camera unit; 214: Operation unit; 221: Analog processing unit; 222: A / D conversion unit; 223: Image processing unit; 224: Image output I / F unit; 225: First processor; 226: Memory; 227: Light source device; 228: Light source; 229: Light source driver; 411: First holding housing; 412: Operating handle; 413: First switch; 414 415: Long handle; 416: Holding part; 417: Main body of retaining housing; 418: Fixed handle; 419: First holding member; 419: Second holding member; 419a: Jaw; 419b: Resin pad; 421: Second retaining housing; 422: Second switch; 423: End effector; 431: Disposal energy output part; 432: Second processor; Ax1, Ax2: Central shaft; C1: First cable; C2: Second cable; C3: Third cable; F1~F7: Camera image; Ar1, Ar3, Ar4: Area; Ar2: Border; Ar5: Pad area; LT: Object part; R1: Arrow; Th1: First threshold; Th3: Third threshold; Th5: Fifth threshold.

Claims

1. A processing system, comprising: A treatment device that applies treatment energy from an end effector to biological tissue based on supplied power, thereby treating the biological tissue; A camera device that generates a camera image of the state in which the treatment energy is applied from the end effector to the biological tissue; and The control device includes a processor for controlling the operation of the treatment apparatus. in, The processor performs the following processing: Acquire the camera image; While the treatment energy is being applied from the end effector to the biological tissue, the system determines whether the end effector has malfunctioned based on the image captured. If the end effector is determined to be malfunctioning, the power supply to the treatment device shall be stopped.

2. The processing system according to claim 1, wherein, The processor performs the following processing: Based on the camera image, detect at least one of the changes in the shape of the end effector and the changes in pixel values ​​within a first region including the end effector; The end effector is determined to be abnormal based on at least one of the detected changes in the shape and the changes in the pixel value.

3. The processing system according to claim 2, wherein, It also has a memory for storing the first threshold. The processor performs the following processing: Based on the camera image, calculate the consistency rate of the end effector's shape with a predetermined shape; If the consistency rate is below the first threshold, it is determined that the end effector has malfunctioned.

4. The treatment system according to claim 2, wherein, It also has a memory for storing the second and third thresholds. The first region is a defined region within the end effector. The processor performs the following processing: Based on the camera image, the area of ​​the specified region and the area of ​​the abnormal region are calculated respectively. The abnormal region is composed of pixels within the specified region whose pixel values ​​are below the second threshold. If the ratio of the area of ​​the abnormal region to the area of ​​the specified region is greater than or equal to the third threshold, the end effector is determined to have malfunctioned.

5. The treatment system according to claim 2, wherein, It also has a memory for storing the fourth and fifth thresholds. The end effector includes a pair of gripping members for gripping the biological tissue. One of the pair of holding members applies ultrasonic energy as the treatment energy to the biological tissue. A resin pad is provided on the other of the pair of gripping members at a position facing the first gripping member. The processor performs the following processing: Based on the camera image, calculate the area of ​​the pad region composed of pixels in the first region whose pixel value is above the fourth threshold; If the area of ​​the pad region is greater than or equal to the fifth threshold, it is determined that the end effector has malfunctioned.

6. The disposal system according to claim 1, wherein, It also has a memory that can store multiple thresholds. The processor performs the following processing: Based on the camera images, the category of the treatment device is determined; Whether the end effector has malfunctioned is determined by comparing a threshold corresponding to the category of the treatment device among the plurality of thresholds with a determination value used to determine the malfunction of the end effector.

7. A control device comprising a processor for controlling the operation of a treatment apparatus, the treatment apparatus applying treatment energy from an end effector to biological tissue based on supplied power, thereby treating the biological tissue. The processor performs the following processing: Acquire a photographic image of the state in which the treatment energy is applied from the end effector to the biological tissue; While the treatment energy is being applied from the end effector to the biological tissue, the system determines whether the end effector has malfunctioned based on the image captured. If the end effector is determined to be malfunctioning, the power supply to the treatment device shall be stopped.

Images