Endoscopy assisting apparatus, program product, and recording medium

Abstract

The endoscopic examination assistance device includes: an examination status information acquisition unit, which acquires examination status information representing the examination status of an endoscopic examination of a subject using an endoscope, based on an endoscopic image obtained by the endoscope taken inside the subject and insertion shape information representing the insertion shape of the insertion part of the endoscope inserted into the subject; an examination assistance information generation unit, which generates operation assistance information to assist the insertion operation of the insertion part performed by a user operating the endoscope during the endoscopic examination, based on the examination status information acquired by the examination status information acquisition unit; and a surgical operation evaluation unit, which evaluates the surgical operation, including the insertion operation of the insertion part performed by the user during the endoscopic examination, based on the examination status information acquired by the examination status information acquisition unit, thereby generating surgical operation evaluation information.

Description

Technical Field

[0001] This invention relates to an endoscopic examination aid device, an operating method of the endoscopic examination aid device, and a non-transitory recording medium containing a program. Background Technology

[0002] In the field of endoscopy, techniques have been proposed to assist various operations performed by the user, such as insertion operations that involve inserting a flexible, elongated insert into the deep part of the body being examined.

[0003] Specifically, for example, International Publication No. 2016 / 135966 discloses an apparatus that detects the shape of the insertion portion of an endoscope and generates operational assistance information indicating the state of the insertion portion based on the shape of the insertion portion.

[0004] However, the structure disclosed in International Publication No. 2016 / 135966 presents a problem, for example, that the information generated by the user during various operations in endoscopy, such as assisting in the insertion of the insert, is limited to information that can be estimated based on the shape of the insert. Therefore, the structure disclosed in International Publication No. 2016 / 135966, for example, when the insert is inserted deep into the body of the patient, presents a problem corresponding to the aforementioned issue: it may be impossible to provide the user with information to facilitate appropriate operations corresponding to the situation during the endoscopy.

[0005] Furthermore, based on the structure disclosed in International Publication No. 2016 / 135966, for example, the following problem arises: it is impossible to provide the user with an evaluation of surgical procedures performed during the endoscopy, such as insertion of the insertion site, after the endoscopy is completed.

[0006] The present invention was made in view of the above circumstances, and its object is to provide an endoscopic examination auxiliary device that can improve the examination quality in endoscopic examinations compared to the past, a method of operating the endoscopic examination auxiliary device, and a non-transitory recording medium on which the program is recorded. Summary of the Invention

[0007] means for solving problems

[0008] An endoscopic examination assist device according to one aspect of the present invention includes: an examination status information acquisition unit that acquires insertion shape information representing the insertion shape of an insertion portion of an endoscope inserted into a subject; and a surgical operation evaluation unit that, based on the insertion shape information obtained by the examination status information acquisition unit, evaluates surgical operations, including insertion operations of the insertion portion performed by a user operating the endoscope during an endoscopic examination, thereby generating surgical operation evaluation information.

[0009] In one aspect of the endoscopic examination auxiliary device of the present invention, an examination status information acquisition unit acquires insertion shape information indicating the insertion shape of the insertion part of the endoscope inserted into the subject body, and a surgical operation evaluation unit evaluates the surgical operation, including the insertion operation of the insertion part performed by the user operating the endoscope during the endoscopic examination, based on the insertion shape information obtained by the examination status information acquisition unit, thereby generating surgical operation evaluation information.

[0010] In one aspect of the present invention, a non-transitory recording medium containing a program causes a computer to perform the following processing: obtaining insertion shape information representing the insertion shape of an insertion portion of an endoscope inserted into a patient; and, based on the insertion shape information, evaluating surgical procedures, including the insertion operation of the insertion portion performed by a user operating the endoscope during an endoscopic examination, thereby generating surgical procedure evaluation information. Attached Figure Description

[0011] Figure 1 This is a diagram showing the structure of the main parts of an endoscope system including an endoscopic examination aid device according to an embodiment.

[0012] Figure 2 This is a block diagram illustrating an example of the structure of the endoscope system according to the first embodiment.

[0013] Figure 3 This is a block diagram illustrating an example of the structure of the inspection status information acquisition unit in the first embodiment.

[0014] Figure 4 This is a diagram illustrating an example of table data used in the processing performed in the endoscope system of the first embodiment.

[0015] Figure 5 This is a block diagram illustrating an example of the structure of the surgical operation evaluation unit in the first embodiment.

[0016] Figure 6 This is a flowchart illustrating an example of the processing performed in the endoscope system of the first embodiment.

[0017] Figure 7 This is a diagram illustrating an example of a display image generated by the processing of the endoscope system according to the first embodiment.

[0018] Figure 8 This is a flowchart illustrating an example of the processing performed in the endoscope system of the first embodiment.

[0019] Figure 9 It is a schematic representation of the basis Figure 8The figure shows an example of the state of the intestine when the insertion part has been inserted.

[0020] Figure 10 It is a schematic representation of the basis Figure 8 The figure shows an example of the state of the intestine when the insertion part has been inserted.

[0021] Figure 11 It is a schematic representation of the basis Figure 8 The figure shows an example of the state of the intestine when the insertion part has been inserted.

[0022] Figure 12 This is a diagram illustrating an example of a display image generated by processing through the endoscope system of the first embodiment.

[0023] Figure 13 This is a diagram illustrating an example of table data used in the processing performed in the endoscope system of the first embodiment.

[0024] Figure 14 This is for explaining the processes performed in the endoscope system of the first embodiment. Figure 6 Flowcharts for different examples.

[0025] Figure 15 This is a block diagram illustrating an example of the structure of the endoscope system according to the second embodiment.

[0026] Figure 16 This is a block diagram illustrating an example of the structure of the inspection status information acquisition unit in the second embodiment.

[0027] Figure 17 This is a block diagram illustrating an example of the structure of the surgical operation evaluation unit in the second embodiment.

[0028] Figure 18 This is a flowchart illustrating an example of the processing performed in the endoscope system of the second embodiment.

[0029] Figure 19 This is a block diagram illustrating an example of the structure of the endoscope system according to the third embodiment.

[0030] Figure 20 This is a block diagram illustrating an example of the structure of the inspection status information acquisition unit in the third embodiment.

[0031] Figure 21 This is a block diagram illustrating an example of the structure of the surgical operation evaluation unit in the third embodiment.

[0032] Figure 22 This is a flowchart illustrating an example of the processing performed in the endoscope system of the third embodiment. Detailed Implementation

[0033] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0034] (First Implementation)

[0035] Figures 1 to 14 This is a diagram of the first embodiment of the present invention.

[0036] For example, such as Figure 1 As shown, the endoscope system 1 is configured to include: an endoscope 10, a main body device 20, an insertion shape observation device 40, an input device 50, and a display device 60. Figure 1 This is a diagram showing the structure of the main parts of an endoscope system including an endoscopic examination aid device according to an embodiment.

[0037] The endoscope 10 is configured to include: an insertion portion 11 that is inserted into the subject, an operating portion 16 located at the base of the insertion portion 11, and a universal flexible cable 17 extending from the operating portion 16. Furthermore, the endoscope 10 is configured to be detachably connected to the main body device 20 via a scope connector (not shown) located at the end of the universal flexible cable 17. Additionally, light guides (not shown) for transmitting illumination light supplied from the main body device 20 are provided inside the insertion portion 11, the operating portion 16, and the universal flexible cable 17.

[0038] The insertion portion 11 is configured to have a flexible and elongated shape. Furthermore, the insertion portion 11 is configured such that, from the front end side, a rigid front end portion 12, a freely bendable curved portion 13, and a flexible, elongated tube portion 14 are sequentially arranged. Additionally, within the front end portion 12, the curved portion 13, and the flexible tube portion 14, multiple source coils 18 are arranged at predetermined intervals along the length direction of the insertion portion 11 to generate a magnetic field corresponding to a coil drive signal supplied from the main body device 20. Furthermore, a treatment instrument channel (not shown) is provided inside the insertion portion 11. This treatment instrument channel has a shape that allows an elongated treatment instrument to be inserted through it, and is formed in such a way that it connects the treatment instrument insertion port (not shown) provided in the operation portion 16 and the treatment instrument protrusion (not shown) provided in the front end portion 12. That is, the endoscope 10 is configured such that an elongated treatment instrument inserted from the treatment instrument insertion port can be inserted through into the interior of the treatment instrument channel, and the front end of the treatment instrument can protrude from the treatment instrument protrusion.

[0039] An illumination window (not shown) is provided at the front end 12, which is used to project illumination light transmitted by a light guide provided inside the insertion part 11 onto the subject. Additionally, a camera part 110 is provided at the front end 12. Figure 1(Not shown in the figure), the camera unit is configured to perform operations corresponding to the camera control signal supplied from the main unit 20, and to capture images of a subject illuminated by illumination light emitted through the illumination window and output an image signal. The camera unit 110 is configured, for example, to have: an observation window for receiving reflected light from the subject illuminated by the illumination light, and an image sensor such as a color CCD for capturing images of the reflected light and outputting an image signal.

[0040] The bending portion 13 is configured to be able to bend according to the operation of the angle knob (not shown) provided on the operation portion 12.

[0041] The operating unit 16 is configured in a shape that allows users, such as surgeons, to hold and operate it. Furthermore, an angle knob is provided on the operating unit 16, which is configured to bend the bending section 13 in four directions (up, down, left, and right) intersecting the length axis of the insertion section 11. Additionally, the operating unit 16 is provided with one or more endoscope switches (not shown) that can provide indications corresponding to user input operations.

[0042] The main unit 20 is configured to have one or more processors 20P and a non-transitory storage medium 20M. Furthermore, the main unit 20 is configured to function as an endoscopic examination auxiliary device. The main unit 20 is configured to be detachably connected to the endoscope 10 via a universal flexible cable 17. The main unit 20 is configured to be detachably connected to each of the insertion shape observation device 40, the input device 50, and the display device 60. The main unit 20 is configured to perform actions corresponding to instructions from the input device 50. The main unit 20 is configured to generate an endoscopic image based on the camera signal output from the endoscope 10 and to display the generated endoscopic image on the display device 60. The main unit 20 is also configured to generate operation assistance information, including guidance information, to assist the user in inserting the insertion unit 11, and to display the generated operation assistance information on the display device 60. In addition, the main device 20 is configured to generate surgical operation evaluation information containing information obtained by evaluating surgical operations, and to perform an action for displaying the generated surgical operation evaluation information on the display device 60, wherein the surgical operation includes the insertion of the insertion part 11 performed by the user during an endoscopic examination.

[0043] The insertion shape observation device 40 is configured to detect the magnetic fields emitted from the source coils 18 disposed on the insertion part 11, and to obtain the positions of the plurality of source coils 18 based on the strength of the detected magnetic fields. Furthermore, the insertion shape observation device 40 is configured to generate insertion position information representing the positions of the plurality of source coils 18 obtained as described above, and output this information to the main body device 20. In other words, the insertion shape observation device 40 is configured to detect the insertion position of the insertion part inserted into the test body to obtain insertion position information, and to output this obtained insertion position information to the main body device 20.

[0044] The input device 50 is configured to have one or more user-operated input interfaces, such as a mouse, keyboard, and touch panel. Furthermore, the input device 50 is configured to output instructions corresponding to the user's operation to the main device 20.

[0045] The display device 60 is configured to include, for example, a liquid crystal monitor. Furthermore, the display device 60 is configured to display endoscopic images output from the main unit 20 on a screen.

[0046] like Figure 2 As shown, the main device 20 is configured to include: a light source unit 210, an image processing unit 220, a coil drive signal generation unit 230, an insertion shape image generation unit 240, a display control unit 250, a sound generation unit 260, and a system control unit 270. Figure 2 This is a block diagram illustrating an example of the structure of the endoscope system according to the first embodiment.

[0047] The light source unit 210 is configured to have one or more LEDs or lamps as the light source. Furthermore, the light source unit 210 is configured to generate illumination light for illuminating the specimen inserted into the insertion unit 11, and is capable of supplying this illumination light to the endoscope 10. Additionally, the light source device 210 is configured to change the amount of illumination light according to a system control signal supplied from the system control unit 270.

[0048] The image processing unit 220 is configured, for example, to have an image processing circuit. In addition, the image processing unit 220 generates an endoscope image by performing a prescribed processing on the camera signal output from the endoscope 10, and outputs the generated endoscope image sequentially to the display control unit 250 and the system control unit 270 frame by frame.

[0049] The coil drive signal generation unit 230 is configured, for example, to have a drive circuit. Furthermore, the coil drive signal generation unit 230 is configured to generate and output a coil drive signal for driving the source coil 18 based on a system control signal supplied from the system control unit 270.

[0050] The insertion shape image generation unit 240 is configured to generate an insertion shape image by visualizing the insertion shape of the insertion part 11 inserted into the subject body in two dimensions based on the insertion position information output from the insertion shape observation device 40. The insertion shape image generation unit 240 is configured to output the insertion shape image generated as described above to the display control unit 250 and the system control unit 270.

[0051] The display control unit 250 performs processing to generate a display image including the endoscopic image output from the image processing unit 220, and performs processing to display the generated display image on the display device 60. Furthermore, the display control unit 250 is configured to perform processing to display an insertion shape image output from the insertion shape image generation unit 240 on the display device 60. Additionally, the display control unit 250 is configured to perform processing to generate visual information such as strings and symbols corresponding to operation assistance information output from the system control unit 270, and performs processing to display a display image including the generated visual information on the display device 60. Furthermore, the display control unit 250 is configured to perform processing to generate visual information such as strings and symbols corresponding to surgical operation evaluation information (described later) output from the system control unit 270, and performs processing to display a display image including the generated visual information on the display device 60.

[0052] The sound generating unit 260 is configured to include, for example, a speaker. Furthermore, the sound generating unit 260 is configured to generate a sound corresponding to the guidance information included in the operation assistance information output from the system control unit 270, and to perform an operation to output the generated sound to the outside of the main unit 20.

[0053] The system control unit 270 is configured to generate and output system control signals for performing actions corresponding to instructions from the operation unit 16 and the input device 50. Furthermore, the system control unit 270 is configured to generate operation assistance information based on at least one of the following: an endoscopic image output from the image processing unit 220, an insertion shape image output from the insertion shape image generation unit 240, and insertion position information output from the insertion shape observation device 40. This generated operation assistance information is then output to the display control unit 250 and the sound generation unit 260. Additionally, the system control unit 270 is configured to generate surgical operation evaluation information based on endoscopic examination information recorded during the endoscopic examination (described later), and this generated surgical operation evaluation information is output to the display control unit 250. The system control unit 270 also includes an examination status information acquisition unit 271, an examination assistance information generation unit 272, a recording unit 273, and a surgical operation evaluation unit 274.

[0054] The examination status information acquisition unit 271 performs processing to acquire examination status information based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, and the insertion position information output from the insertion shape observation device 40. This examination status information corresponds to information indicating the current examination status during the examination of the subject using the endoscope 10 (endoscopic examination). Specifically, for example, such as... Figure 3 As shown, the inspection status information acquisition unit 271 is configured to have a single image recognition processing unit 301, a time series image recognition processing unit 302, and an inspection status detection processing unit 303. Figure 3 This is a block diagram illustrating an example of the structure of the inspection status information acquisition unit in the first embodiment.

[0055] The single image recognition processing unit 301 is configured, for example, to have a recognizer CLP, which is created by installing an object detection algorithm such as YOLO onto a specified neural network such as Darknet and learning it. Alternatively, the single image recognition processing unit 301 is configured, for example, to have a recognizer CLP, which is created by learning the connection coefficients (weights) of a multi-layer neural network including an input layer, one or more convolutional layers, and an output layer using a learning method such as deep learning. Furthermore, the single image recognition processing unit 301 processes the image using the recognizer CLP created as described above, thereby obtaining a recognition result that identifies the examination condition corresponding to one frame of endoscopic image generated by the image processing unit 220 as one of a specified number of conditions.

[0056] In the creation of the aforementioned CLP (Continuous Level Detector), machine learning is performed, for example, using training data that includes a one-frame endoscopic image of the intestine and labels representing the classification of the examination condition corresponding to the one-frame endoscopic image into one of a set of conditions.

[0057] Therefore, according to the aforementioned recognizer CLP, for example, by acquiring multidimensional data such as the pixel values ​​of each pixel contained in a one-frame endoscopic image generated by the image processing unit 220, and inputting this multidimensional data as input data into the input layer of a neural network, multiple likelihoods can be obtained as output data output from the output layer of the neural network. These multiple likelihoods correspond to various conditions that can be classified as the examination condition corresponding to the one-frame endoscopic image. Furthermore, according to the processing using the aforementioned recognizer CLP, for example, a condition corresponding to the highest likelihood among the multiple likelihoods contained in the output data output from the output layer of the neural network can be obtained as the recognition result of the examination condition corresponding to the one-frame endoscopic image generated by the image processing unit 220.

[0058] Furthermore, the single image recognition processing unit 301 is configured to process the image using the aforementioned CLP to obtain, for example, at least one of the following pieces of information as the information included in the recognition result of the examination status corresponding to the one frame of endoscopic image generated by the image processing unit 220: the presence or absence of a lumen within the field of view of the camera unit 110 of the insertion unit 11 inserted into the intestine, the type of lumen present within the field of view, the anatomical location within the intestine corresponding to the field of view, the position of the lumen present within the field of view, the direction of the lumen present within the field of view, whether narrow-band light observation of the intestine is performed, whether magnified observation of the intestine is performed, whether pigments (such as indigo carmine) are distributed within the intestine, whether a treatment instrument is used within the intestine, whether tissue is collected from the intestine, whether there are any residues within the field of view, whether there are any diverticula within the field of view, and whether there are any lesions within the field of view.

[0059] In this embodiment, through the processing of a single image recognition processing unit 301, for example, when the entire or nearly entire area of ​​the endoscopic image turns red due to the anterior end portion 12 being too close to the inner wall of the intestine, a recognition result indicating that there is no lumen within the field of view of the camera unit 110, which is provided in the insertion portion 11 inserted into the intestine, can be obtained. Furthermore, in this embodiment, as the recognition result for the type of lumen, for example, a recognition result can be obtained that determines which of the following lumens corresponds to the lumen present within the field of view of the camera unit 110 provided in the insertion portion 11: an open lumen corresponding to a lumen in a location where the destination of movement of the anterior end portion 12 is not blocked and can be easily visually identified; a closed lumen corresponding to a lumen in a location where the destination of movement of the anterior end portion 12 is blocked; and a folded lumen corresponding to a lumen in a location where the destination of movement of the anterior end portion 12 is not blocked and can be difficult to visually identify.

[0060] The time-series image recognition processing unit 302 is configured, for example, to have a recognizer CLQ created by learning a 3D-CNN (3D convolutional neural network) using learning methods such as deep learning. Furthermore, the time-series image recognition processing unit 302 is configured to process the image using the CLQ created as described above, thereby obtaining a recognition result that identifies the examination condition corresponding to multiple temporally consecutive endoscopic images generated by the image processing unit 220 as one of a set of predetermined conditions.

[0061] When creating the aforementioned CLQ recognizer, machine learning is performed, for example, using training data that includes multiple frames of endoscopic images obtained by continuously capturing images inside the intestine, and labels representing the classification results of the examination conditions corresponding to the multiple frames of endoscopic images into one of a set of conditions.

[0062] Therefore, according to the aforementioned recognizer CLQ, for example, by acquiring multidimensional data such as pixel values ​​of each pixel contained in a series of temporally consecutive endoscopic images generated by the image processing unit 220, and inputting this multidimensional data as input data into the input layer of a neural network, multiple likelihoods can be obtained as output data output from the output layer of the neural network. These multiple likelihoods correspond to various conditions that can be classified as examination conditions corresponding to the multiple frames of endoscopic images. Furthermore, according to the processing using the aforementioned recognizer CLQ, for example, a condition corresponding to the highest likelihood among the multiple likelihoods contained in the output data output from the output layer of the neural network can be obtained as the recognition result of the examination condition corresponding to the series of temporally consecutive endoscopic images generated by the image processing unit 220.

[0063] Furthermore, the time-series image recognition processing unit 302 is configured to process the image using the aforementioned recognizer CLQ, for example, to obtain at least one of the following pieces of information as information included in the recognition result of the examination status corresponding to multiple temporally consecutive endoscopic images generated by the image processing unit 220: the change in distance of the lumen present in the field of view of the camera unit 110, the change in distance of the folds present in the field of view, the presence or absence of water delivery in the field of view, the presence or absence of air delivery in the field of view, and the presence or absence of suction in the field of view, etc. The camera unit 110 is provided in the insertion part 11 inserted into the intestine.

[0064] In addition, in this embodiment, by processing the time-series image recognition processing unit 302, for example when the intestinal wall is close to the anterior end 12 in a state where no displacement or approximately no displacement occurs, a recognition result indicating that aspiration is taking place within the field of view of the camera unit 110, which is provided in the insertion part 11 inserted into the intestine, can be obtained.

[0065] The inspection status detection processing unit 303 processes at least one of the insertion position information output from the insertion shape observation device 40, the recognition result obtained by the single image recognition processing unit 301, the recognition result obtained by the time series image recognition processing unit 302, and the insertion shape image output from the insertion shape image generation unit 240, thereby obtaining inspection status information representing the current inspection status of the examination (endoscopic examination) performed on the subject using the endoscope 10.

[0066] Furthermore, in this embodiment, the inspection condition detection processing unit 303 is configured to calculate the current insertion length of the insertion part 11 and the current bending angle of the bending part 13 based on at least one of the insertion position information output from the insertion shape observation device 40 and the insertion shape image output from the insertion shape image generation unit 240. Additionally, in this embodiment, the inspection condition detection processing unit 303 is configured to perform, for example, a process substantially the same as the analysis process of the insertion shape analysis processing unit 312 described later, based on at least one of the insertion position information output from the insertion shape observation device 40 and the insertion shape image output from the insertion shape image generation unit 240, thereby calculating the estimated pressing pressure currently applied to the intestinal wall by the insertion part 11, which is estimated to be inserted into the intestine. In addition, in this embodiment, the following will be explained: by the processing of the inspection status detection processing unit 303, inspection status information is obtained, for example, the current operation amount of one or more parameters related to the insertion operation of the insertion unit 11, such as the rotation angle when the insertion unit 11 is rotated about the insertion axis (length axis).

[0067] The inspection assistance information generation unit 272 is configured to perform processing to generate operation assistance information including guidance information based on the inspection status information obtained by the inspection status information acquisition unit 271, and to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260. The operation assistance information assists the insertion operation of the insertion unit 11 performed by the user operating the endoscope 10 during an endoscopy.

[0068] Specifically, the auxiliary information generation unit 272 is configured, for example, to read from the storage medium 20M. Figure 4 The table data TDA shown is read from multiple guidance information contained in the table data TDA. One guidance information is selected that corresponds to the current inspection status determined based on the inspection status information obtained by the inspection status information acquisition unit 271. Operation assistance information containing the selected guidance information is generated and output to the display control unit 250 and the sound generation unit 260. Figure 4 This is a diagram illustrating an example of table data used in the processing performed in the endoscope system of the first embodiment.

[0069] exist Figure 4 The "Inspection Status" column of the TDA table data contains multiple items representing the overview of various inspection statuses that can be detected by the inspection status detection processing unit 303 (such as "N-shaped bend (rope) formation" and "excessive pressing pressure"). Additionally, in Figure 4The "Guide Information" column of the TDA table data contains, for example, multiple items indicating the type of guide corresponding to each item in the "Inspection Status" column (e.g., "N-shaped bend release" and "reduced pressure") and the corresponding operating method (e.g., "Please press" and "Please turn right"). Additionally, in Figure 4 The "Operation Check Parameters" column of the TDA table data contains multiple items with one or more parameters set to determine whether the operation corresponding to each item in the "Guidance Information" column has been completed.

[0070] Therefore, for example, if the current inspection status determined by the inspection status information obtained by the inspection status information acquisition unit 271 is inspection status EJA, the inspection assistance information generation unit 272 selects the guidance information EGA corresponding to the inspection status EJA, generates operation assistance information including the guidance information EGA, and outputs the operation assistance information to the display control unit 250 and the sound generation unit 260. Additionally, if the current inspection status determined by the inspection status information obtained by the inspection status information acquisition unit 271 is inspection status EJB, the inspection assistance information generation unit 272 selects the guidance information EGB corresponding to the inspection status EJB, generates operation assistance information including the guidance information EGB, and outputs the operation assistance information to the display control unit 250 and the sound generation unit 260.

[0071] The inspection assistance information generation unit 272 is configured to perform processing to determine whether the operation assistance information in the current output needs to be updated, based on the inspection status information obtained by the inspection status information acquisition unit 271, for example, whenever one frame of endoscopic image is output from the image processing unit 220.

[0072] Specifically, the auxiliary information generation unit 272 is inspected, for example, from... Figure 4 If boot information EGA is selected from the boot information contained in the table data TDA, the parameter EPA is obtained as the operation check parameter corresponding to the boot information EGA. The obtained parameter EPA and the check status information obtained by the check status information acquisition unit 271 are used to perform threshold determination and other processing to determine whether the operation assistance information containing the boot information EGA in the current output needs to be updated. Additionally, the check assistance information generation unit 272, for example, from... Figure 4 If the boot information EGB is selected from the boot information contained in the table data TDA, the parameters EPB and EPC are obtained as operation check parameters corresponding to the boot information EGB. The obtained parameters EPB and EPC, as well as the check status information obtained by the check status information acquisition unit 271, are used to perform threshold determination and other processing, thereby determining whether it is necessary to update the operation auxiliary information containing the boot information EGB in the current output.

[0073] The inspection assistance information generation unit 272 is configured such that, upon receiving a determination result indicating that the operation assistance information in the current output needs to be updated, it performs processing to generate new operation assistance information containing guidance information different from the guidance information already included in the operation assistance information, based on the inspection status information obtained by the inspection status information acquisition unit 271, and performs processing to output the generated new operation assistance information to the display control unit 250 and the sound generation unit 260. Furthermore, if a determination result indicating that the operation assistance information in the current output does not need to be updated is received, the inspection assistance information generation unit 272 performs processing to re-output the operation assistance information to the display control unit 250 and the sound generation unit 260.

[0074] The recording unit 273 is configured to record images and information obtained during an examination of the subject using the endoscope 10 (endoscopic examination) as endoscopic examination information.

[0075] Specifically, during endoscopic examinations, the recording unit 273 performs operations such as recording the endoscopic image output from the image processing unit 220, the insertion position information output from the insertion shape observation device 40, and the insertion shape image output from the insertion shape image generation unit 240. Furthermore, whenever an operation assistance information is generated by the examination assistance information generation unit 272, the recording unit 273 records information indicating the duration of the operation assistance information output to the display control unit 250 as operation assistance time information.

[0076] That is, in the recording unit 273, as endoscopy examination information in one endoscopy examination, information obtained by associating the endoscopy image group, the insertion position information group, the insertion shape image group, and the operation assistance time information group are recorded. The endoscopy image group includes multiple frames of endoscopy images (obtained in one endoscopy examination) sequentially output from the image processing unit 220 in one endoscopy examination. The insertion position information group includes multiple insertion position information (obtained in one endoscopy examination) sequentially output from the insertion shape observation device 40 in one endoscopy examination. The insertion shape image group includes multiple insertion shape images sequentially output from the insertion shape image generation unit 240 in one endoscopy examination. The operation assistance time information group indicates the output time and output content of each of the multiple operation assistance information output from the examination assistance information generation unit 272 to the display control unit 250 in one endoscopy examination. That is, the operation assistance time information group recorded in the recording unit 273 contains information that can determine the prompting time for the multiple guidance messages contained in the multiple operation assistance messages generated in one endoscopy to be presented to the user, according to each guidance message.

[0077] The surgical procedure evaluation unit 274 is configured to generate procedure evaluation information based on endoscopic examination information recorded in the recording unit 273, which includes information on evaluating the surgical procedure. It then performs processing to output the generated surgical procedure evaluation information to the display control unit 250. The surgical procedure includes the insertion of the insertion unit 11 by the user during an examination of the subject using the endoscope 10 (endoscopic examination). Specifically, for example, as... Figure 5 As shown, the surgical operation evaluation unit 274 is configured to include: an endoscopic image analysis and processing unit 311, an insertion shape analysis and processing unit 312, and an evaluation processing unit 313. Figure 5 This is a block diagram illustrating an example of the structure of the surgical operation evaluation unit in the first embodiment.

[0078] The endoscope image analysis and processing unit 311 is configured to perform analysis and processing on the endoscope image group contained in the endoscope examination information recorded in the recording unit 273, thereby obtaining an analysis result including the analysis value of one or more evaluation indicators corresponding to the operations estimated to be performed by the user during the endoscope examination.

[0079] Specifically, the endoscopic image analysis and processing unit 311 is configured to perform analysis and processing based on the endoscopic image set contained in the endoscopic examination information recorded in the recording unit 273, for example, to obtain analysis results including analysis values ​​of at least one of the following evaluation indicators: presence or absence of lumens in the intestine that is the object of the endoscopic examination, type of lumens in the intestine, location of lumens in the intestine, presence or absence of residues in the intestine, location of residues in the intestine, proportion of images containing residues in the endoscopic image set, number of times water is delivered into the intestine, number of times aspiration is performed into the intestine, presence or absence of lesions in the intestine, number of times tissue is collected from the intestine, distance between the tissue that is the object of collection and the anterior end 12 in the intestine, number of times pigments (indigo carmine, etc.) are distributed in the intestine, number of times magnified observation is used in the intestine, time of magnified observation in the intestine, number of times narrowband light observation is used in the intestine, time of narrowband light observation in the intestine, and the amount of rotation of the insertion part 11 inserted into the intestine, etc.

[0080] Furthermore, in this embodiment, when the examination time, which is equivalent to the time required for one endoscopy, is defined as PEA, the analysis values ​​corresponding to the presence or absence of lumens in the intestine are, for example, information that can determine the proportion of time PLY (when any lumen in the intestine enters the field of view of the camera unit 110) in the examination time PEA, and the proportion of time PLN (when any lumen in the intestine does not enter the field of view) in the examination time PEA. The time required for one endoscopy is calculated based on the total number of frames of the endoscopic image group included in the endoscopy examination information recorded in the recording unit 273. In addition, in this embodiment, the analysis values ​​corresponding to the types of lumens in the intestine are, for example, information that can determine the proportion of time PLA (when an open lumen enters the field of view of the camera unit 110) in time PLY, the proportion of time PLB (when a closed lumen enters the field of view of the camera unit 110) in time PLY, and the proportion of time PLC (when a folded lumen enters the field of view of the camera unit 110) in time PLY. Furthermore, in this embodiment, the analysis value corresponding to the position of the lumen within the intestine is, for example, information that can determine the coordinate position of the lumen contained in each endoscopic image obtained within time PLY. Additionally, in this embodiment, the analysis value corresponding to the presence or absence of residue in the intestine is, for example, information that can determine the proportion of time PRY during which residue present in the intestine enters the field of view of the camera unit 110 within the examination time PEA, and the proportion of time PRN during which residue present in the intestine does not enter the field of view within the examination time PEA. Furthermore, in this embodiment, the analysis value corresponding to the position of residue in the intestine is, for example, information that can determine the position of residue contained in each endoscopic image obtained within time PRY. Additionally, in this embodiment, the analysis value corresponding to the amount of rotation of the insertion part 11 inserted into the intestine is information indicating the rotation angle when the insertion part 11 is rotated about the insertion axis (length axis).

[0081] The insertion shape analysis and processing unit 312 is configured to perform analysis and processing based on the insertion shape image group and insertion position information group contained in the endoscopic examination information recorded in the recording unit 273, thereby obtaining an analysis result including the analysis value of one or more evaluation indicators corresponding to the operation estimated to be performed by the user during the endoscopic examination.

[0082] Specifically, the insertion shape analysis processing unit 312 is configured to perform analysis processing based on the insertion shape image group and insertion position information group included in the endoscopic examination information recorded in the recording unit 273, thereby obtaining, for example, an analysis result including an analysis value of at least one of the following evaluation indicators: estimated dwell time of each anatomical part in the intestine of the endoscopic examination object, insertion length of the insertion part 11 inserted into the intestine, insertion speed of the insertion part 11 when inserted into the intestine, insertion acceleration of the insertion part 11 when inserted into the intestine, insertion jolt of the insertion part 11 when inserted into the intestine, withdrawal speed of the insertion part 11 when withdrawn from the intestine, withdrawal acceleration of the insertion part 11 when withdrawn from the intestine, withdrawal jolt of the insertion part 11 when withdrawn from the intestine, speed of the insertion tip 12 into the intestine, acceleration of the insertion tip 12 into the intestine, jolt of the insertion tip 12 into the intestine, insertion length of the insertion tip 12 into the intestine, insertion speed of the insertion tip 12 into the intestine, insertion length of the insertion tip 1 ... The bending angle of the curved portion 13 inside the intestine, the bending angular velocity of the curved portion 13 inserted into the intestine, the bending angular acceleration of the curved portion 13 inserted into the intestine, the bending angular jump of the curved portion 13 inserted into the intestine, the rotation amount of the insertion portion 11 inserted into the intestine, the path length of the insertion portion 11 inserted into the intestine, the estimated pressure exerted by the insertion portion 11 inserted into the intestine on the intestinal wall, the number of times the insertion portion 11 inserted into the intestine is compressed, the duration of the compression of the insertion portion 11 inserted into the intestine, the number of times the insertion portion 11 inserted into the intestine forms a rod shape, the duration of the rod shape, the number of times the insertion portion 11 inserted into the intestine forms a curved shape, the duration of the curved shape, the size of the curved shape, and the rotation amount.

[0083] Furthermore, in this embodiment, the analytical value of the evaluation index corresponding to the estimated residence time of each anatomical location within the intestine is, for example, information indicating the residence time of the anterior portion 12 at each location within the intestine. This information is based on at least one of the insertion shape image group and the insertion position information group included in the endoscopic examination information recorded in the recording unit 273, and the insertion length of the insertion portion 11 inserted into the intestine. Additionally, in this embodiment, the analytical value of the evaluation index corresponding to the path length of the insertion portion 11 inserted into the intestine is, for example, information indicating the total amount of movement accumulated according to various operations that cause displacement of the anterior portion 12, such as insertion, withdrawal, and bending of the insertion portion 11. Furthermore, in this embodiment, as an analysis value of an evaluation index corresponding to the estimated pressure exerted on the intestinal wall by the insertion portion 11 estimated to be inserted into the intestine, information indicating the magnitude of the force is obtained, for example, based on the following information: an insertion shape determined according to at least one of the insertion shape image group and the insertion position information group included in the endoscopic examination information recorded in the recording unit 273, and an estimated value of the bending stiffness estimated based on the insertion shape. Additionally, in this embodiment, as an analysis value of an evaluation index corresponding to the size of the bending shape formed by the insertion portion 11 inserted into the intestine, information indicating the longitudinal (corresponding to the vertical direction of the human body) length of the portion corresponding to the bending shape and information indicating the transverse (corresponding to the horizontal direction of the human body) length of the portion corresponding to the bending shape are obtained, for example.

[0084] The evaluation processing unit 313 is configured to process the analysis results obtained by the endoscope image analysis processing unit 311, the analysis results obtained by the insertion shape analysis processing unit 312, and the operation assistance time information group included in the endoscope examination information recorded in the recording unit 273, thereby generating surgical operation evaluation information, and performing processing for outputting the generated surgical operation evaluation information to the display control unit 250.

[0085] In this embodiment, at least a portion of the functions of the main device 20 can be implemented by the processor 20P. Furthermore, in this embodiment, at least a portion of the main device 20 can be configured as various electronic circuits, or as circuit blocks in an integrated circuit such as an FPGA (Field Programmable Gate Array). Additionally, by appropriately modifying the structure of this embodiment, for example, a computer can read a program from a storage medium 20M, such as a memory, to execute at least a portion of the functions of the main device 20, and perform actions corresponding to the read program.

[0086] like Figure 2As shown, the insertion shape observation device 40 is configured to have a receiving antenna 410 and an insertion position information acquisition unit 420.

[0087] The receiving antenna 410 is configured, for example, to have multiple coils for 3D detection of magnetic fields emitted from multiple source coils 18 respectively. Furthermore, the receiving antenna 410 is configured to detect magnetic fields emitted from multiple source coils 18 respectively, generate a magnetic field detection signal corresponding to the strength of the detected magnetic field, and output the magnetic field detection signal to the insertion position information acquisition unit 420.

[0088] The insertion position information acquisition unit 420 acquires the positions of each of the plurality of source coils 18 based on the magnetic field detection signal output from the receiving antenna 410. Furthermore, the insertion position information acquisition unit 420 is configured to generate insertion position information representing the positions of each of the plurality of source coils 18 acquired as described above, and output this insertion position information to the insertion shape image generation unit 240 and the system control unit 270.

[0089] Specifically, as the positions of the multiple source coils 18, the insertion position information acquisition unit 420 acquires, for example, multiple 3D coordinate values ​​in a hypothetical spatial coordinate system with the origin or reference point being a predetermined position (such as the anus) of the subject into which the insertion unit 11 is inserted. Furthermore, the insertion position information acquisition unit 420 generates insertion position information including the multiple 3D coordinate values ​​acquired as described above, and outputs this insertion position information to the insertion shape image generation unit 240 and the system control unit 270. In this case, the insertion shape image generation unit 240 performs the following processes: acquiring multiple 2D coordinate values ​​corresponding to the multiple 3D coordinate values ​​included in the insertion position information output from the insertion position information acquisition unit 420; interpolating the acquired multiple 2D coordinate values; and generating an insertion shape image corresponding to the interpolated multiple 2D coordinate values.

[0090] That is, the insertion shape information, which indicates the insertion shape of the insertion part 11 inserted into the subject, includes insertion position information and an insertion shape image. The insertion position information indicates the insertion position of the insertion part 11 inserted into the subject, and the insertion shape image is obtained by visualizing the insertion shape of the insertion part 11 inserted into the subject. In addition, the insertion shape information group recorded in the recording unit 273 during an endoscopy includes an insertion position information group and an insertion shape image group. The insertion position information group contains multiple insertion position information sequentially output from the insertion shape observation device 40 during the endoscopy, and the insertion shape image group contains multiple insertion shape images sequentially output from the insertion shape image generation unit 240 during the endoscopy.

[0091] In this embodiment, at least a portion of the insertion shape observation device 40 can be configured as an electronic circuit, or as a circuit block in an integrated circuit such as an FPGA (Field Programmable Gate Array). Additionally, in this embodiment, the insertion shape observation device 40 can also be configured to have one or more processors (CPUs, etc.).

[0092] Next, refer to Figure 6 The function of this embodiment will be explained. Figure 6 This is a flowchart illustrating an example of the processing performed in the endoscope system of the first embodiment.

[0093] After connecting the various parts of the endoscope system 1 and turning on the power, the user begins the operation of inserting the insertion part 11 from the anus of the subject into the interior of the large intestine.

[0094] According to the user's operation as described above, illumination light supplied from the light source unit 210 shines on the subject, and the subject illuminated by the illumination light is captured by the imaging unit 110. The endoscopic image obtained from capturing the subject is output from the image processing unit 220 to the display control unit 250 and the system control unit 270. Furthermore, according to the user's operation as described above, a coil drive signal is supplied from the coil drive signal generation unit 230. Magnetic fields are emitted from the plurality of source coils 18 according to the coil drive signal. Insertion position information obtained by detecting the magnetic field is output from the insertion position information acquisition unit 420 to the insertion shape image generation unit 240 and the system control unit 270. An insertion shape image generated based on the insertion position information is output from the insertion shape image generation unit 240 to the system control unit 270.

[0095] For example, after the user has just inserted the insertion part 11 into the large intestine of the subject, the user can instruct the user to start the operation of the examination start switch (not shown) on the input device 50 to begin actions related to assisting the endoscopic examination of the subject.

[0096] Upon detecting an indication of an inspection start switch from the input device 50, the recording unit 273 begins to record the output time and output content of the endoscope image output from the image processing unit 220, the insertion position information output from the insertion shape observation device 40, the insertion shape image output from the insertion shape image generation unit 240, and the operation assistance information output from the inspection assistance information generation unit 272 to the display control unit 250.

[0097] Upon detecting an indication to start the inspection from the input device 50, the inspection status information acquisition unit 271 and the inspection assistance information generation unit 272 begin cyclic LA processing, which is a process for assisting operation corresponding to the current inspection status during the endoscopic examination. Figure 6 Step S1).

[0098] The examination status information acquisition unit 271 performs processing to acquire examination status information indicating the current examination status during the endoscopic examination, based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, and the insertion position information output from the insertion shape observation device 40. Figure 6 Step S2).

[0099] The inspection assistance information generation unit 272 performs processing to generate operation assistance information based on the inspection status information obtained by the inspection status information acquisition unit 271, and performs processing to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260. Figure 6 Step S3).

[0100] The display control unit 250 performs operations such as generating [data] based on the endoscopic image output from the image processing unit 220 and the operation assistance information output from the system control unit 270. Figure 7 The process of generating the display image DGA as shown is performed, and the process of generating the display image DGA is performed for the display device 60 to display the display image DGA. Figure 7 This is a diagram illustrating an example of a display image generated by the processing of the endoscope system according to the first embodiment.

[0101] The Display Area of ​​Images (DGA) includes: Endoscopic Image Display Area (EDA), Marker Display Area (MDA), Guide Display Area (GDA), and Operational Volume Display Area (SDA).

[0102] The endoscope image display area EDA is set to be the area for displaying the endoscope image EG output from the image processing unit 220.

[0103] exist Figure 7 In this context, the marked display area is, for example, set as four triangular regions located outside the endoscope image display area (EDA), and positioned in four directions (upper left, lower left, upper right, and lower right) when viewed from the center of the EDA (see reference). Figure 7Alternatively, the marker display area MDA can be set at any position around the octagonal endoscope image display area EDA. Furthermore, when there is no lumen within the endoscope image EG currently displayed in the endoscope image display area EDA, the marker display area MDA is set to display a marker MG, such as an arrow, which indicates the direction of the lumen closest to the field of view of the camera unit 110 corresponding to the endoscope image EG.

[0104] The boot display area GDA is configured to display strings and symbols generated based on operation assistance information output from the system control unit 270. Additionally, within the boot display area GDA, for example... Figure 7 As shown, it includes the overview display area GDB, the boot type display area GDC, the boot content display area GDD, and the operation example display area GDE.

[0105] The overview display area GDB is configured to display a string representing an overview of the current examination status during the endoscopy. Furthermore, the content displayed in the overview display area GDB varies according to the guidance information included in the operation assistance information output from the system control unit 270.

[0106] The Guide Type Display Area (GDC) is configured to display a string indicating the type of operation guidance used to assist the insertion operation of the insertion unit 11 based on the current examination status during endoscopic examination. Furthermore, the content displayed in the Guide Type Display Area (GDC) varies according to the guidance information included in the operation assistance information output from the system control unit 270.

[0107] The boot content display area GDD is configured to display a string indicating the specific operation method of the insertion unit 11 corresponding to the type of operation guide currently displayed in the boot type display area GDC. Furthermore, the content displayed in the boot content display area GDD varies according to the boot information included in the operation assistance information output from the system control unit 270.

[0108] The operation example display area GDE is set to display a graphic representing the change in the insertion shape of the insertion part 11 before and after an operation corresponding to the specific operation method currently displayed in the guide content display area GDD is performed. Specifically, the operation example display area GDE is, for example, as follows: Figure 7As shown, the area is set to display the solid curve CEA and the dashed curve CEB. The solid curve CEA represents the insertion shape of the insertion part 11 before the operation related to the release of the N-shaped bend is performed, and the dashed curve CEB represents the insertion shape of the insertion part 11 after the operation is performed. Furthermore, the display content of the operation example display area GDE changes according to the guidance information included in the operation assistance information output from the system control unit 270.

[0109] The operation quantity display area SDA is configured to display a graph representing the difference between the target operation quantity required to terminate an operation corresponding to the content displayed in the boot type display area GDC and the boot content display area GDD, and the current operation quantity in that operation. Specifically, the operation quantity display area SDA is, for example, as shown in the graph. Figure 7 As shown, the area is configured to display the insertion length meter MIA, which represents the difference DIA between the target insertion length TIA of the insertion unit 11 and the current insertion length CIA of the insertion unit 11. Additionally, the operation quantity display area SDA is, for example, as shown... Figure 7 As shown, the area is configured to display the rotation measurement meter MRA and the insertion length meter MIA together. The rotation measurement meter MRA represents the difference DRA between the target rotation measurement TRA of the insertion section 11 and the current rotation measurement CRA of the insertion section 11. That is, Figure 7 The operation quantity display area SDA illustrates the following situation: the insertion length and rotation amount of the insertion unit 11 are used as parameters for operation checks, and a threshold determination is performed based on the target insertion length and target rotation amount of the insertion unit 11.

[0110] Furthermore, according to this embodiment, for example, display methods corresponding to VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality) can also be used to display at least a portion of the information contained in each display area of ​​the display image DGA.

[0111] The sound generating unit 260 generates a sound corresponding to the guidance information included in the operation assistance information output from the system control unit 270, and performs an operation to output the generated sound to the outside of the main device 20 at regular intervals.

[0112] Specifically, the sound generation unit 260 generates a sound, for example, that corresponds to a string in the guide content display area GDD currently displayed on the display image DGA, and performs an operation to output the generated sound to the outside of the main device 20 at regular intervals.

[0113] The inspection assistance information generation unit 272, based on the inspection status information obtained by the inspection status information acquisition unit 271, performs a process to determine whether an update is needed whenever one frame of endoscopic image is output from the image processing unit 220. Figure 6 The processing of the operation auxiliary information generated in step S3 (in the current output) Figure 6 Step S4).

[0114] Specifically, the inspection assistance information generation unit 272 maintains the current inspection status determined based on the inspection status information obtained by the inspection status information acquisition unit 271, for example, by using the inspection status information to determine the current inspection status. Figure 4 The table data TDA specifies one of the inspection conditions in each inspection condition, and the detection is performed by threshold determination of operation inspection parameters, etc., to determine if there is a problem with the condition. Figure 6 If the operation corresponding to the guidance information generated in step S3 has ended, it is estimated that the operation has been completed, and the information that needs to be updated is obtained. Figure 6 The determination result of the operation assistance information generated in step S3. Additionally, the inspection assistance information generation unit 272, based on the inspection status information obtained by the inspection status information acquisition unit 271, such as the detection of a condition related to the operation inspection parameters threshold determination, etc. Figure 6 In step S3, if the operation corresponding to the guidance information generated in the operation assistance information is not yet completed, the current inspection status determined based on the inspection status information is determined from the following state: Figure 4 If one of the inspection conditions specified by TDA in the table data changes to another inspection condition, it is estimated that the operation was not completed and needs to be updated. Figure 6 The determination result of the operation assistance information generated in step S3. Furthermore, the inspection assistance information generation unit 272, based on the inspection status information obtained by the inspection status information acquisition unit 271, maintains the current inspection status determined according to that inspection status information at the level specified by the inspection assistance information generation unit 272. Figure 4 The table data TDA specifies one of the inspection conditions in each inspection condition, and the condition is determined by threshold judgment of the operation inspection parameters, etc., to detect a condition that is not present in the table. Figure 6 If the operation corresponding to the guidance information generated in step S3 is not yet completed, it is estimated that the operation is in progress, and no update is needed. Figure 6 The determination result of the operation assistance information generated in step S3.

[0115] The auxiliary information generation unit 272 checked the information that needed to be updated. Figure 6If the determination result of the operation assistance information generated in step S3 is true (S4: Yes), the process of recording the information representing the estimation result together with the operation assistance time information into the recording unit 273 is performed, and then the process described later is carried out. Figure 6 The processing in step S5 involves estimating whether the operation corresponding to the guidance information contained in the operation assistance information has been completed. Additionally, the assistance information generation unit 272 checks whether updates are needed in the process. Figure 6 If the determination result of the operation assistance information generated in step S3 is (S4: No), the operation assistance information is output to the display control unit 250 and the sound generation unit 260 again, and then the process is repeated. Figure 6 The processing of step S4.

[0116] In addition, the auxiliary information generation unit 272 checks, for example, a certain amount of time has elapsed since a determination result indicating that the auxiliary information for this operation does not need to be updated was obtained, and has performed the operation a predetermined number of times. Figure 6 In any case of the processing in step S4, the process also proceeds to the following description. Figure 6 The processing of step S5 is not limited to obtaining the information that needs to be updated. Figure 6 The determination result of the operation auxiliary information generated in step S3.

[0117] For example, after the user has just finished withdrawing the insertion part 11 that was inserted into the large intestine of the subject, the user can indicate by operating the inspection end switch (not shown) of the input device 50 to stop the actions related to assisting the endoscopic examination of the subject.

[0118] The inspection status information acquisition unit 271 and the inspection auxiliary information generation unit 272 perform cyclic LA terminal processing. Figure 6 Step S5). Specifically, the inspection status information acquisition unit 271 and the inspection assistance information generation unit 272, for example, if they fail to detect an indication from the inspection end switch of the input device 50, return to... Figure 6 In step S1, the loop LA process is performed again. Additionally, the inspection status information acquisition unit 271 and the inspection assistance information generation unit 272, for example, upon detecting an indication from the inspection end switch of the input device 50, end the loop LA process and terminate the process. Figure 6 A series of processes.

[0119] When the recording unit 273 detects an indication from the inspection end switch of the input device 50, it stops the operation of recording the output time and output content of the endoscope image output from the image processing unit 220, the insertion position information output from the insertion shape observation device 40, the insertion shape image output from the insertion shape image generation unit 240, and the operation assistance information output from the inspection assistance information generation unit 272 to the display control unit 250.

[0120] Next, refer to Figure 8 Flowcharts, etc., are repeatedly performed during endoscopic examinations. Figure 6 Specific examples of the processing in steps S3 and S4 will be described below. Furthermore, the insertion operation of the insertion section 11 will be described below as an example of an operation following the axis-maintaining shortening method when passing through the sigmoid colon. Additionally, the case where there is no lumen of a type corresponding to a closed lumen will be described below. Figure 8 This is a flowchart illustrating an example of the processing performed in the endoscope system of the first embodiment.

[0121] The inspection auxiliary information generation unit 272 performs processing to determine whether the anterior end 12 has passed the S-top corresponding to the top of the sigmoid colon, based on the inspection status information obtained by the inspection status information acquisition unit 271. Figure 8 Step S101).

[0122] Specifically, the inspection auxiliary information generation unit 272 detects, for example, based on the inspection status information obtained by the inspection status information acquisition unit 271, inspection auxiliary information generation unit 272. Figure 9 If the current bending angle of the curved portion 13, corresponding to the angle θα, is 100 degrees or more, and a cavity exists within the field of view of the camera unit 110, the front end portion 12 is determined to have passed the S-top. Alternatively, if the inspection auxiliary information generation unit 272 detects, based on the inspection status information obtained by the inspection status information acquisition unit 271, that the current bending angle of the curved portion 13 is less than 100 degrees, and no cavity exists within the field of view of the camera unit 110, the front end portion 12 is determined to have failed the S-top. Figure 9 It is a schematic representation of the basis Figure 8 The figure shows an example of the state of the intestine when the insertion part has been inserted.

[0123] If the auxiliary information generation unit 272 receives a determination result that the front end 12 has not passed S-top (S101: No), it generates, for example, operational auxiliary information containing guidance information for displaying operational guidance, and then performs the operation again. Figure 8In step S101, the operation guidance prompts a pressing operation equivalent to advancing the insertion part 11. Additionally, if the auxiliary information generation unit 272 receives a determination that the front end 12 has passed S-top (S101: Yes), it generates and outputs operation auxiliary information containing guidance information for displaying the operation guidance. Figure 8 Step S102), then, continue with the procedure described later. Figure 8 The processing of step S103 guides and prompts a pulling operation equivalent to the operation of retracting the insertion part 11.

[0124] Alternatively, according to this embodiment, for example, the auxiliary information generation unit 272 may be checked in... Figure 8 In step S101, if the current bending angle of the bent portion 13 is less than 100 degrees, and a determination result indicating that the front end portion 12 has not passed S-top is obtained based on the detection result that a lumen is detected within the field of view of the camera portion 110, operation assistance information that does not display operation guidance related to the operation of the insertion portion 11 is generated. Furthermore, according to this embodiment, for example, in... Figure 8 In step S101, if it is determined that the current bending angle of the bending portion 13 is 100 degrees or more and there is no lumen within the field of view of the camera unit 110, the inspection assistance information generation unit 272 can also generate operation assistance information containing guidance information for displaying operation guidance, which prompts the operation of positioning the front end portion 12 at a position where the lumen can be visually identified.

[0125] The inspection auxiliary information generation unit 272 performs processing to determine whether the current insertion length of the insertion unit 11 is less than 20 cm based on the inspection status information obtained by the inspection status information acquisition unit 271. Figure 8 Step S103).

[0126] If the auxiliary information generation unit 272 receives a determination result that the current insertion length of the insertion unit 11 is 20cm or less (S103: Yes), it continues with the process described later. Figure 8 The processing in step S104. Additionally, if the auxiliary information generation unit 272 receives a determination result that the current insertion length of the insertion unit 11 is greater than 20cm (S103: No), it outputs again... Figure 8 The operation assistance information generated in step S102 (including guidance information for displaying operation guidance to prompt the pulling operation) is then performed again. Figure 8 The processing of step S103.

[0127] After the user made a interaction Figure 8In the case of the operations corresponding to the processing from step S101 to step S103, for example, Figure 9 The intestine, as shown, is deformed by being pulled closer to the anus. Figure 10 The shape shown. Figure 10 It is a schematic representation of the basis Figure 8 The figure shows an example of the state of the intestine when the insertion part has been inserted.

[0128] The inspection auxiliary information generation unit 272 performs processing to determine whether the current bending angle of the bent portion 13 is 100 degrees or more, based on the inspection status information obtained by the inspection status information acquisition unit 271. Figure 8 Step S104).

[0129] If the auxiliary information generation unit 272 determines that the current bending angle of the bending section 13 is less than 100 degrees (S104: No), it continues with the process described later. Figure 8 The processing of step S106. Additionally, if the auxiliary information generation unit 272 determines that the current bending angle of the bending section 13 is 100 degrees or more (S104: Yes), it generates operation auxiliary information containing guidance information for displaying operation guidance. Figure 8 Step S106), then, repeat. Figure 8 The process of step S104 guides the operation of reducing the bending angle of the bent portion 13.

[0130] The inspection auxiliary information generation unit 272 performs processing to determine whether a lumen exists within the field of view of the camera unit 110 based on the inspection status information obtained by the inspection status information acquisition unit 271. Figure 8 Step S106).

[0131] If the auxiliary information generation unit 272 receives a determination result indicating that a lumen exists within the field of view of the camera unit 110 (S106: Yes), it continues with the process described later. Figure 8 The processing in step S108. Additionally, if the auxiliary information generation unit 272 determines that there is no lumen within the field of view of the camera unit 110 (S106: No), it generates operation auxiliary information including guidance information. Figure 8 Step S107), then proceed to the steps described later. Figure 8 The processing of step S113, the guidance information is used to display operation guidance that prompts the pulling operation.

[0132] The inspection auxiliary information generation unit 272 performs processing to determine the type of lumen present within the field of view of the camera unit 110 based on the inspection status information obtained by the inspection status information acquisition unit 271. Figure 8 Step S108).

[0133] Upon receiving a determination that the type of lumen present within the field of view of the camera unit 110 is an open lumen, the inspection assistance information generation unit 272 generates operation assistance information including guidance information. Figure 8 Step S109), then proceed to the steps described later. Figure 8 In step S111, the guidance information is used to display an operation guide that prompts the pressing operation. Additionally, if the auxiliary information generation unit 272 determines that the type of lumen present within the field of view of the camera unit 110 is a folded lumen, it generates operation auxiliary information including the guidance information. Figure 8 Step S110), then proceed to the steps described later. Figure 8 The processing of step S111, the guidance information is used to display operation guidance that prompts the operation of bringing the front end 12 close to and into the lumen.

[0134] The inspection auxiliary information generation unit 272 performs processing based on the inspection status information obtained by the inspection status information acquisition unit 271 to determine whether the intestinal wall within the field of view of the camera unit 110 is close to (the anterior end 12 side). Figure 8 Step S111).

[0135] If the auxiliary information generation unit 272 receives a determination result indicating that the intestinal wall is close (to the anterior end 12 side) within the field of view of the camera unit 110 (S111: Yes), it continues with the process described later. Figure 8 The processing of step S113. Additionally, if the auxiliary information generation unit 272 receives a determination result that the intestinal wall (anterior end 12 side) is not approached within the field of view of the camera unit 110 (S111: No), it generates operational auxiliary information containing guidance information. Figure 8 Step S112), then proceed as described later. Figure 8 The processing of step S113, the guidance information is used to display operation guidance that prompts the pulling operation and the torsional operation equivalent to the operation of rotating the insertion part 11 about the insertion axis (length axis).

[0136] The inspection auxiliary information generation unit 272 performs processing to determine whether the anterior end 12 has completed the passage of the sigmoid colon based on the inspection status information obtained by the inspection status information acquisition unit 271. Figure 8 Step S113).

[0137] Specifically, the examination auxiliary information generation unit 272, based on the examination status information obtained by the examination status information acquisition unit 271, detects, for example, that the anatomical location corresponding to the field of view of the camera unit 110 is the descending colon, that the current insertion length of the insertion unit 11 is 25 cm or more, and that the insertion unit 11 is in a straight line (see reference). Figure 11 In the case of [missing information], the anterior portion 12 is determined to have completed the passage of the sigmoid colon. Additionally, the examination assistance information generation unit 272, based on the examination status information obtained by the examination status information acquisition unit 271, detects at least one of the following: the anatomical location corresponding to the field of view of the camera unit 110 is not the descending colon; the current insertion length of the insertion unit 11 is less than 25 cm; or the insertion unit 11 is not in a straight line. In this case, the anterior portion 12 is determined to have not completed the passage of the sigmoid colon. Figure 11 It is a schematic representation of the basis Figure 8 The figure shows an example of the state of the intestine when the insertion part has been inserted.

[0138] If the auxiliary information generation unit 272 receives a determination result indicating that the anterior end 12 has not completed the passage of the sigmoid colon (S113: No), it continues with the process described later. Figure 8 The processing of step S114. Additionally, if the auxiliary information generation unit 272 receives a determination result indicating that the anterior end 12 has completed the passage of the sigmoid colon (S113: Yes), the process ends. Figure 8 A series of processes are then performed, followed by the processing involved in generating operational assistance information corresponding to the sites following the descending colon.

[0139] The inspection auxiliary information generation unit 272 performs processing to determine whether the current insertion length of the insertion unit 11 is less than 30cm based on the inspection status information obtained by the inspection status information acquisition unit 271. Figure 8 Step S114).

[0140] If the auxiliary information generation unit 272 receives a determination result that the current insertion length of the insertion unit 11 is greater than 30cm (S114: No), it returns. Figure 8 Step S101 is performed. Additionally, if the auxiliary information generation unit 272 receives a determination result that the current insertion length of the insertion unit 11 is 30cm or less (S114: Yes), it returns... Figure 8 The process is carried out in step S104.

[0141] Furthermore, in this embodiment, during the cyclic LA process, the inspection assistance information generation unit 272 performs the following processes: determining whether the insertion part 11 has been bent, determining whether the insertion part 11 has formed a rod shape, and determining whether the estimated pressing pressure applied by the insertion part 11 to the intestinal wall is excessive. Therefore, in this embodiment, when the inspection assistance information generation unit 272 obtains a determination result that the insertion part 11 has been bent during endoscopic examination, it performs processing to generate and output operational assistance information including guidance information, which displays operational guidance to facilitate the elimination of the bending. Additionally, in this embodiment, when the inspection assistance information generation unit 272 obtains a determination result that the insertion part 11 has formed a rod shape during endoscopic examination, it performs processing to generate and output operational assistance information including guidance information, which displays operational guidance to facilitate the elimination of the rod shape. In addition, in this embodiment, when the examination assistance information generation unit 272 obtains the determination result that the estimated pressing pressure applied by the insertion unit 11 to the intestinal wall during the endoscopy is excessive, it performs processing to generate and output operation assistance information containing guidance information, which is used to display operation guidance to prompt the reduction of the estimated pressing pressure.

[0142] The user instructs the user to operate the surgical operation evaluation display switch (not shown) of the input device 50 at any time after the inspection end switch of the operation input device 50, for example, to display the evaluation results of the surgical operation performed during the endoscopic examination.

[0143] When the surgical operation evaluation unit 274 detects an instruction from the surgical operation evaluation display switch of the input device 50, it generates surgical operation evaluation information based on the endoscopic examination information recorded in the recording unit 273, and outputs the generated surgical operation evaluation information to the display control unit 250.

[0144] The display control unit 250 performs, for example, the generation of surgical operation evaluation information output from the surgical operation evaluation unit 274. Figure 12 The process of displaying the image DGB as shown is performed, and the process of displaying the generated image DGB is performed for the display device 60 to display it. Figure 12 This is a diagram illustrating an example of a display image generated by the processing of the endoscope system according to the first embodiment.

[0145] The displayed image DGB includes: Time Series Information Display Area (ADA), Surgical Operation Evaluation Value Display Area (BDA), Operation Completion Rate Display Area (CDA), Comments Display Area (DDA), and Operation Level Display Area (FDA).

[0146] The time-series information display area (ADA) is set to display a chart showing the time change of one of the evaluation indicators included in the surgical operation evaluation information output from the surgical operation evaluation unit 274 after the endoscopy. Additionally, within the time-series information display area (ADA), charts such as... Figure 12 As shown, there is an additional period of interest information (ADB), which indicates the period of time during which a dangerous or special operation was estimated to have been performed in each operation corresponding to the guidance information included in the operation assistance information generated by the examination assistance information generation unit 272 during the endoscopy.

[0147] Specifically, the time periods for additional attention time period information (ADB) can be exemplified by the following time periods: the time period during which the insertion part 11 inserted into the intestine was operated according to the axis-maintaining shortening method; the time period during which an operation was performed to straighten the transverse colon in the intestine; the time period during which an operation related to the passage of the splenic flexure in the intestine was performed; the time period during which an operation related to the passage of the hepatic flexure in the intestine was performed; the time period during which an operation was performed to eliminate the deflection of the insertion part 11 inserted into the intestine was performed; the time period during which an operation was performed to eliminate the rod-like shape formed by the insertion part 11 inserted into the intestine was performed; the time period during which the estimated excessive pressing pressure applied by the insertion part 11 inserted into the intestine to the intestinal wall was detected; and the time period during which no lumen was present in the field of view of the camera unit 110 provided in the insertion part 11 inserted into the intestine.

[0148] In this embodiment, for example, the surgical operation evaluation unit 274 can also perform processing to display a chart showing how one of the evaluation indicators in the surgical operation evaluation information output from the surgical operation evaluation unit 274 changes according to the insertion length of the insertion unit 11. Furthermore, in this embodiment, for example, when an operation is performed to select a time period information ADB within the time series information display area ADA, the surgical operation evaluation unit 274 can perform processing to display the endoscopic image and insertion shape image recorded in the recording unit 273 during the period corresponding to that time period information ADB as a dynamic image on the display device 60.

[0149] The Surgical Operation Evaluation Value Display Area (BDA) is set up to display surgical operation evaluation values, such as surgical operation scores, which represent a comprehensive evaluation of surgical operations performed during an endoscopy.

[0150] Here, an example of the calculation method for the surgical operation score displayed in the surgical operation evaluation value display area BDA will be explained. Furthermore, for convenience, the following explanation will focus on the case where the analysis result obtained by the endoscopic image analysis processing unit 311 includes the analysis value SPV corresponding to the evaluation index SP, and the analysis result obtained by the insertion shape analysis processing unit 312 includes the analysis value SQV corresponding to the evaluation index SQ.

[0151] The evaluation processing unit 313 performs processing to set the maximum score SMAX, the specified score AMP of the evaluation index SP in the maximum score SMAX, and the specified score AMQ of the evaluation index SQ in the maximum score SMAX, respectively. The maximum score SMAX is equivalent to the maximum value of the surgical operation score SSV, which represents the comprehensive evaluation of the surgical operation performed in the endoscopic examination.

[0152] In addition, the specified scores AMP and AMQ can be set to values ​​corresponding to the relative importance of the evaluation metrics SP and SQ, or they can be set to values ​​obtained by dividing the maximum score SMAX into two equal parts.

[0153] The evaluation processing unit 313 reads, for example, distribution information JSP containing the following parameters (mean and standard deviation of the analysis values ​​of the evaluation index SP) from the storage medium 20M, and performs calculations using the distribution information JSP and the analysis value SPV to calculate the deviation value HPV. This parameter represents the distribution status of the analysis values ​​of the evaluation index SP obtained in past endoscopic examinations.

[0154] The evaluation processing unit 313 reads, for example, distribution information JSQ containing the following parameters (mean and standard deviation of the analysis values ​​of the evaluation index SQ) from the storage medium 20M, and performs calculations using the distribution information JSQ and the analysis value SPQ to calculate the deviation value HPQ. This parameter represents the distribution status of the analysis values ​​of the evaluation index SQ obtained in past endoscopic examinations.

[0155] Evaluation processing unit 313, for example, reads from storage medium 20M Figure 13 The table data TDP, as shown, represents the correspondence between deviation values ​​and scores. The table data TDP, along with the specified scores AMP and AMQ, are used to perform calculations to obtain the scores PVS corresponding to the deviation value HPV and QVS corresponding to the deviation value HPQ, respectively.

[0156] exist Figure 13 The "Deviation Value" column of the TDP table data contains multiple items representing the range of deviation values ​​calculated by the evaluation processing unit 313. Additionally, in Figure 13The “Score” column of the table data TDP contains multiple items with information on the calculation method that determines the score. The score calculation method uses a predetermined score set by the evaluation processing unit 313 and a coefficient such as a score rate preset for each range of deviation values ​​shown in the “Deviation Value” column.

[0157] The evaluation processing unit 313 calculates the surgical operation score SSV by adding the scores PVS and QVS obtained as described above, generates surgical operation evaluation information containing the calculated surgical operation score SSV, and outputs the surgical operation evaluation information to the display control unit 250. Figure 13 This is a diagram illustrating an example of table data used in the processing performed in the endoscope system of the first embodiment.

[0158] Furthermore, in this embodiment, it is not limited to displaying the surgical operation score SSV calculated by the above method in the surgical operation evaluation value display area BDA. For example, the surgical operation evaluation value obtained by evaluating the difference between the surgical operation score SSV and the maximum score SMAX through multi-stage evaluation can also be displayed in the surgical operation evaluation value display area BDA.

[0159] The operation completion rate display area CDA is, for example, set to display the operation completion rate, which is equivalent to the proportion of operations completed in an endoscopy corresponding to each operation guide displayed on the display device 60.

[0160] That is, the surgical operation evaluation unit 274 calculates the operation completion rate, which is the proportion of operations completed in the endoscopy corresponding to each guidance information provided, based on the information representing the estimation result processed and recorded in the recording unit 273 by the examination assistance information generation unit 272. It generates surgical operation evaluation information containing information representing the calculated operation completion rate and outputs the surgical operation evaluation information to the display control unit 250.

[0161] The Comment Display Area (DDA) is set as an area for displaying comments containing strings related to the evaluation of surgical procedures performed during the endoscopy. Furthermore, the comments displayed in the Comment Display Area (DDA) are, for example, pre-set to correspond to the types of evaluation indicators and the analytical values ​​of those evaluation indicators included in the surgical procedure evaluation information output from the surgical procedure evaluation unit 274 after the endoscopy.

[0162] The Operational Level Display Area (FDA) is, for example, set up to display a graph showing the difference between the analyzed value and the target value of the evaluation indicators included in the surgical operation evaluation information output from the Surgical Operation Evaluation Unit 274 after the endoscopy. Specifically, the Operational Level Display Area (FDA) can, for example, display a radar graph or similar graph showing the difference between the analyzed value and the target value of each of the multiple evaluation indicators included in the surgical operation evaluation information output from the Surgical Operation Evaluation Unit 274.

[0163] Furthermore, according to this embodiment, the target value used when displaying the chart in the operation level display area FDA is, for example, set to the average value of the analytical values ​​obtained when a skilled person performs an endoscopy. Additionally, according to this embodiment, the operation level display area FDA may, for example, display a chart showing the difference between the latest analytical value and past analytical values ​​among the evaluation indicators included in the surgical operation evaluation information output from the surgical operation evaluation unit 274.

[0164] According to this embodiment, for example, display methods corresponding to VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality) can also be used to display at least a portion of the information contained in each display area of ​​the display image DGB.

[0165] As described above, according to this embodiment, the user can be prompted with the operation method of the insertion section 11 corresponding to the current examination status during the endoscopy. Furthermore, according to this embodiment, after the endoscopy is completed, the user can be prompted with an evaluation of the surgical procedures performed during the endoscopy. Therefore, according to this embodiment, the quality of the endoscopy can be improved compared to conventional methods.

[0166] Furthermore, according to this embodiment, for example, when it is possible to selectively choose between a formal mode corresponding to an endoscopy for a human body and a training mode corresponding to an endoscopy for a colon model through operation of the input device 50, processing can also be performed to differentiate the displayed content related to the evaluation results of the surgical procedure. Specifically, for example, when the training mode is selected, the operation completion rate is displayed in the operation completion rate display area CDA; on the other hand, when the formal mode is selected, processing can also be performed to not display the operation completion rate in the operation completion rate display area CDA.

[0167] Alternatively, according to this embodiment, the system control unit 270 may be configured to control the movement of the robot capable of operating the endoscope 10 based on the inspection status information obtained by the inspection status information acquisition unit 271.

[0168] Furthermore, according to this embodiment, the system control unit 270 may, for example, be configured as an inspection evaluation value calculation unit. This unit calculates an inspection evaluation value for evaluating the quality of the endoscopic examination based on the endoscopic examination information recorded in the recording unit 273. The system control unit 270 may also be configured to output the calculated inspection evaluation value to a database or cloud system external to the main device 20. Moreover, according to this embodiment, the system control unit 270 may be configured to calculate, for example, at least one of the following: ileocecal reach rate, intestinal cleansing degree, and adenoma detection rate, as the aforementioned inspection evaluation value.

[0169] Furthermore, according to this embodiment, the system control unit 270 may be configured to generate electronic medical record information based on endoscopic examination information recorded in the recording unit 273, and to output the generated electronic medical record information to an electronic medical record system outside the main device 20. Moreover, according to this embodiment, the system control unit 270 may generate information including at least one of the following as the aforementioned electronic medical record information: an endoscopic image obtained by photographing the intestine, the location where the endoscopic image was obtained, the time when the endoscopic image was photographed, the location where magnified observation was used in the intestine, the time when magnified observation was used in the intestine, the location where narrowband light observation was used in the intestine, the time when narrowband light observation was used in the intestine, the location where tissue was collected in the intestine, the time when tissue was collected in the intestine, the location where pigment was distributed in the intestine, the time when pigment was distributed in the intestine, the location where a lesion was detected in the intestine, the time when a lesion was detected in the intestine, the type of treatment device used in the intestine, and the time when the insertion part 11 was withdrawn from the intestine.

[0170] Furthermore, according to this embodiment, at least a portion of the functions implemented by the examination status information acquisition unit 271 and the surgical operation evaluation unit 274 can be achieved by combining the following processes: processing to sequentially generate 3D shape data corresponding to the shape of the intestine inserted by the insertion unit 11 based on the endoscopic image output from the image processing unit 220, and processing to detect the movement of the anterior end portion 12 corresponding to the time change of the endoscopic image. In addition, in this embodiment, as the processing for generating the aforementioned 3D shape data, processing following the Structure from Motion method can be used, for example. Furthermore, in this embodiment, as the processing for detecting the aforementioned movement of the anterior end portion 12, processing following the optical flow method can be used, for example.

[0171] Furthermore, according to this embodiment, for example, each endoscope image recorded in the recording unit 273 during an endoscopy can also be used as a learning image when creating a classifier that has the same function as the recognizer CLP.

[0172] Furthermore, according to this embodiment, for example, each endoscope image recorded in the recording unit 273 during an endoscopy can also be used as a learning image when creating a classifier that has the same function as the recognizer CLQ.

[0173] Additionally, according to this embodiment, for example, multiple table data with the following characteristics can also be stored in the storage medium 20M: Figure 4 The table data TDA has the same structure and is set to provide guidance information in different modes for each doctor or each tissue. Alternatively, in such cases, for example, the examination assistance information generation unit 272 may select one of the multiple table data stored in the storage medium 20M based on either the instruction from the input device 50 or the examination status information obtained by the examination status information acquisition unit 271, and perform processing to generate operational assistance information including guidance information obtained from the selected table data.

[0174] Additionally, according to this embodiment, for example, multiple table data with the following characteristics can also be stored in the storage medium 20M: Figure 4 The table data TDA has the same structure and is configured to provide guidance information in multiple different modes for each operational difficulty related to the insertion operation of the insertion unit 11. Alternatively, in such a case, for example, the inspection assistance information generation unit 272 may select one table data from multiple table data stored in the storage medium 20M based on either an instruction from the input device 50 or inspection status information obtained by the inspection status information acquisition unit 271, and perform processing to generate operation assistance information including guidance information obtained from the selected table data. That is, the inspection assistance information generation unit 272 may also be configured to provide guidance information in different modes for each operational difficulty related to the insertion operation of the insertion unit 11.

[0175] Here, refer to Figure 14 The processing of variations of this embodiment will be explained. Figure 14 This is for explaining the processes performed in the endoscope system of the first embodiment. Figure 6 Flowcharts for different examples.

[0176] Furthermore, for convenience, specific explanations related to the application of the previously described actions will be omitted hereafter. Additionally, the following example will be used for explanation: Table data TDN for normal difficulty, configured to provide guidance information in a mode corresponding to the insertion operation of the normal difficulty insertion unit 11, and table data TDV for high difficulty, configured to provide guidance information in a mode corresponding to the insertion operation of the high difficulty insertion unit 11, are stored in storage medium 20M, and one of these two table data is selected based on the inspection status information obtained by the inspection status information acquisition unit 271.

[0177] After connecting the various parts of the endoscope system 1 and turning on the power, the user gives instructions through the operation input device 50 so as to prompt the operation method in a mode corresponding to the insertion operation of the difficult insertion part 11.

[0178] The auxiliary information generation unit 272 reads the advanced table data TDV from the storage medium 20M according to the instruction from the input device 50, thereby setting the prompt mode of the operation method related to the insertion operation of the insertion unit 11 to the advanced mode. Figure 14 Step S201).

[0179] After receiving instructions related to the operation mode of the insertion unit 11, the user begins the operation of inserting the insertion unit 11 from the anus of the subject into the large intestine. Alternatively, for example, after initially inserting the insertion unit 11 into the large intestine of the subject, the user may instruct the user to begin actions related to assisting with the endoscopic examination of the subject by operating the examination start switch on the input device 50.

[0180] Upon detecting an indication from the inspection start switch of the input device 50, the inspection status information acquisition unit 271 and the inspection assistance information generation unit 272 begin the cyclic LM process, which is an operation assistance process corresponding to the current inspection status in the endoscopy. Figure 14 Step S202).

[0181] The examination status information acquisition unit 271 performs processing to acquire examination status information indicating the current examination status during the endoscopic examination, based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, and the insertion position information output from the insertion shape observation device 40. Figure 14 Step S203).

[0182] The inspection assistance information generation unit 272 performs processing to generate operation assistance information based on the inspection status information obtained by the inspection status information acquisition unit 271, and performs processing to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260. Figure 14 Step S204).

[0183] The display control unit 250, based on the endoscopic image output from the image processing unit 220 and the operation assistance information output from the system control unit 270, performs operations to generate an image that includes the endoscopic image from the image processing unit 220 and the operation assistance information output from the system control unit 270. Figure 7 The display image DGM is processed to obtain different information from the display image DGA, and the processing is performed to make the display device 60 display the generated display image DGM.

[0184] The display image DGM is generated, for example, as follows: In the Boot Type Display Area (GDC), the display representation is... Figure 7 The display image DGA shows the types of operation guides compared to the subdivided strings of operation guide types, and in the boot content display area GDD, it generates a higher level of comparison. Figure 7 The DGA display image shows a more stringent operating method.

[0185] The sound generating unit 260 generates a sound corresponding to the guidance information included in the operation assistance information output from the system control unit 270, and performs an operation to output the generated sound to the outside of the main device 20 at regular intervals.

[0186] The inspection assistance information generation unit 272, based on the inspection status information obtained by the inspection status information acquisition unit 281, performs a process to determine whether an update is needed whenever one frame of endoscopic image is output from the image processing unit 220. Figure 14 The processing of the operation auxiliary information generated in step S204 (in the current output) Figure 14 Step S205).

[0187] The auxiliary information generation unit 272 checked the information that needed to be updated. Figure 14 If the determination result of the operation assistance information generated in step S204 is obtained (S205: Yes), processing is performed to record the information representing the estimation result together with the operation assistance time information into the recording unit 273, and then the following process is performed. Figure 14 The processing in step S206 involves estimating whether the operation corresponding to the guidance information contained in the operation assistance information has been completed. Additionally, the assistance information generation unit 272 checks whether updates are needed in the process. Figure 14If the determination result of the operation assistance information generated in step S204 is (S205: No), the operation assistance information is output to the display control unit 250 and the sound generation unit 260 again, and then the process is repeated. Figure 14 The processing of step S205.

[0188] In addition, the auxiliary information generation unit 272 checks, for example, a certain amount of time has elapsed since a determination result indicating that the auxiliary information for this operation does not need to be updated was obtained, and has performed the operation a predetermined number of times. Figure 14 In any case of the processing in step S205, the process also proceeds to the following description. Figure 14 The processing of step S206 is not limited to obtaining the update required in Figure 14 The determination result of the operation assistance information generated in step S204.

[0189] The auxiliary information generation unit 272 checks based on... Figure 14 The information representing the estimation result recorded in the recording unit 273 during step S205 processing is used to perform a prompt mode for determining whether to change the currently set operation method. Figure 14 Step S206).

[0190] Specifically, the auxiliary information generation unit 272 checks the information based on... Figure 14 The information representing the estimation result recorded in the recording unit 273 during step S205 processing, if it is detected that the number of times the operation corresponding to the guidance information contained in the high-difficulty table data TDV has not been completed exceeds a predetermined number, a determination result of the prompt mode for changing the currently set operation method is obtained. Furthermore, specifically, the auxiliary information generation unit 272 checks the information based on... Figure 14 The information representing the estimation result is recorded in the recording unit 273 during the processing of step S205. If it is detected that the number of times the operation corresponding to the guidance information contained in the table data TDV for high difficulty is not completed is less than a predetermined number, a determination result is obtained that the prompt mode does not change the operation method in the current setting.

[0191] If the auxiliary information generation unit 272 receives a determination result indicating that the prompt mode for changing the currently set operation method is changed (S206: Yes), for example, it reads the table data TDN for normal difficulty from the storage medium 20M, and resets the prompt mode for the operation method related to the insertion operation of the insertion unit 11 to the normal difficulty mode. Figure 14 Step S207), and then, proceed as described later. Figure 14The processing in step S208. Additionally, if the auxiliary information generation unit 272 determines that the prompt mode does not change the currently set operation method (S206: No), then the process described later is performed. Figure 14 The processing of step S208.

[0192] The inspection status information acquisition unit 271 and the inspection auxiliary information generation unit 272 perform terminal processing of the loop LM. Figure 14 Step S208). Specifically, the inspection status information acquisition unit 271 and the inspection assistance information generation unit 272, for example, if they fail to detect an indication from the inspection end switch of the input device 50, return to... Figure 14 In step S202, the cyclic LM process is performed again. Additionally, the inspection status information acquisition unit 271 and the inspection assistance information generation unit 272, for example, upon detecting an indication from the inspection end switch of the input device 50, end the cyclic LM process and terminate... Figure 14 A series of processes.

[0193] Furthermore, in this modified example, if the auxiliary information generation unit 272 reads the table data TDN used in normal difficulty, for example, the display control unit 250 performs a process to generate data containing... Figure 7 The processing includes the same information as the displayed image DGA. Furthermore, in this modified example, the display control unit 250 performs processing to generate a display image containing information similar to... Figure 12 The same information as the displayed image DGB is included in the processing of the displayed image.

[0194] As described above, according to this modified example, the user can be prompted with the operating method of the insertion section 11 corresponding to the current examination status during the endoscopy. Furthermore, according to this modified example, after the endoscopy is completed, the user can be prompted with an evaluation of the surgical procedures performed during the endoscopy. Therefore, according to this modified example, the quality of the endoscopy examination can be improved compared to conventional methods.

[0195] (Second Implementation)

[0196] Figures 15 to 18 This is a diagram of the second embodiment.

[0197] In addition, in this embodiment, detailed descriptions of parts having the same structure as the first embodiment are appropriately omitted, and descriptions are mainly given of parts having a structure different from the first embodiment.

[0198] For example, such as Figure 15As shown, the endoscope system 1A is configured to include: an endoscope 10, a main body device 20A, an insertion shape observation device 40, an input device 50, a display device 60, a camera 71, a motion sensor 72, a line-of-sight sensor 73, and a throat microphone 74. Figure 15 This is a block diagram illustrating an example of the structure of the endoscope system according to the second embodiment.

[0199] The main unit 20A is configured to have a system control unit 280 instead of the system control unit 270 in the main unit 20. Furthermore, the main unit 20A is configured to have one or more processors 20P and storage media 20M.

[0200] The camera 71 is positioned, for example, to capture images of at least the upper body of the user operating the endoscope 10. Furthermore, the camera 71 is configured to capture images of the user operating the endoscope 10 and output these images to the main unit 20A. Alternatively, in this embodiment, a TOF camera or the like may be used instead of the camera 71.

[0201] The motion sensor 72 is configured, for example, to be a device that can be worn on the hands of a user operating the endoscope 10. In addition, the motion sensor 72 is configured to acquire motion information equivalent to information that can determine the movement of the user operating the endoscope 10, and output the acquired information to the main device 20A.

[0202] Furthermore, in this embodiment, as a sensor for obtaining motion information of the user operating the endoscope 10, a geomagnetic sensor, a pressure sensor, and a bending sensor may also be used.

[0203] The gaze sensor 73 is configured, for example, to be installed near the eyes of the user operating the endoscope 10. Furthermore, the gaze sensor 73 is configured to acquire gaze information equivalent to information that can determine the gaze of the user operating the endoscope 10, and output this acquired information to the main unit 20A.

[0204] The throat microphone 74 is configured, for example, to be mounted on the surface of the larynx of a user operating the endoscope 10. Furthermore, the throat microphone 74 is configured to acquire sound corresponding to vibrations in the larynx of the user operating the endoscope 10 and output the acquired sound to the main unit 20A.

[0205] The system control unit 280 is configured to include: an examination status information acquisition unit 281, an examination auxiliary information generation unit 282, a recording unit 283, and a surgical operation evaluation unit 284.

[0206] The examination status information acquisition unit 281 is configured to process examination status information based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, the insertion position information output from the insertion shape observation device 40, the user's image output from the camera 71, the user's motion information output from the motion sensor 72, the user's gaze information output from the gaze sensor 73, and the user's voice output from the throat microphone 74. This examination status information corresponds to information indicating the current examination status during the endoscopic examination. Specifically, as... Figure 16 As shown, the inspection status information acquisition unit 281 is configured, for example, to have a single image recognition processing unit 301, a time series image recognition processing unit 302, an operation status detection processing unit 304, and an inspection status detection processing unit 305. Figure 16 This is a block diagram illustrating an example of the structure of the inspection status information acquisition unit in the second embodiment.

[0207] The operation status detection and processing unit 304 processes the detection results obtained by detecting at least one of the user's image output from the camera 71, the user's motion information output from the motion sensor 72, the user's gaze information output from the gaze sensor 73, and the user's voice output from the throat microphone 74, thereby obtaining operation status information, which is equivalent to information indicating the current operation status of the user who is operating the endoscope 10.

[0208] Specifically, the operation status detection and processing unit 304 is configured to acquire at least one of the following as information included in the operation status information: the positional relationship between the endoscope 10 and the user's body, the positional relationship between the user's left and right hands during the operation of the insertion part 11, the user's twisting method of the insertion part 11, the height of the user's elbow, the height of the user's right hand relative to the height of the examination table where the subject is placed, the range of motion of the user's elbow, the range of motion of the user's hand, the presence or absence of operations related to the user changing the hardness of the insertion part 11, and the instructions given by the user to the assistant assisting the user.

[0209] The inspection status detection processing unit 305 is configured to process at least one of the following: insertion position information output from the insertion shape observation device 40, recognition result obtained by the single image recognition processing unit 301, recognition result obtained by the time series image recognition processing unit 302, operation status information obtained by the operation status detection processing unit 304, and insertion shape image output from the insertion shape image generation unit 240, thereby obtaining inspection status information representing the current inspection status during endoscopic examination.

[0210] The auxiliary information generation unit 282 is configured to perform processing based on user information input into the input device 50 to determine whether the skill level of the user operating the endoscope 10 is equivalent to that of a skilled person.

[0211] Furthermore, in this embodiment, the user information mentioned above can be input as at least one of the following: the number of years of experience in endoscopic examinations performed by the user operating the endoscope 10, the number of cases performed by the user, the user's clinical department, whether the user has a certified physician qualification, and whether the user has a specialist physician qualification. That is, in this embodiment, it is sufficient for the user information to include at least one piece of information related to the skill level of the user operating the endoscope 10 (performing an endoscopic examination). Additionally, in this embodiment, for example, the examination assistance information generation unit 282 can also perform processing to obtain the aforementioned user information from a database or the like outside the main device 20. Furthermore, according to this embodiment, for example, if the user performing the endoscopic examination has a certified physician qualification or a specialist physician qualification and the number of cases performed by the user is a predetermined number or more, a judgment result indicating that the user's skill level is equivalent to that of a skilled practitioner can be obtained. Furthermore, according to this embodiment, for example, if the user performing the endoscopic examination has a certified physician qualification or a specialist physician qualification, but on the other hand, the number of cases performed by the user is less than a predetermined number, a judgment result indicating that the user's skill level is not equivalent to that of a skilled practitioner can be obtained.

[0212] The inspection assistance information generation unit 282 is configured such that, when it is determined that the skill level of the user operating the endoscope 10 is not equivalent to that of a skilled user, it performs processing to generate operation assistance information including guidance information for non-skilled users based on the inspection status information obtained by the inspection status information acquisition unit 281, and performs processing to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260.

[0213] Specifically, the inspection assistance information generation unit 282 is configured such that, when it is determined that the skill level of the user operating the endoscope 10 is not equivalent to that of a skilled user, it reads table data TDB for non-skilled users from the storage medium 20M, selects one piece of guidance information from the multiple pieces of guidance information contained in the read table data TDB that corresponds to the current inspection status determined based on the inspection status information obtained by the inspection status information acquisition unit 281, generates operation assistance information containing the selected guidance information, and outputs the operation assistance information to the display control unit 250 and the sound generation unit 260.

[0214] The table data TDB used by inexperienced users is created as follows: Figure 4The operation methods in each item of the "Guidance Information" column of the TDA table data have been replaced with detailed operation methods that take into account the user's operation situation (such as "Please look at the lumen and press" and "Please rotate your left wrist and twist to the right").

[0215] Furthermore, in this embodiment, for example, if the inspection assistance information generation unit 282 determines that the skill level of the user operating the endoscope 10 is not equivalent to that of a skilled user, processing can also be performed to determine whether the user's skill level is equivalent to a beginner or an intermediate user. Alternatively, in this embodiment, for example, if the inspection assistance information generation unit 282 determines that the user's skill level is equivalent to that of a beginner, operation assistance information can be generated by reading table data for beginners from the storage medium 20M; and if the determination is that the user's skill level is equivalent to that of an intermediate user, operation assistance information can be generated by reading table data for intermediate users from the storage medium 20M.

[0216] The inspection assistance information generation unit 282 is configured such that, after obtaining a determination result that the skill level of the user operating the endoscope 10 is equivalent to that of a skilled person, it performs processing to generate operation assistance information, including guidance information for skilled persons, based on the inspection status information obtained by the inspection status information acquisition unit 281, and performs processing to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260.

[0217] Specifically, the inspection assistance information generation unit 282 is configured such that, upon obtaining a determination that the skill level of the user operating the endoscope 10 is equivalent to that of a skilled person, it reads the table data TDC used by skilled persons from the storage medium 20M, selects one guidance information from the multiple guidance information contained in the read table data TDC that corresponds to the current inspection status determined based on the inspection status information obtained by the inspection status information acquisition unit 281, generates operation assistance information containing the selected guidance information, and outputs the operation assistance information to the display control unit 250 and the sound generation unit 260.

[0218] The table data TDC used by skilled users is prepared as follows: The "Parameters for Operation Check" column of the table data TDA is deleted, and... Figure 4 The types of guidance and operation methods in each item of the "Guidance Information" column of the TDA table data are replaced with the work content estimated based on the user's operation status (e.g., "The first stage of the operation related to bending release is in progress").

[0219] That is, the inspection assistance information generation unit 282 is configured to generate operation assistance information based on the inspection status information obtained by the inspection status information acquisition unit 281 and user information including information related to the skill level of the user operating the endoscope 10, providing different guidance information based on whether the user's skill level is equivalent to that of a skilled person. Furthermore, the inspection assistance information generation unit 282 is configured to, when a determination result indicating that the user's skill level is equivalent to that of a skilled person is obtained based on the user information input into the input device 50, perform processing to generate operation assistance information including guidance information used to indicate the currently performed task to the user.

[0220] The inspection assistance information generation unit 282 is configured to perform processing to determine whether the operation assistance information in the current output needs to be updated, based on the inspection status information obtained by the inspection status information acquisition unit 281, for example, whenever one frame of endoscopic image is output from the image processing unit 220.

[0221] Specifically, the inspection assistance information generation unit 282, for example, when obtaining guidance information from the table data TDB used by non-skilled users, performs the same processing as when obtaining guidance information from the table data TDA, thereby determining whether the operation assistance information in the current output needs to be updated. Furthermore, for example, when obtaining guidance information from the table data TDC used by skilled users, the inspection assistance information generation unit 282 determines whether the operation assistance information in the current output needs to be updated based on whether the current inspection status, as indicated by the inspection status information obtained by the inspection status information acquisition unit 281, has changed from one of the inspection statuses specified by the table data TDC to another inspection status.

[0222] When the inspection assistance information generation unit 282 determines that the operation assistance information in the current output needs to be updated, it performs processing to generate new operation assistance information containing guidance information different from the guidance information included in the operation assistance information, based on the inspection status information obtained by the inspection status information acquisition unit 281. It then performs processing to output the generated new operation assistance information to the display control unit 250 and the sound generation unit 260. Conversely, when the inspection assistance information generation unit 282 determines that the operation assistance information in the current output does not need to be updated, it performs processing to output the operation assistance information again to the display control unit 250 and the sound generation unit 260.

[0223] The recording unit 283 is configured to perform an action of recording images and information obtained during endoscopic examination as endoscopic examination information.

[0224] Specifically, during endoscopic examination, the recording unit 283 performs operations such as recording the endoscopic image output from the image processing unit 220, the insertion position information output from the insertion shape observation device 40, the insertion shape image output from the insertion shape image generation unit 240, and the operation status information obtained by the operation status detection processing unit 304. Furthermore, whenever an operation assistance information is generated by the examination assistance information generation unit 282, the recording unit 283 records information indicating the duration for which that operation assistance information is continuously output to the display control unit 250.

[0225] That is, in the recording unit 283, as endoscopy examination information in one endoscopy examination, information obtained by associating the endoscopy image group, the insertion position information group, the insertion shape image group, the operation status information group, and the operation assistance time information group are recorded. The endoscopy image group includes multiple frames of endoscopy images sequentially output from the image processing unit 220 in one endoscopy examination. The insertion position information group includes multiple insertion position information sequentially output from the insertion shape observation device 40 in one endoscopy examination. The insertion shape image group includes multiple insertion shape images sequentially output from the insertion shape image generation unit 240 in one endoscopy examination. The operation status information group includes multiple operation status information obtained according to the user's actions in one endoscopy examination. The operation assistance time information group indicates the output time and output content of each of the multiple operation assistance information output from the examination assistance information generation unit 282 to the display control unit 250 in one endoscopy examination.

[0226] The surgical operation evaluation unit 284 is configured to generate surgical operation evaluation information based on endoscopic examination information recorded in the recording unit 283, and to perform processing for outputting the generated surgical operation evaluation information to the display control unit 250. This surgical operation evaluation information includes information obtained by evaluating surgical operations, including the insertion operation of the insertion unit 11 performed by the user during the endoscopic examination. Specifically, for example, as... Figure 17 As shown, the surgical operation evaluation unit 284 is configured to include: an endoscope image analysis and processing unit 311, an insertion shape analysis and processing unit 312, an operation status analysis and processing unit 314, and an evaluation processing unit 315. Figure 17 This is a block diagram illustrating an example of the structure of the surgical operation evaluation unit in the second embodiment.

[0227] The operation status analysis and processing unit 314 is configured to perform analysis and processing on the operation status information group contained in the endoscopy examination information recorded in the recording unit 283, thereby obtaining analysis results including analysis values ​​of one or more evaluation indicators corresponding to the user's actions during the endoscopy examination.

[0228] Specifically, the operation status analysis and processing unit 314 performs analysis and processing based on the operation status information group contained in the endoscopy examination information recorded in the recording unit 283. For example, it can obtain analysis results including the analysis value of at least one evaluation index such as the distance between the left and right hands of the user who is undergoing endoscopy, the coordinate value indicating the position of the user's line of sight, and the range of motion of the user's hand.

[0229] The evaluation processing unit 315 uses the analysis results obtained by the endoscope image analysis processing unit 311, the analysis results obtained by the insertion shape analysis processing unit 312, the analysis results obtained by the operation status analysis processing unit 314, and the operation assistance time information group included in the endoscope examination information recorded in the recording unit 283 to generate surgical operation evaluation information, and performs processing for outputting the generated surgical operation evaluation information to the display control unit 250.

[0230] In this embodiment, at least a portion of the functions of the main device 20A can be implemented by the processor 20P. Furthermore, in this embodiment, at least a portion of the main device 20A can be configured as various electronic circuits, or as circuit blocks in an integrated circuit such as an FPGA (Field Programmable Gate Array). Additionally, by appropriately modifying the structure of this embodiment, for example, a computer can read a program from a storage medium 20M, such as a memory, for executing at least a portion of the functions of the main device 20A, and perform actions corresponding to the read program.

[0231] Next, refer to Figure 18 The function of this embodiment will be explained. Figure 18 This is a flowchart illustrating an example of the processing performed in the endoscope system of the second embodiment.

[0232] After connecting the various parts of the endoscope system 1A and turning on the power, the user inputs user information used to determine the user's skill level through the operation input device 50.

[0233] According to the user's operation as described above, illumination light supplied from the light source unit 210 shines on the subject, and the subject illuminated by the illumination light is captured by the imaging unit 110. The endoscopic image obtained from capturing the subject is output from the image processing unit 220 to the display control unit 250 and the system control unit 280. Furthermore, according to the user's operation as described above, a coil drive signal is supplied from the coil drive signal generation unit 230. Magnetic fields are emitted from multiple source coils 18 according to the coil drive signal. Insertion position information obtained by detecting the magnetic fields is output from the insertion position information acquisition unit 420 to the insertion shape image generation unit 240 and the system control unit 280. The insertion shape image generated based on the insertion position information is output from the insertion shape image generation unit 240 to the system control unit 280. Additionally, according to the user's operation as described above, the user's image obtained by the camera 71, the user's motion information obtained by the motion sensor 72, the user's gaze information obtained by the gaze sensor 73, and the user's voice obtained by the throat microphone 74 are each output to the examination status information acquisition unit 281.

[0234] The auxiliary information generation unit 282 performs processing based on the user information input into the input device 50 to determine whether the skill level of the user operating the endoscope 10 is equivalent to that of a skilled operator. Figure 18 Step S11).

[0235] If the auxiliary information generation unit 282 receives a judgment result indicating that the skill level of the user operating the endoscope 10 is not equivalent to that of a skilled user (S11: No), it reads the table data TDB for non-skilled users from the storage medium 20M. Figure 18 Step S12), and after processing to record the information representing the determination result together with the operation status information to the recording unit 283, is transferred to the process described later. Figure 18 The processing of step S13. Additionally, if the auxiliary information generation unit 282 obtains a determination that the skill level of the user operating the endoscope 10 is equivalent to that of a skilled operator (S11: Yes), it reads the skilled operator's table data TDC from the storage medium 20M. Figure 18 After step S18), and following the processing of recording the information representing the determination result together with the operation status information to the recording unit 283, the process is transferred to the section described later. Figure 18 The processing of step S19.

[0236] After entering user information, the user begins the operation of inserting the insertion unit 11 from the anus of the subject into the large intestine. Additionally, for example, after initially inserting the insertion unit 11 into the large intestine of the subject, the user instructs the user to activate the examination start switch on the input device 50 to begin actions related to assisting with the endoscopic examination of the subject.

[0237] Upon reading the examination status information acquisition unit 281 and the examination assistance information generation unit 282, after detecting an indication from the examination start switch of the input device 50 after reading the table data TDB used by non-skilled personnel, they begin the loop LB process. This loop LB process is used to perform operation assistance processing corresponding to the current examination status in the endoscopy examination. Figure 18 Step S13).

[0238] The examination status information acquisition unit 281 processes examination status information corresponding to the current examination status during the endoscopy based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, the insertion position information output from the insertion shape observation device 40, the user's image output from the camera 71, the user's motion information output from the motion sensor 72, the user's gaze information output from the gaze sensor 73, and the user's voice output from the throat microphone 74. Figure 18 Step S14).

[0239] The inspection auxiliary information generation unit 282 performs operations based on the inspection status information obtained by the inspection status information acquisition unit 281 and... Figure 18 In step S12, the table data TDB read in is used to generate operation assistance information, and the generated operation assistance information is then output to the display control unit 250 and the sound generation unit 260. Figure 18 Step S15).

[0240] The display control unit 250, based on the endoscopic image output from the image processing unit 220 and the operation assistance information output from the system control unit 280, performs, for example, the generation of information containing... Figure 7 The display image with the same information as the display image DGA is processed, and the processing is performed to make the display device 60 display the generated display image.

[0241] The sound generating unit 260 generates a sound corresponding to the guidance information included in the operation assistance information output from the system control unit 280, and performs an operation to output the generated sound to the outside of the main device 20 at regular intervals.

[0242] The inspection assistance information generation unit 282, based on the inspection status information obtained by the inspection status information acquisition unit 281, performs a process to determine whether an update is needed whenever one frame of endoscopic image is output from the image processing unit 220. Figure 18 Processing of the operation assistance information generated in step S15 (in the current output) Figure 18 Step S16).

[0243] In addition, in this embodiment, as Figure 18 The processing in step S16 involves... Figure 6 The same process as step S4 can be followed.

[0244] The auxiliary information generation unit 282 checked the information that needed to be updated. Figure 18 If the determination result of the operation assistance information generated in step S15 is obtained (S16: Yes), processing is performed to record the information representing the estimation result together with the operation assistance time information into the recording unit 283, and then the following process is performed. Figure 18 The processing in step S17 involves estimating whether the operation corresponding to the guidance information contained in the operation assistance information has been completed. Additionally, the assistance information generation unit 282 checks whether updates are needed in the process. Figure 18 If the determination result of the operation assistance information generated in step S15 is (S16: No), the operation assistance information is output to the display control unit 250 and the sound generation unit 260 again, and then the process is repeated. Figure 18 The processing of step S16.

[0245] In addition, the auxiliary information generation unit 282 checks, for example, a certain amount of time has elapsed since the determination result that the auxiliary information for this operation does not need to be updated was obtained, and has performed the operation a predetermined number of times. Figure 18 In any case of the processing in step S16, the process also proceeds to the following description. Figure 18 The processing of step S17 is not limited to obtaining the update required in Figure 18 The determination result of the operation assistance information generated in step S15.

[0246] For example, after the user has just finished withdrawing the insertion part 11 that was inserted into the large intestine of the subject, the user can indicate by operating the inspection end switch of the input device 50 to stop the actions related to assisting the endoscopic examination of the subject.

[0247] The inspection status information acquisition unit 281 and the inspection auxiliary information generation unit 282 perform cyclic load balancing terminal processing. Figure 18Step S17). Specifically, the inspection status information acquisition unit 281 and the inspection assistance information generation unit 282, for example, if they fail to detect an indication from the inspection end switch of the input device 50, return to... Figure 18 In step S13, the loop LB process is performed again. Additionally, the inspection status information acquisition unit 281 and the inspection assistance information generation unit 282, for example, upon detecting an indication from the inspection end switch of the input device 50, end the loop LB process and terminate the operation. Figure 18 A series of processes.

[0248] Upon reading the inspection status information acquisition unit 281 and the inspection assistance information generation unit 282, and upon detecting an indication from the inspection start switch of the input device 50 after reading the table data TDC used by the skilled personnel, they begin the cyclic LC processing. This cyclic LC processing is used to prompt the work content corresponding to the current inspection status in the endoscopy. Figure 18 Step S19).

[0249] The examination status information acquisition unit 281 processes examination status information corresponding to the current examination status during the endoscopy based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, the insertion position information output from the insertion shape observation device 40, the user's image output from the camera 71, the user's motion information output from the motion sensor 72, the user's gaze information output from the gaze sensor 73, and the user's voice output from the throat microphone 74. Figure 18 Step S20).

[0250] The inspection auxiliary information generation unit 282 performs operations based on the inspection status information obtained by the inspection status information acquisition unit 281 and... Figure 18 In step S18, the table data TDC read in is used to generate operation assistance information, and the generated operation assistance information is then output to the display control unit 250 and the sound generation unit 260. Figure 18 Step S21).

[0251] The display control unit 250 performs operations to generate a display image based on the endoscopic image output from the image processing unit 220 and the operation assistance information output from the system control unit 280. Figure 7 The display image DGA is processed differently from the display image DGC, and the processing is performed to make the display device 60 display the generated display image DGC.

[0252] The image DGC is generated, for example, as the following image: This image is from... Figure 7Remove the Boot Type Display Area (GDC), Operation Example Display Area (GDE), and Operation Quantity Display Area (SDA) from the Display Image DGA, and replace the display content of the Boot Content Display Area (GDD) in the Display Image DGA with an image obtained from the content of the user's current operation.

[0253] The inspection assistance information generation unit 282, based on the inspection status information obtained by the inspection status information acquisition unit 281, performs a process to determine whether an update is needed whenever one frame of endoscopic image is output from the image processing unit 220. Figure 18 Processing of the operation assistance information generated in step S21 (in the current output) Figure 18 Step S22).

[0254] Specifically, the inspection auxiliary information generation unit 282, for example, if it detects that the current inspection status indicated by the inspection status information obtained by the inspection status information acquisition unit 281 is maintained at one of the inspection statuses specified by the table data TDC, obtains information that does not need to be updated. Figure 18 The determination result of the operation assistance information generated in step S21. Additionally, the inspection assistance information generation unit 282, for example, if it detects that the current inspection status indicated by the inspection status information obtained by the inspection status information acquisition unit 281 has changed from one of the inspection statuses specified by the table data TDC to another inspection status, obtains the information that needs to be updated in... Figure 18 The determination result of the operation assistance information generated in step S21.

[0255] The auxiliary information generation unit 282 checked the information that needed to be updated. Figure 18 If the determination result of the operation assistance information generated in step S21 is true (S22: Yes), then proceed as described later. Figure 18 The processing of step S23. Additionally, the auxiliary information generation unit 282 checks if it has obtained information that does not need updating. Figure 18 If the determination result of the operation assistance information generated in step S21 is (S22: No), the operation assistance information is output to the display control unit 250 and the sound generation unit 260 again, and then the process is repeated. Figure 18 The processing of step S22.

[0256] In addition, the auxiliary information generation unit 282 checks, for example, a certain amount of time has elapsed since the determination result that the auxiliary information for this operation does not need to be updated was obtained, and has performed the operation a predetermined number of times. Figure 18 In any case of the processing in step S22, the process also proceeds to the following description. Figure 18 The processing of step S23 is not limited to obtaining the update required in Figure 18The determination result of the operation assistance information generated in step S21.

[0257] For example, after the user has just finished withdrawing the insertion part 11 that was inserted into the large intestine of the subject, the user can indicate by operating the inspection end switch of the input device 50 to stop the actions related to assisting the endoscopic examination of the subject.

[0258] The inspection status information acquisition unit 281 and the inspection auxiliary information generation unit 282 perform terminal processing of the cyclic LC. Figure 18 Step S23). Specifically, the inspection status information acquisition unit 281 and the inspection assistance information generation unit 282, for example, if they fail to detect an indication from the inspection end switch of the input device 50, return to... Figure 18 In step S19, the cyclic LC process is performed again. Additionally, the inspection status information acquisition unit 281 and the inspection assistance information generation unit 282, for example, upon detecting an indication from the inspection end switch of the input device 50, end the cyclic LC process and terminate the process. Figure 18 A series of processes.

[0259] For example, at any time after operating the inspection end switch of the input device 50, the user can instruct the user to operate the surgical operation evaluation display switch of the input device 50 to display the evaluation results of the surgical operation performed during the endoscopic examination.

[0260] When the surgical operation evaluation unit 284 detects an instruction from the surgical operation evaluation display switch of the input device 50, it processes the information recorded in the recording unit 283 to determine whether the endoscopic examination information was obtained from a novice or a skilled person.

[0261] If the endoscopic examination information recorded in the recording unit 283 is determined to be obtained from a non-skilled operator, the surgical operation evaluation unit 284 generates surgical operation evaluation information for non-skilled operators and outputs it to the display control unit 250.

[0262] The display control unit 250 performs operations based on surgical operation evaluation information for inexperienced operators output from the surgical operation evaluation unit 284, for example, to generate an evaluation system containing information related to surgical operation evaluation for inexperienced operators. Figure 12 The display image, including the same information as the display image DGB, is processed, and processing is performed to make the display device 60 display the generated display image.

[0263] When the surgical operation evaluation unit 284 receives the judgment result that the endoscopic examination information recorded in the recording unit 283 was obtained from a skilled operator, it generates surgical operation evaluation information for skilled operators and outputs the surgical operation evaluation information to the display control unit 250.

[0264] The display control unit 250 performs operations based on the surgical operation evaluation information for skilled personnel output from the surgical operation evaluation unit 284, for generating and... Figure 12 The display image DGB is processed differently from the display image DGD, and processing is performed to make the display device 60 display the generated display image DGD.

[0265] The image DGD is generated, for example, from... Figure 12 The image obtained by deleting the operation completion rate display area CDA from the display image DGB.

[0266] As described above, according to this embodiment, the operation method of the insertion unit 11 can be prompted according to the skill level of the user performing the endoscopy and the current examination status during the endoscopy. Furthermore, according to this embodiment, an evaluation of the surgical procedures performed during the endoscopy can be prompted to the user after the endoscopy. Therefore, according to this embodiment, the examination quality during endoscopy can be improved compared to conventional methods.

[0267] Furthermore, according to this embodiment, for example, it may be configured such that, when the examination assistance information generation unit 282 detects that the user information input into the input device 50 contains information that can be determined for multiple users separately, it performs processing to distinguish and record the endoscopy examination information obtained in the endoscopy examination in the recording unit 283 for each user. Also, according to this embodiment, for example, the system control unit 280 may be configured to output the endoscopy examination information recorded in the recording unit 283 in a user-distinguished manner to a database or cloud system external to the main device 20A.

[0268] (Third Implementation)

[0269] Figures 19 to 22 This is a diagram of the third embodiment.

[0270] Furthermore, in this embodiment, detailed descriptions related to parts having the same structure as at least one of the first and second embodiments are appropriately omitted, and descriptions are mainly directed to parts having structures different from both the first and second embodiments. Additionally, in this embodiment, an example of performing an endoscopic examination on a patient, equivalent to the subject, will be used for explanation.

[0271] For example, such as Figure 19 As shown, the endoscope system 1B is configured to include: an endoscope 10, a main body device 20B, an insertion shape observation device 40, an input device 50, a display device 60, and a pain information acquisition device 80. Figure 19 This is a block diagram illustrating an example of the structure of the endoscope system according to the third embodiment.

[0272] The main unit 20B is configured to have a system control unit 290 instead of the system control unit 270 in the main unit 20. Furthermore, the main unit 20B is configured to have one or more processors 20P and storage media 20M.

[0273] The pain information acquisition device 80 is configured to acquire pain information equivalent to information indicating the state of pain generation in a patient during an endoscopy, and output the acquired pain information to the main device 20B.

[0274] Specifically, the pain information acquisition device 80 is configured to include, for example, any one of the following devices as a device for acquiring the aforementioned pain information: a manual switch pressed by the patient during an endoscopic examination when the patient feels pain, an electroencephalogram (EEG) measuring the brain waves emitted by the patient, and a heart rate meter measuring the patient's heart rate, etc.

[0275] The system control unit 290 is configured to include: an examination status information acquisition unit 291, an examination auxiliary information generation unit 292, a recording unit 293, and a surgical operation evaluation unit 294.

[0276] The examination status information acquisition unit 291 performs processing to acquire examination status information based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, the insertion position information output from the insertion shape observation device 40, and the pain information output from the pain information acquisition device 80. This examination status information corresponds to information indicating the current examination status during the endoscopic examination. Specifically, for example, as... Figure 20 As shown, the examination status information acquisition unit 291 is configured to include: a single image recognition processing unit 301, a time series image recognition processing unit 302, a physical condition detection processing unit 306, and an examination status detection processing unit 307. Figure 20 This is a block diagram illustrating an example of the structure of the inspection status information acquisition unit in the third embodiment.

[0277] The physical condition detection and processing unit 306 processes the detection results obtained by detecting the pain information output from the pain information acquisition device 80, thereby obtaining physical condition information equivalent to information indicating the current physical condition of the patient during the endoscopy.

[0278] Specifically, the physical condition detection and processing unit 306 is configured to obtain at least one piece of information, such as whether or not the patient experiences pain during an endoscopic examination, and the number of times the patient experiences pain, as information included in the physical condition information.

[0279] The examination status detection processing unit 307 is configured to process at least one of the following: insertion position information output from the insertion shape observation device 40, recognition result obtained by the single image recognition processing unit 301, recognition result obtained by the time series image recognition processing unit 302, body condition information obtained by the body condition detection processing unit 306, and insertion shape image output from the insertion shape image generation unit 240, thereby obtaining examination status information representing the current examination status in the endoscopy examination.

[0280] The auxiliary information generation unit 292 is configured to perform processing to determine whether the insertion difficulty of the insertion unit 11 into a patient undergoing endoscopy is high, based on the patient information input in the input device 50.

[0281] Furthermore, in this embodiment, the patient information mentioned above can be input as at least one of the following: the patient's age, gender, height, weight, medical history, experience with endoscopy, level of pain experienced during past endoscopy examinations, and type of endoscope used in past endoscopy examinations. That is, in this embodiment, at least one piece of information related to the difficulty of inserting the insertion unit 11 into the patient undergoing endoscopy is included in the aforementioned patient information. Additionally, in this embodiment, for example, the examination assistance information generation unit 292 can also process the acquisition of the aforementioned patient information from an electronic medical record system or similar external to the main unit 20. Furthermore, according to this embodiment, for example, if the patient undergoing endoscopy has previously undergone digestive system surgery, a determination result indicating high difficulty in inserting the insertion unit 11 into that patient can be obtained. Furthermore, according to this embodiment, for example, if the patient undergoing endoscopy has not previously undergone digestive system surgery, a determination result indicating low difficulty in inserting the insertion unit 11 into that patient can be obtained.

[0282] The examination assistance information generation unit 292 is configured such that, when it is determined that the insertion difficulty of the insertion unit 11 in a patient undergoing endoscopy is not high, it performs processing to generate operation assistance information including guidance information for normal difficulty based on the examination status information obtained by the examination status information acquisition unit 291, and performs processing to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260.

[0283] Specifically, the examination assistance information generation unit 292 is configured such that, when it is determined that the insertion difficulty of the insertion unit 11 into the patient undergoing endoscopic examination is not high, it reads the table data TDD for normal difficulty from the storage medium 20M, selects one guidance information from the multiple guidance information contained in the read table data TDD that corresponds to the current examination status determined based on the examination status information obtained by the examination status information acquisition unit 291, generates operation assistance information containing the selected guidance information, and outputs the operation assistance information to the display control unit 250 and the sound generation unit 260.

[0284] The table data typically used in TDD is created to have similar difficulty levels. Figure 4 The table data has the same structure as the TDA data.

[0285] The examination assistance information generation unit 292 is configured such that, when a determination is obtained that the insertion unit 11 is inserted into a patient undergoing an endoscopy and the insertion difficulty is high, it performs processing to generate operation assistance information, including guidance information for high-difficulty cases, based on the examination status information obtained by the examination status information acquisition unit 291, and performs processing to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260.

[0286] Specifically, the examination assistance information generation unit 292 is configured such that, when it receives a determination that the insertion difficulty of the insertion unit 11 into a patient undergoing endoscopic examination is high, it reads high difficulty table data TDE from the storage medium 20M, selects one guidance information from the multiple guidance information contained in the read table data TDE that corresponds to the current examination status determined based on the examination status information obtained by the examination status information acquisition unit 291, generates operation assistance information containing the selected guidance information, and outputs the operation assistance information to the display control unit 250 and the sound generation unit 260.

[0287] The complex table data TDE was created as follows: as a... Figure 4 The data is obtained by adding new items to the "Examination Status", "Guidance Information" and "Operation Examination Parameters" columns of the TDA table data, based on the reasons for the increase in insertion difficulty predicted based on past case data (such as intestinal adhesions and the ease of bending of the insertion site).

[0288] That is, the examination assistance information generation unit 292 is configured to generate operation assistance information based on the examination status information obtained by the examination status information acquisition unit 291 and patient information including information related to the difficulty of insertion of the insertion unit 11 into the patient undergoing endoscopy, and to provide different guidance information according to whether the difficulty of insertion of the insertion unit 11 into the patient is high.

[0289] The recording unit 293 is configured to perform actions to record images and information obtained during endoscopic examinations as endoscopic examination information.

[0290] Specifically, during endoscopic examination, the recording unit 293 performs actions such as recording endoscopic images output from the image processing unit 220, insertion position information output from the insertion shape observation device 40, insertion shape images output from the insertion shape image generation unit 240, body condition information obtained by the body condition detection processing unit 306, and patient information input via the input device 50. Furthermore, whenever an operation assistance information is generated by the examination assistance information generation unit 292, the recording unit 293 records information indicating the duration of the operation assistance information output to the display control unit 250.

[0291] That is, as endoscopic examination information in a single endoscopic examination, the recording unit 293 records information obtained by associating endoscopic image groups, insertion position information groups, insertion shape image groups, body condition information groups, operation assistance time information groups, and patient information. The endoscopic image group includes multiple frames of endoscopic images sequentially output from the image processing unit 220 during the single endoscopic examination. The insertion position information group includes multiple insertion position information sequentially output from the insertion shape observation device 40 during the single endoscopic examination. The insertion shape image group includes multiple insertion shape images sequentially output from the insertion shape image generation unit 240 during the single endoscopic examination. The body condition information group includes multiple body condition information obtained based on the patient's pain status during the single endoscopic examination. The operation assistance time information group indicates the output time and output content of each of the multiple operation assistance information output from the examination assistance information generation unit 282 to the display control unit 250 during the single endoscopic examination. The patient information is used to determine the insertion difficulty of the insertion unit 11 into the patient undergoing the single endoscopic examination.

[0292] The surgical operation evaluation unit 294 is configured to generate surgical operation evaluation information based on endoscopic examination information recorded in the recording unit 293, and to perform processing for outputting the generated surgical operation evaluation information to the display control unit 250. This surgical operation evaluation information includes information obtained by evaluating surgical operations, including the insertion operation of the insertion unit 11 performed by the user during the endoscopic examination. Specifically, for example, such as... Figure 21 As shown, the surgical operation evaluation unit 294 is configured to include: an endoscopic image analysis and processing unit 311, an insertion shape analysis and processing unit 312, a body condition analysis and processing unit 316, and an evaluation processing unit 317. Figure 21 This is a block diagram illustrating an example of the structure of the surgical operation evaluation unit in the third embodiment.

[0293] The physical condition analysis and processing unit 316 is configured to perform analysis and processing based on the physical condition information group contained in the endoscopic examination information recorded in the recording unit 293, thereby obtaining analysis results including the analysis value of one or more evaluation indicators corresponding to the physical condition of the patient during the endoscopic examination.

[0294] Specifically, the physical condition analysis and processing unit 316 performs analysis and processing based on the group of operational condition information contained in the endoscopic examination information recorded in the recording unit 293. As a result, for example, it can obtain analysis results including the analysis value of at least one evaluation indicator such as the frequency of pain occurrence in the patient during the endoscopic examination, the intensity of the patient's pain, and the duration of the patient's pain occurrence.

[0295] The evaluation processing unit 317 is configured to process the analysis results obtained by the endoscope image analysis processing unit 311, the analysis results obtained by the insertion shape analysis processing unit 312, the analysis results obtained by the operation status analysis processing unit 316, the operation assistance time information group included in the endoscope examination information recorded in the recording unit 293, and the patient information to generate surgical operation evaluation information, and to process the generated surgical operation evaluation information to output to the display control unit 250.

[0296] In this embodiment, at least a portion of the functions of the main device 20B can be implemented by the processor 20P. Furthermore, in this embodiment, at least a portion of the main device 20B can be configured as various electronic circuits, or as circuit blocks in an integrated circuit such as an FPGA (Field Programmable Gate Array). Additionally, by appropriately modifying the structure of this embodiment, for example, a computer can read a program from a storage medium 20M, such as a memory, to execute at least a portion of the functions of the main device 20B, and perform actions corresponding to the read program.

[0297] Next, refer to Figure 22 The function of this embodiment will be explained. Figure 22 This is a flowchart illustrating an example of the processing performed in the endoscope system of the third embodiment.

[0298] After connecting all parts of the endoscope system 1B and turning on the power, the user inputs patient information through the operation input device 50 to determine the difficulty of insertion of the insertion unit 11 into the patient undergoing endoscopy.

[0299] According to the user's operation as described above, illumination light supplied from the light source unit 210 is irradiated onto the subject, and the subject irradiated by the illumination light is captured by the imaging unit 110. The endoscopic image obtained from capturing the subject is output from the image processing unit 220 to the display control unit 250 and the system control unit 290. Furthermore, according to the user's operation as described above, a coil drive signal is supplied from the coil drive signal generation unit 230. Magnetic fields are emitted from multiple source coils 18 based on this coil drive signal. Insertion position information obtained by detecting this magnetic field is output from the insertion position information acquisition unit 420 to the insertion shape image generation unit 240 and the system control unit 290. An insertion shape image generated based on this insertion position information is output from the insertion shape image generation unit 240 to the system control unit 290.

[0300] The auxiliary information generation unit 292 performs processing based on the patient information input into the input device 50 to determine whether the insertion difficulty of the insertion unit 11 into a patient undergoing endoscopy is high. Figure 22 Step S31).

[0301] If the auxiliary information generation unit 292 receives a determination that the insertion difficulty of the insertion unit 11 in a patient undergoing endoscopy is not high (S31: No), it reads the table data TDD (for normal difficulty) from the storage medium 20M. Figure 22 In step S32), processing is performed to record the information representing the determination result together with the physical condition information into the recording unit 293, and then the process is transferred to the following steps. Figure 22 The processing of step S33. Additionally, if the auxiliary information generation unit 292 receives a determination result indicating high insertion difficulty among patients undergoing endoscopic examination via insertion unit 11 (S31: Yes), it reads the high-difficulty table data TDE (Table Data for High Difficulty) from the storage medium 20M. Figure 22 In step S38), processing is performed to record the information representing the determination result together with the physical condition information into the recording unit 293, and then the process is transferred to the following steps. Figure 22 The processing of step S39.

[0302] After entering patient information, the user begins the operation of inserting the insertion device 11 from the patient's anus into the large intestine. Additionally, for example, after initially inserting the insertion device 11 into the large intestine of the subject, the user instructs the user to activate the examination start switch on the input device 50 to begin actions related to assisting with the endoscopic examination of that subject.

[0303] Based on the user's operation as described above, pain information, such as the pain status of the patient during an endoscopy, is output from the pain information acquisition device 80 to the main device 20B.

[0304] Upon reading the table data TDD for normal difficulty and detecting an indication from the inspection start switch of the input device 50, the inspection status information acquisition unit 291 and the inspection assistance information generation unit 292 begin the cyclic LD process. This cyclic LD process is used to perform operation assistance processing corresponding to the current inspection status in the endoscopy. Figure 22 Step S33).

[0305] The examination status information acquisition unit 291 performs processing to acquire examination status information based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, the insertion position information output from the insertion shape observation device 40, and the pain information output from the pain information acquisition device 80. This examination status information is equivalent to information indicating the current examination status during the endoscopic examination. Figure 22 Step S34).

[0306] The inspection auxiliary information generation unit 292, based on the inspection status information obtained by the inspection status information acquisition unit 291 and in... Figure 22 In step S32, the table data TDD read in is processed to generate operation assistance information, and then processed to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260. Figure 22 Step S35).

[0307] The display control unit 250, based on the endoscopic image output from the image processing unit 220 and the operation assistance information output from the system control unit 290, performs, for example, the generation of information containing... Figure 7 The display image, including the same information as the display image DGA, is processed, and processing is performed to make the display device 60 display the generated display image.

[0308] The sound generating unit 260 generates a sound corresponding to the guidance information included in the operation assistance information output from the system control unit 290, and performs an operation to output the generated sound to the outside of the main device 20 at regular intervals.

[0309] The inspection assistance information generation unit 292, based on the inspection status information obtained by the inspection status information acquisition unit 291, performs a process to determine whether an update is needed whenever one frame of endoscopic image is output from the image processing unit 220. Figure 22 Processing of the operation assistance information generated in step S35 (in the current output) Figure 22 Step S36).

[0310] In addition, in this embodiment, as Figure 22 The processing in step S36 involves... Figure 6 The same process as step S4 can be followed.

[0311] The auxiliary information generation unit 292 checked the information that needed to be updated. Figure 22 If the determination result of the operation assistance information generated in step S35 is obtained (S36: Yes), processing is performed to record the information representing the estimation result together with the operation assistance time information into the recording unit 293, and then the following process is performed. Figure 22 In step S37, the estimation result is obtained by estimating whether the operation corresponding to the guidance information contained in the operation assistance information has been completed. Additionally, the assistance information generation unit 292 checks whether updating is unnecessary. Figure 22 If the determination result of the operation assistance information generated in step S35 is (S36: No), the operation assistance information is output to the display control unit 250 and the sound generation unit 260 again, and then the process is repeated. Figure 22 The processing of step S36.

[0312] In addition, the auxiliary information generation unit 292 checks, for example, that a certain amount of time has elapsed since a determination result indicating that the auxiliary information for this operation does not need to be updated was obtained, and has performed the check a predetermined number of times. Figure 22 In any case of the processing in step S36, the process also proceeds to the following description. Figure 22 The processing of step S37 is not limited to obtaining the update required in Figure 22 The determination result of the operation assistance information generated in step S35.

[0313] For example, after the user has just finished withdrawing the insertion part 11 that was inserted into the large intestine of the subject, the user can indicate by operating the inspection end switch of the input device 50 to stop the actions related to assisting the endoscopic examination of the subject.

[0314] The inspection status information acquisition unit 291 and the inspection auxiliary information generation unit 292 perform terminal processing of the loop LD. Figure 22Step S37). Specifically, the inspection status information acquisition unit 291 and the inspection assistance information generation unit 292, for example, if they fail to detect an indication from the inspection end switch of the input device 50, return to... Figure 22 In step S33, the cyclic LD process is performed again. Additionally, the inspection status information acquisition unit 291 and the inspection auxiliary information generation unit 292, for example, upon detecting an indication from the inspection end switch of the input device 50, end the cyclic LD process and terminate the operation. Figure 22 A series of processes.

[0315] Upon reading the high-difficulty table data TDE and detecting an indication from the inspection start switch of the input device 50, the inspection status information acquisition unit 291 and the inspection assistance information generation unit 292 begin cyclic LE processing. This cyclic LE processing is used to perform operation assistance processing corresponding to the current inspection status in the endoscopy. Figure 22 Step S39).

[0316] The examination status information acquisition unit 291 performs processing to acquire examination status information based on the endoscopic image output from the image processing unit 220, the insertion shape image output from the insertion shape image generation unit 240, the insertion position information output from the insertion shape observation device 40, and the pain information output from the pain information acquisition device 80. This examination status information is equivalent to information indicating the current examination status during the endoscopic examination. Figure 22 Step S40).

[0317] The inspection auxiliary information generation unit 292, based on the inspection status information obtained by the inspection status information acquisition unit 291 and in... Figure 22 In step S38, the table data TDE read in is processed to generate operation assistance information, and then processed to output the generated operation assistance information to the display control unit 250 and the sound generation unit 260. Figure 22 Step S41).

[0318] The display control unit 250, based on the endoscopic image output from the image processing unit 220 and the operation assistance information output from the system control unit 290, generates an image that can contain the endoscopic image from the image processing unit 220 and the operation assistance information output from the system control unit 290. Figure 7 The display image DGE, which contains different information from the display image DGA, is processed, and the processing is performed to make the display device 60 display the generated display image DGE.

[0319] The display image DGE is generated, for example, as follows: In each of the overview display area GDB, boot type display area GDC, and boot content display area GDD, images are selectively displayed that correspond to the information displayed in the image. Figure 7The information displayed in the DGA image is the same as the information displayed in the image, and any information that corresponds to the phenomena that may occur when the insertion of the insertion part 11 is difficult (such as intestinal adhesion).

[0320] The inspection assistance information generation unit 292, based on the inspection status information obtained by the inspection status information acquisition unit 291, performs a process to determine whether an update is needed whenever one frame of endoscopic image is output from the image processing unit 220. Figure 22 Processing of the operation assistance information generated in step S41 (in the current output) Figure 22 Step S42).

[0321] In addition, in this embodiment, as Figure 22 Step S42 processing, performing with Figure 6 The same process as step S4 can be followed.

[0322] The auxiliary information generation unit 292 checked the information that needed to be updated. Figure 22 If the determination result of the operation assistance information generated in step S41 is true (S42: Yes), then proceed as described later. Figure 22 The processing of step S43. Additionally, the auxiliary information generation unit 292 checks if it has obtained information that does not need updating. Figure 22 If the determination result of the operation assistance information generated in step S41 is (S42: No), the operation assistance information is output to the display control unit 250 and the sound generation unit 260 again, and then the process is repeated. Figure 22 The processing of step S42.

[0323] In addition, the auxiliary information generation unit 292 checks, for example, that a certain amount of time has elapsed since a determination result indicating that the auxiliary information for this operation does not need to be updated was obtained, and has performed the check a predetermined number of times. Figure 22 In any case of the processing in step S42, the process also proceeds to the following description. Figure 22 The processing of step S43 is not limited to obtaining the information that needs to be updated. Figure 22 The determination result of the operation assistance information generated in step S41.

[0324] For example, after the user has just finished withdrawing the insertion part 11 that was inserted into the large intestine of the subject, the user can indicate by operating the inspection end switch of the input device 50 to stop the actions related to assisting the endoscopic examination of the subject.

[0325] The inspection status information acquisition unit 291 and the inspection auxiliary information generation unit 292 perform cyclic LE terminal processing. Figure 22Step S43). Specifically, the inspection status information acquisition unit 291 and the inspection assistance information generation unit 292, for example, if they fail to detect an indication from the inspection end switch of the input device 50, return to... Figure 22 In step S39, the LE loop process is performed again. Additionally, the inspection status information acquisition unit 291 and the inspection assistance information generation unit 292, for example, upon detecting an indication from the inspection end switch of the input device 50, end the LE loop process and terminate the operation. Figure 22 A series of processes.

[0326] For example, the user may operate the surgical operation evaluation display switch of the input device 50 at any time after operating the inspection end switch of the input device 50, thereby providing an instruction for displaying the evaluation results of the surgical operation performed during the endoscopic examination.

[0327] When the surgical operation evaluation unit 294 detects an instruction from the surgical operation evaluation display switch of the input device 50, it processes information to determine whether the endoscopic examination information recorded in the recording unit 293 was obtained under normal difficulty or high difficulty conditions.

[0328] When the surgical operation evaluation unit 294 receives the endoscopic examination information recorded in the recording unit 293 and determines that the difficulty level is normal, it generates surgical operation evaluation information for normal difficulty and outputs it to the display control unit 250.

[0329] Specifically, when the evaluation processing unit 317 of the surgical operation evaluation unit 294 receives the judgment result that the endoscopic examination information recorded in the recording unit 293 was obtained at the normal difficulty level, it calculates the surgical operation score by, for example, using the table data TDQ, which represents the correspondence between the deviation value and the score at the normal difficulty level, generates surgical operation evaluation information containing the calculated surgical operation score, and outputs the surgical operation evaluation information to the display control unit 250. Furthermore, the aforementioned table data TDQ is, for example, made in accordance with... Figure 13 The data in the table should be the same as the TDP data.

[0330] The display control unit 250, based on surgical operation evaluation information of typical difficulty output from the surgical operation evaluation unit 294, performs, for example, the generation of information including... Figure 12 The display image, including the same information as the display image DGB, is processed, and processing is performed to make the display device 60 display the generated display image.

[0331] When the surgical operation evaluation unit 294 receives the determination result that the endoscopic examination information recorded in the recording unit 293 is obtained under high difficulty, it generates surgical operation evaluation information for high difficulty and outputs the surgical operation evaluation information to the display control unit 250.

[0332] Specifically, when the evaluation processing unit 317 of the surgical operation evaluation unit 294 receives a determination that the endoscopic examination information recorded in the recording unit 293 was obtained under high difficulty conditions, it reads, for example, a table data TDR representing the correspondence between deviation values ​​and scores under high difficulty conditions from the storage medium 20M, performs calculations using the read table data TDR to calculate the surgical operation score, generates surgical operation evaluation information containing the calculated surgical operation score, and outputs the surgical operation evaluation information to the display control unit 250. Furthermore, the aforementioned table data TDR can be, for example, configured such that a score of one or more is obtained within a deviation value range that is the same as the range of deviation values ​​contained in the table data TDQ, calculated using the calculation method of the table data TDQ.

[0333] Furthermore, according to this embodiment, for example, if the evaluation processing unit 317 detects a predetermined insertion difficulty condition, such as a large amount of residue in the intestine after endoscopic examination, based on at least one of the analysis results obtained by the endoscope image analysis processing unit 311 and the analysis results obtained by the insertion shape analysis processing unit 312, it may cause the evaluation processing unit 317 to adjust the insertion method according to the specified insertion difficulty condition. Figure 22 The determination result of the insertion difficulty of the insertion unit 11 recorded in the recording unit 293 is invalidated due to the processing of step S31, and the determination result that the endoscopy examination information recorded in the recording unit 293 was obtained when the difficulty was high is obtained again.

[0334] The display control unit 250 generates a system based on the high-difficulty surgical operation evaluation information output from the surgical operation evaluation unit 294, which includes information related to the surgical operation evaluation information. Figure 12 The display image DGF is processed, including different information from the display image DGB, and the processing is performed to make the display device 60 display the generated display image DGF.

[0335] The display image DGF is generated, for example, as shown below, which is displayed in the comment display area DDA, and is shown in... Figure 12 The comment displayed in the image DGB is the same as the comment, and the comment corresponds to at least one of the phenomena that may occur when the insertion of the insertion part 11 is difficult (such as intestinal adhesion).

[0336] As described above, according to this embodiment, it is possible to provide instructions on the operation of the insertion unit 11 corresponding to the difficulty of insertion into a patient undergoing endoscopy and the current examination status during the endoscopy. Furthermore, according to this embodiment, it is possible to provide the user with an evaluation of the surgical procedures performed during the endoscopy after the procedure is completed. Therefore, according to this embodiment, the quality of examinations during endoscopy can be improved compared to conventional methods.

[0337] Furthermore, according to this embodiment, for example, it may be configured such that, when the examination assistance information generation unit 292 detects that the patient information input into the input device 50 includes information that can identify multiple patients, it performs processing to differentiate and record the endoscopic examination information obtained during the endoscopic examination to each patient in the recording unit 293. Furthermore, according to this embodiment, for example, the system control unit 290 may be configured to output the endoscopic examination information recorded in the recording unit 293 in a patient-specific manner to an external database or cloud system of the main device 20B.

[0338] In addition, by appropriately combining the structure of the second embodiment and the structure of this embodiment, for example, the following processing can be performed to generate operation assistance information corresponding to examination status information representing the current examination status in the endoscopy, user information described in the second embodiment, and patient information described in this embodiment.

[0339] Furthermore, by appropriately combining the structures of the second embodiment and this embodiment, for example, processing can be performed to generate surgical operation evaluation information corresponding to the following information: analysis results obtained by analyzing a set of endoscopic images recorded in a single endoscopic examination, analysis results obtained by analyzing a set of insertion shape images and insertion position information recorded in the single endoscopic examination, analysis results obtained by analyzing a set of operation status information recorded in the single endoscopic examination, analysis results obtained by analyzing a set of body condition information recorded in the single endoscopic examination, and operation assistance time information and subject information recorded in the single endoscopic examination.

[0340] Furthermore, the present invention is not limited to the embodiments described above, and various modifications and applications can be made without departing from the spirit of the invention.

Claims

1. An endoscopic examination auxiliary device, characterized in that, have: The examination status information acquisition unit acquires examination status information representing the examination status of the endoscopic examination performed on the subject using the endoscope, based on insertion shape information representing the insertion shape of the insertion part of the endoscope inserted into the subject and endoscopic images obtained by the endoscope within the subject. The surgical operation evaluation unit evaluates surgical operations, including the insertion operation of the insertion unit performed by the user operating the endoscope during an endoscopic examination, based on the examination status information obtained by the examination status information acquisition unit, thereby generating surgical operation evaluation information. as well as The recording unit records the images and information obtained during the endoscopic examination of the subject using the endoscope as endoscopic examination information. The surgical operation evaluation unit determines whether the endoscopic examination information was obtained when the insertion operation at the insertion point was of normal difficulty or when the insertion operation at the insertion point was of high difficulty, and generates surgical operation evaluation information for difficulty corresponding to the determined difficulty.

2. The endoscopic examination auxiliary device according to claim 1, characterized in that, The endoscopic examination assist device also includes an examination assist information generation unit, which generates operation assist information based on the examination status information obtained by the examination status information acquisition unit to assist the insertion operation of the insertion part performed by the user operating the endoscope during the endoscopic examination.

3. The endoscopic examination auxiliary device according to claim 2, characterized in that, The recording unit also associates an endoscope image group containing multiple endoscope images obtained during the endoscopy, an insertion shape information group containing multiple insertion shape information, and an operation assistance time information group containing multiple operation assistance information, and records them as endoscopy examination information. The surgical procedure evaluation unit generates surgical procedure evaluation information based on the endoscopic examination information recorded in the recording unit.

4. The endoscopic examination auxiliary device according to claim 2, characterized in that, The inspection status information acquisition unit has the following features: The first identification processing unit is configured to perform processing to identify an examination condition corresponding to one of the endoscopic images and obtain a first identification result; The second recognition processing unit is configured to perform processing to identify the examination conditions corresponding to a plurality of endoscopic images that are sequential in time and obtain a second recognition result. as well as The inspection status detection processing unit is configured to process at least one of the first identification result, the second identification result, the insertion position information, and the insertion shape image to obtain the inspection status information. The insertion position information is information included in the insertion shape information and is information indicating the insertion position of the insertion part inserted into the subject body. The insertion shape image is information included in the insertion shape information and is information obtained by visualizing the insertion shape of the insertion part inserted into the subject body.

5. The endoscopic examination auxiliary device according to claim 2, characterized in that, The recording unit also associates an endoscope image group containing multiple endoscope images obtained during the endoscopy, an insertion shape information group containing multiple insertion shape information, and an operation assistance time information group containing multiple operation assistance information, and records them as endoscopy examination information. The operation assistance time information group includes information that determines the timing for presenting the user with the multiple guidance messages contained in each of the multiple operation assistance messages generated during the endoscopic examination, based on each guidance message. The surgical procedure evaluation unit has the following features: The first analysis and processing unit is configured to perform analysis and processing based on the endoscopic image group included in the endoscopic examination information, thereby obtaining a first analysis result including the analysis value of one or more evaluation indicators corresponding to the operation of the endoscope, wherein the operation of the endoscope is estimated to have been performed by the user in one endoscopic examination. The second analysis and processing unit is configured to perform analysis and processing based on a group of insertion shape information containing multiple insertion shape information included in the endoscopy examination information, thereby obtaining a second analysis result including the analysis value of one or more evaluation indicators corresponding to the operation of the endoscope, wherein the operation of the endoscope is estimated to have been performed by the user in one endoscopy examination. as well as The evaluation processing unit is configured to process the first analysis result, the second analysis result, and the operation assistance time information group included in the endoscopic examination information to generate the surgical operation evaluation information.

6. The endoscopic examination auxiliary device according to claim 2, characterized in that, The recording unit also associates an endoscope image group containing multiple endoscope images obtained during the endoscopy, an insertion shape information group containing multiple insertion shape information, and an operation assistance time information group containing multiple operation assistance information, and records them as endoscopy examination information. The inspection assistance information generation unit is configured to perform processing to cause the recording unit to record information representing the estimation result, which is obtained by estimating whether the operation corresponding to the guidance information contained in the operation assistance information has been completed.

7. The endoscopic examination auxiliary device according to claim 6, characterized in that, The surgical operation evaluation unit is configured to calculate the operation completion rate based on information representing the estimation result recorded in the recording unit, and generate surgical operation evaluation information including information representing the calculated operation completion rate, wherein the operation completion rate is equivalent to the proportion of operations completed in a single endoscopic examination corresponding to each prompting guidance information.

8. The endoscopic examination auxiliary device according to claim 1, characterized in that, The surgical operation evaluation unit is configured to calculate a surgical operation score, which represents a comprehensive evaluation of the surgical operation performed in one endoscopic examination, and generate surgical operation evaluation information containing the calculated surgical operation score.

9. The endoscopic examination auxiliary device according to claim 1, characterized in that, The endoscopic examination auxiliary device also features: The inspection evaluation value calculation unit is configured to calculate an inspection evaluation value for evaluating the quality of a single endoscopic examination based on the endoscopic examination information recorded in the recording unit.

10. The endoscopic examination auxiliary device according to claim 9, characterized in that, The examination evaluation value calculation unit is configured to calculate at least one of the following: ileocecal reach rate, intestinal cleansing degree, and adenoma detection rate, as the examination evaluation value.

11. The endoscopic examination auxiliary device according to claim 2, characterized in that, The endoscopic examination auxiliary device also includes a display control unit, which is configured to perform processing for generating visual information corresponding to the operation assistance information and displaying the visual information on a display device, and to perform processing for generating visual information corresponding to the surgical operation evaluation information and displaying the visual information on a display device.

12. The endoscopic examination auxiliary device according to claim 2, characterized in that, The endoscopic examination assistance device also includes a sound generating unit configured to generate and output a sound corresponding to the operation assistance information.

13. The endoscopic examination auxiliary device according to claim 2, characterized in that, The inspection assistance information generation unit is configured to process the operation assistance information to provide different modes for each operation difficulty related to the insertion operation of the insertion unit.

14. The endoscopic examination auxiliary device according to claim 4, characterized in that, The endoscopic examination assist device also includes an operation status detection and processing unit, which is configured to acquire operation status information equivalent to information indicating the current operation status of the user who is operating the endoscope. The inspection status detection and processing unit is configured to process at least one of the first identification result, the second identification result, the insertion position information, the insertion shape image, and the operation status information to obtain the inspection status information.

15. The endoscopic examination auxiliary device according to claim 14, characterized in that, The operation status detection processing unit is configured to process at least one of the following to obtain the operation status information: an image obtained by photographing the user, motion information that can determine the user's movement, gaze information that can determine the user's gaze, and sound obtained from the user.

16. The endoscopic examination auxiliary device according to claim 14, characterized in that, The examination assistance information generation unit is configured to generate operation assistance information based on the examination status information and user information including information related to the skill level of the user operating the endoscope, for providing different guidance information according to whether the user's skill level is equivalent to that of a skilled person.

17. The endoscopic examination auxiliary device according to claim 16, characterized in that, User information containing information related to the skill level of the user operating the endoscope includes at least one of the following: the number of years of experience in endoscopy performed by the user, the number of cases performed by the user, the clinical department of the user, whether the user has a certified physician qualification, and whether the user has a specialist physician qualification.

18. The endoscopic examination auxiliary device according to claim 14, characterized in that, The examination assistance information generation unit is configured to, when a determination result is obtained based on user information that the user's skill level is equivalent to that of a skilled person, perform processing to generate operation assistance information containing the following guidance information: the guidance information is used to prompt the user about the current work being performed, and the user information includes information related to the skill level of the user operating the endoscope.

19. The endoscopic examination auxiliary device according to claim 14, characterized in that, The recording unit associates an endoscope image group containing multiple endoscope images obtained during the endoscopy, an insertion shape information group containing multiple insertion shape information, an operation assistance time information group containing multiple operation assistance information, and an operation status information group containing multiple operation status information obtained based on the user's actions during a single endoscopy, and records them as endoscopy examination information.

20. The endoscopic examination auxiliary device according to claim 19, characterized in that, The endoscopic examination auxiliary device also has: The first analysis and processing unit is configured to perform analysis and processing on an endoscope image group containing multiple endoscope images obtained during the endoscope examination, which is included in the endoscope examination information, thereby obtaining a first analysis result including the analysis value of one or more evaluation indicators corresponding to the operation of the endoscope, wherein the operation of the endoscope is estimated to have been performed by the user in one endoscope examination. The second analysis and processing unit is configured to perform analysis and processing based on the insertion shape information group included in the endoscopy examination information, thereby obtaining a second analysis result including the analysis value of one or more evaluation indicators corresponding to the operation of the endoscope, wherein the operation of the endoscope is estimated to have been performed by the user in one endoscopy examination. The third analysis and processing unit is configured to perform analysis and processing based on the operation status information group included in the endoscopy examination information, thereby obtaining a third analysis result including the analysis value of one or more evaluation indicators corresponding to the user's actions in one endoscopy examination. as well as The evaluation processing unit is configured to process the first analysis result, the second analysis result, the third analysis result, and the operation assistance time information group to generate the surgical operation evaluation information.

21. The endoscopic examination auxiliary device according to claim 4, characterized in that, The endoscopic examination auxiliary device also includes a body condition detection and processing unit, which is configured to acquire body condition information equivalent to information representing the current body condition of the subject during a single endoscopic examination. The inspection status detection and processing unit is configured to process at least one of the first identification result, the second identification result, the insertion position information, the insertion shape image, and the body condition information to obtain the inspection status information.

22. The endoscopic examination auxiliary device according to claim 21, characterized in that, The physical condition detection and processing unit is configured to process the detection results obtained based on the detected pain information to obtain the physical condition information, wherein the pain information is equivalent to information indicating the pain state of the subject during one of the endoscopic examinations.

23. The endoscopic examination auxiliary device according to claim 22, characterized in that, The pain information is obtained by any one of the following methods: a manual switch pressed by the subject when feeling pain during an endoscopy, an electroencephalogram (EEG) measuring the brain waves emitted by the subject, or a heart rate monitor measuring the subject's heart rate.

24. The endoscopic examination auxiliary device according to claim 21, characterized in that, The examination assistance information generation unit is configured to generate operation assistance information based on the examination status information and subject information, including information related to the difficulty of inserting the insertion part in the subject, to provide different guidance information according to whether the insertion difficulty of the insertion part in the subject is high or low.

25. The endoscopic examination auxiliary device according to claim 24, characterized in that, The subject information includes at least one of the following: the subject's age, the subject's gender, the subject's height, the subject's weight, the subject's medical history, whether the subject has experience with endoscopy, the level of pain experienced by the subject during past endoscopy examinations, and the type of endoscope used in the subject's past endoscopy examinations.

26. The endoscopic examination auxiliary device according to claim 24, characterized in that, The recording unit is configured to associate an endoscope image group containing multiple endoscope images obtained during the endoscopy, an insertion shape information group containing multiple insertion shape information, an operation assistance time information group contained in the endoscopy examination information, the subject information, and a body condition information group including multiple body condition information obtained based on the pain state of the subject during a single endoscopy, and record them as the endoscopy examination information.

27. The endoscopic examination auxiliary device according to claim 26, characterized in that, The endoscopic examination auxiliary device also has: The first analysis and processing unit is configured to perform analysis and processing on an endoscope image group containing multiple endoscope images obtained during the endoscope examination, which is included in the endoscope examination information obtained during the endoscope examination, thereby obtaining a first analysis result including the analysis value of one or more evaluation indicators corresponding to the operation of the endoscope, wherein the operation of the endoscope is estimated to have been performed by the user in one endoscope examination. The second analysis and processing unit is configured to perform analysis and processing based on the insertion shape information group included in the endoscopy examination information, thereby obtaining a second analysis result including the analysis value of one or more evaluation indicators corresponding to the operation of the endoscope, wherein the operation of the endoscope is estimated to have been performed by the user in one endoscopy examination. The third analysis and processing unit is configured to perform analysis and processing based on the body condition information group included in the endoscopic examination information, thereby obtaining a third analysis result including the analysis value of one or more evaluation indicators corresponding to the body condition of the patient in one endoscopic examination. as well as The evaluation processing unit is configured to process the first analysis result, the second analysis result, the third analysis result, the operation assistance time information group, and the subject information to generate the surgical operation evaluation information.

28. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, performs the following processes: The examination status information acquisition unit acquires examination status information indicating the examination status of an endoscopic examination performed on the subject using the endoscope, based on insertion shape information indicating the insertion shape of the insertion portion of the endoscope into the subject and endoscopic images obtained by the endoscope within the subject. The surgical procedure evaluation unit evaluates surgical procedures, including the insertion of the insertion unit performed by the user operating the endoscope during an endoscopic examination, based on the examination status information obtained by the examination status information acquisition unit, thereby generating surgical procedure evaluation information. The recording unit will record the images and information obtained during the endoscopic examination of the subject using the endoscope as endoscopic examination information. The surgical operation evaluation unit determines whether the endoscopic examination information was obtained when the insertion operation at the insertion point was of normal difficulty or when the insertion operation at the insertion point was of high difficulty, and generates surgical operation evaluation information for difficulty corresponding to the determined difficulty.

29. A recording medium having a program recorded thereon that causes a computer to perform the following processes: Based on insertion shape information, which represents the insertion shape of the insertion part of the endoscope into the subject, and an endoscope image obtained by the endoscope taking pictures of the subject, examination status information, which represents the examination status of the subject during an endoscopic examination using the endoscope, is obtained. Based on the examination status information, the surgical operation, including the insertion operation of the insertion part performed by the user operating the endoscope during the endoscopic examination, is evaluated to generate surgical operation evaluation information. as well as Images and information obtained during endoscopic examination of the subject using the endoscope will be recorded as endoscopic examination information. Specifically, it is determined whether the endoscopic examination information was obtained when the insertion operation at the insertion site was of normal difficulty or when the insertion operation at the insertion site was of high difficulty, and surgical operation evaluation information corresponding to the determined difficulty is generated.

Images