Processor device, working method of processor device, non-transitory computer readable medium, and endoscope system

Abstract

The present application provides a processor device capable of accurately displaying information related to the current position when the distal end of an endoscope is reciprocally moved in a folded-back position in the body, a working method of the processor device, a non-transitory computer readable medium, and an endoscope system. A movement trajectory calculation section (31) calculates a movement trajectory representing the trajectory of movement of the endoscope (12) using the movement status of the endoscope (12). A first recognition processing section (32) performs recognition of the arrival of the distal end (12d) of the endoscope at the folded-back position and recognition of which of the outward journey or the return journey the movement trajectory corresponds to, based on the endoscope image obtained by the endoscope (12). A display control section (33) displays a movement trajectory display screen (50) representing the movement trajectory divided into the outward journey or the return journey on a display (15).

Description

TECHNICAL FIELD

[0001] The present application relates to a processor device for an endoscope, a method of operating a processor device, a non-transitory computer readable medium, and an endoscope system. BACKGROUND

[0002] In the current medical field, an endoscope system equipped with a light source device, an endoscope, and a processor device is becoming widespread. In the endoscope system, by displaying an endoscope image obtained by the endoscope on a display, the situation in the body can be grasped. In addition, as shown in Patent Literature 1, when an image of the current position is displayed on the display, a portion corresponding to the current position is displayed in combination with a progress bar indicating the insertion length. Thereby, it is possible to grasp which position in the body the image displayed on the display is an image of.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Laid-Open No. 2003-93326 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] In the endoscope system, when observing the digestive tract in the body such as the stomach, the large intestine, and the like, an operation of reciprocating the front end portion of the endoscope at the turning-back position is performed. For example, when observing the large intestine, the endoscope is inserted into the large intestine while being straightened or folded, and is moved to the turning-back position. When the front end portion of the endoscope reaches the turning-back position, an operation of pulling out the endoscope is performed. Therefore, at the time of insertion and at the time of pulling out of the endoscope, the insertion length of the endoscope and the shape of the large intestine are different. In this regard, the progress bar of Patent Literature 1 is difficult to accurately display the current position because it does not take into account the deformation of the large intestine at the time of insertion and at the time of pulling out.

[0008] An object of the present application is to provide a processor device, a method of operating a processor device, a non-transitory computer readable medium, and an endoscope system that accurately display information related to the current position in a case where the front end portion of the endoscope is reciprocated at the turning-back position in the body.

[0009] MEANS FOR SOLVING THE PROBLEMS

[0010] The present application provides a processor device having a processor that acquires a movement state of an endoscope moving in a body, calculates a movement trajectory representing a trajectory of the endoscope movement using the movement state of the endoscope, in a case where a distal end portion of the endoscope is reciprocated between a turnaround position in the body, performs, based on an endoscope image obtained by the endoscope, recognition of arrival of the distal end portion at the turnaround position and recognition of which one of a forward stroke or a return stroke the movement trajectory corresponds to, and displays, on a display, a movement trajectory display screen representing the movement trajectory divided into the forward stroke or the return stroke.

[0011] It is preferable that, on the movement trajectory display screen, a forward stroke display straight line representing the movement trajectory of the forward stroke and a return stroke display straight line representing the movement trajectory of the return stroke are displayed. It is preferable that the forward stroke display straight line and the return stroke display straight line are connected by a turnaround display mark representing the turnaround position. It is preferable that, on the movement trajectory display screen, an insertion length display reading representing an insertion length of the endoscope is displayed with respect to the forward stroke display straight line or the return stroke display straight line. It is preferable that, on the movement trajectory display screen, a forward stroke display curve representing the movement trajectory of the forward stroke and a return stroke display curve representing the movement trajectory of the return stroke are displayed.

[0012] It is preferable that the processor recognizes a plurality of sites including at least a first site and a second site between an insertion port of the endoscope and the turnaround position, and displays, in the movement trajectory display screen, the movement trajectory classified into a plurality of intervals determined based on the plurality of sites. It is preferable that, in the movement trajectory display screen, a message is displayed indicating an interval in which the distal end portion is currently located.

[0013] It is preferable that the processor recognizes a region of interest from the endoscope image, and displays, in the movement trajectory display screen, a position of the region of interest on the movement trajectory. It is preferable that the processor classifies a category of the region of interest, and displays, in the movement trajectory display screen, the position of the region of interest on the movement trajectory in a different display manner according to a classification result of the category. It is preferable that, in the movement trajectory display screen, category information including the category of the region of interest is displayed with respect to the movement trajectory.

[0014] It is preferable that the processor recognizes a plurality of sites including at least a first site and a second site between an insertion port of the endoscope and the turnaround position, determines a plurality of intervals based on the plurality of sites, the movement trajectory is composed of a plurality of sub-interval movement trajectories respectively set for the intervals, and the movement trajectory display screen displays a sub-interval movement trajectory display screen displaying each sub-interval movement trajectory. It is preferable that the movement state is a movement amount of the distal end portion of the endoscope, and the movement amount of the distal end portion is calculated based on at least an insertion length of the endoscope.

[0015] The present application provides an endoscope system including a processor and a display. The processor acquires a movement state of an endoscope moving in a body, calculates a movement trajectory representing a trajectory of the endoscope movement using the movement state of the endoscope, in a case where a distal end portion of the endoscope reciprocates at a turnaround position in the body, performs recognition of arrival of the distal end portion at the turnaround position and recognition of which one of a forward stroke or a return stroke the movement trajectory corresponds to based on an endoscope image obtained by the endoscope, and displays a movement trajectory display screen representing the movement trajectory divided into the forward stroke or the return stroke on the display.

[0016] The present application provides a method for operating a processor device having a processor. The processor acquires a movement state of an endoscope moving in a body, calculates a movement trajectory representing a trajectory of the endoscope movement using the movement state of the endoscope, in a case where a distal end portion of the endoscope reciprocates at a turnaround position in the body, performs recognition of arrival of the distal end portion at the turnaround position and recognition of which one of a forward stroke or a return stroke the movement trajectory corresponds to based on an endoscope image obtained by the endoscope, and displays a movement trajectory display screen representing the movement trajectory divided into the forward stroke or the return stroke on the display.

[0017] The present application provides a non-transitory computer readable medium storing a computer executable program for causing a computer to function as a processor device. The computer executable program causes the computer to perform a function of acquiring a movement state of an endoscope moving in a body, a function of calculating a movement trajectory representing a trajectory of the endoscope movement using the movement state of the endoscope, in a case where a distal end portion of the endoscope reciprocates at a turnaround position in the body, a function of performing recognition of arrival of the distal end portion at the turnaround position and recognition of which one of a forward stroke or a return stroke the movement trajectory corresponds to based on an endoscope image obtained by the endoscope, and a function of displaying a movement trajectory display screen representing the movement trajectory divided into the forward stroke or the return stroke on the display.

[0018] Effects of the Invention

[0019] According to the present application, in a case where a distal end portion of an endoscope reciprocates at a turnaround position in a body, information related to a current position can be accurately displayed. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a schematic view of an endoscope system.

[0021] Figure 2 is a block diagram representing functions of a processor device.

[0022] Figure 3 is an explanatory view for indicating a movement direction of a distal end portion of an endoscope.

[0023] Figure 4is an image diagram of an image of two endoscopic images that are different in acquisition timing.

[0024] Figure 5 is an explanatory diagram showing a marker for measuring an insertion length of an endoscope and a marker detection sensor.

[0025] Figure 6 is a schematic view of a large intestine.

[0026] Figure 7 is an image diagram of a movement locus display screen that divides a movement locus into a forward movement and a return movement and displays the movement locus with a straight line.

[0027] Figure 8 is an image diagram of a movement locus display screen that divides a movement locus into a forward movement and a return movement and displays the movement locus with a curved line.

[0028] Figure 9 is an explanatory diagram showing a movement locus ((A), (B), (C)) that is displayed by being classified into three intervals, i.e., a current position, a movement completion, and a movement scheduled movement locus.

[0029] Figure 10 is an explanatory diagram showing a movement locus ((A), (B)) that is displayed by being classified into two intervals, i.e., a current position and a movement completion.

[0030] Figure 11 is an image diagram of a movement locus display screen that divides a movement locus into a forward movement and a return movement and displays the movement locus with a straight line, and displays a position of a region of interest on the movement locus.

[0031] Figure 12 is an image diagram of a movement locus display screen that divides a movement locus into a forward movement and a return movement and displays the movement locus with a straight line and a curved line, and displays a position of a region of interest on the movement locus.

[0032] Figure 13 is an image diagram of a divided-interval movement locus display screen corresponding to three intervals.

[0033] Figure 14 is a flowchart showing a series of flows of a display method of a movement locus. DETAILED DESCRIPTION

[0034] As Figure 1As shown, the endoscope system 10 has an endoscope 12, a light source device 13, a processor device 14, a display 15, and a user interface 16. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The endoscope 12 has an insertion portion 12a inserted into a body of an observation object, an operation portion 12b provided at a base end portion of the insertion portion 12a, and a bending portion 12c and a front end portion 12d provided at a front end side of the insertion portion 12a. The bending portion 12c performs a bending action by operation of the operation portion 12b. The front end portion 12d is directed toward a desired direction by the bending action of the bending portion 12c.

[0035] Operation switches 18a, 18b for various operations by a user are provided at the operation portion 12b. Each of the operation switches 18a, 18b is assigned an operation command for performing various operations via the user interface 16. In addition, a zoom operation portion 19 for magnification or reduction of an image of the observation object is provided at the operation portion 12b.

[0036] The light source device 13 generates illumination light for illuminating the observation object and supplies the generated illumination light to the endoscope 12. The endoscope 12 irradiates the illumination light from the light source device 13 toward the observation object and photographs the observation object illuminated by the illumination light. The endoscope 12 transmits an endoscope image obtained by photographing the observation object to the processor device 14.

[0037] The processor device 14 is electrically connected to the display 15 and the user interface 16. In the processor device 14, various image processing such as tone adjustment processing or structure emphasis processing is performed on the image from the endoscope 12. The image on which the various image processing is performed is transmitted to the display 15. The display 15 outputs an image of the observation object, information accompanying the image of the observation object, and the like. The user interface 16 has a keyboard, a mouse, a touch panel, a microphone, and the like, and has a function of accepting an input operation of a function setting or the like.

[0038] Further, an extension processor device (not shown) for performing AI processing or the like for detecting a lesion portion by AI (Artificial Intelligence) can be connected to the processor device 14. In this case, in order to display an image or the like processed by the extension processor device, an extension display (not shown) separate from the display 15 can be connected to the extension processor device. Therefore, the "processor device" of the present application corresponds to the extension processor device in addition to the processor device 14. Likewise, the "display" of the present application corresponds to the extension display in addition to the display 15.

[0039] As Figure 2As shown, the processor device 14 is provided with a movement state acquisition section 30, a movement trajectory calculation section 31, a first recognition processing section 32, a display control section 33, a second recognition processing section 34, and a category classification section 35. In the processor device 14, a processor device program for executing various processes is loaded in a program memory (not shown). A central control section (not shown) constituted by a processor is provided in the processor device 14. The central control section executes the processor device program in the program memory, whereby the functions of the above-described movement state acquisition section 30, movement trajectory calculation section 31, first recognition processing section 32, display control section 33, second recognition processing section 34, and category classification section 35 are realized.

[0040] The movement state acquisition section 30 acquires the movement state of the endoscope moving in the body. Specifically, the movement state acquisition section 30 preferably acquires the movement state on the basis of the movement amount of the distal end portion 12d of the endoscope. As the movement amount of the distal end portion 12d, as shown in Figure 3 as shown, the movement amount of the distal end portion 12d in the up-down-left-right directions (X-axis direction or Y-axis direction) with respect to the movement direction (Z-axis direction) of the distal end portion 12d, and the rotation amount (θ (when the Z-axis is the center of rotation)) of the distal end portion 12d (parameters of X, Y, Z, θ) are preferably used.

[0041] As the movement amount of the distal end portion 12d, for example, as shown in Figure 4 as shown, the movement state acquisition section 30 preferably calculates from at least two endoscope images (Nth endoscope image P(N), (N+1)th endoscope image P(N+1)) different in the timing of acquisition of the endoscope. When the endoscope images P(N) and P(N+1) are compared, the region X moves by a certain distance. In the calculation of the movement amount of the distal end portion 12d using the endoscope images, the alignment processing of the endoscope image P(N) and the endoscope image P(N+1) is preferably used.

[0042] In the alignment processing. The parallel movement processing of moving the endoscope image P(N) or the endoscope image P(N+1) in parallel, and the enlargement / reduction processing of enlarging or reducing the endoscope image P(N) or the endoscope image P(N+1) are included. By the parallel movement processing, it is possible to calculate the movement amount of the distal end portion 12d in the X-axis direction or the Y-axis direction, and the rotation amount θ of the distal end portion 12d with the Z-axis as the center of rotation. In addition, by the enlargement / reduction processing, it is possible to calculate the movement amount of the distal end portion 12d at the time of insertion or at the time of extraction with respect to the movement direction Z.

[0043] In addition, the movement state acquisition section 30 can also acquire the movement amount of the distal end portion 12d on the basis of the insertion length of the insertion portion 12a inserted into the body. In this case, as shown in Figure 5As shown, the insertion portion 12a and the insertion port of the endoscope 12 (the anus 44 (refer to Figure 6 )) are provided with a marker 40 such as a magnet at constant intervals or the like, and a marker detection sensor 41 that detects the marker 40 is provided. The marker detection sensor 41 detects the marker 40 by movement (movement in the Z-axis direction) of the insertion portion 12a, and the insertion length of the insertion portion 12a is calculated on the basis of the detection result thereof. Further, the amount of movement in the X-axis direction or the Y-axis direction of the distal end portion 12d is preferably calculated using a movement amount measuring sensor 42 provided at the operation portion 12b. The movement amount measuring sensor 42 calculates the amount of movement in the X-axis direction or the Y-axis direction of the distal end portion 12d in accordance with the amount of operation of the bending portion 12c. Further, the amount of movement of the distal end portion 12d of the endoscope can also be measured using a magnetic sensor such as a colonoscope position detecting unit.

[0044] The movement locus calculating portion 31 calculates a movement locus that represents the locus of movement of the endoscope 12 using the movement state of the endoscope 12. Specifically, the movement locus can be calculated by accumulating the movement state of the endoscope 12. In the case where the movement state of the endoscope 12 is represented by the amount of movement (X, Y, Z, θ) of the distal end portion 12d, the movement locus can be represented by a three-dimensional curve by using all four parameters (X, Y, Z, θ). Alternatively, the movement locus can be represented as a two-dimensional curve by performing a projection process on the three-dimensional curve (refer to Figure 8 ). Further, the movement locus can be represented by a straight line by using only Z among the four parameters (X, Y, Z, θ) (refer to Figure 7 ).

[0045] In the case where the distal end portion 12d of the endoscope is reciprocated at a turning-back position in the body, the first recognition processing portion 32 performs recognition of the arrival of the distal end portion 12d at the turning-back position and recognition of which one of the movement locus coincides with the outward journey or the return journey on the basis of the endoscope image obtained by the endoscope 12. Specifically, as shown in Figure 6 , in the case where the body is the large intestine, the turning-back position corresponds to the ileocecal junction 43. The first recognition processing portion 32 performs processing to recognize the ileocecal junction 43 on the basis of the endoscope image. Therefore, until the first recognition processing portion 32 recognizes the ileocecal junction 43 (before the ileocecal junction 43 is recognized), the first recognition processing portion 32 recognizes that the movement locus coincides with the outward journey. On the other hand, after the ileocecal junction 43 is recognized by the first recognition processing portion 32 (after the ileocecal junction 43 is recognized), the first recognition processing portion 32 recognizes that the movement locus coincides with the return journey. Further, the first recognition processing portion 32 preferably uses a learning completed model that has been subjected to machine learning using the endoscope image as an input image and the correct answer (the site, the position, and the like).

[0046] Further, the first recognition processing section 32 recognizes a plurality of sites including at least the first site or the second site between the insertion port and the turning-back position of the endoscope 12. Specifically, in the case where the body lumen is the large intestine, the insertion port of the endoscope 12 corresponds to the anus 44, and the turning-back position corresponds to the ileocecal junction 43. Further, the first site on the anus side corresponds to the splenic flexure 45, and the second site on the ileocecal junction side corresponds to the hepatic flexure 46. By thus recognizing the plurality of sites, a plurality of intervals determined in accordance with the plurality of sites can be recognized. If the body lumen is the large intestine, by recognizing the splenic flexure 45 and the hepatic flexure 46, three intervals of the descending colon 47, the transverse colon 48, and the ascending colon 49 can be recognized.

[0047] In the case where neither the splenic flexure 45 nor the hepatic flexure 46 is recognized, the interval of the descending colon 47 where the movement locus is on the way to the insertion is recognized. Further, in the case where the splenic flexure 45 is recognized but the hepatic flexure 46 is not recognized, the interval of the transverse colon 48 where the movement locus is on the way to the insertion is recognized. Further, in the case where the hepatic flexure 46 is recognized, the interval of the ascending colon 49 where the movement locus is on the way to the insertion or on the way to the extraction is recognized. Further, in the case where the hepatic flexure 46 is recognized and then the hepatic flexure 46 is recognized again, the interval of the transverse colon 48 where the movement locus is on the way to the extraction is recognized. Further, in the case where the hepatic flexure 46 is recognized and then the splenic flexure 45 is recognized, the interval of the descending colon 47 where the movement locus is on the way to the extraction is recognized.

[0048] The display control section 33 displays, on the display 15, a movement locus display screen which represents the movement locus divided into the way to the insertion or the way to the extraction. Thus, since the insertion (the way to the insertion) or the extraction (the way to the extraction) of the endoscope 12 is divided and the locus of the distal end portion 12d is visible, the influence of the shape change in the body lumen can be reduced. In the display control section 33, specifically, in the case where the movement locus is represented by a straight line, as shown in FIG. 6, a straight line for the way to the insertion 51 and a straight line for the way to the extraction 52 are displayed on the movement locus display screen 50. Figure 7

[0049] Further, the straight line for the way to the insertion 51 and the straight line for the way to the extraction 52 are connected by a turning-back display mark 53 which represents the turning-back position. The turning-back display mark 53 is provided as a semicircle, but can be other shapes (for example, a straight line (see FIG. 7 (A) and the like)). Figure 9 (A) and the like)). Further, the solid line portion LY of the straight line for the way to the insertion 51, the straight line for the way to the extraction 52, and the turning-back display mark 53 represents the position where the distal end portion 12d is currently located, and the broken line portion LX represents the portion where the movement of the distal end portion 12d is completed or the predetermined portion to be moved.

[0050] ​In addition, on the movement locus display screen 50, an insertion length display reading 54 indicating the insertion length of the endoscope 12 can be displayed for the straight line 51 for the forward movement display or the straight line 52 for the return movement display. It is preferable that the insertion length display reading 54 be arranged to the side of the straight line 51 for the forward movement display or the straight line 52 for the return movement display, and the display (scroll display) of the insertion length display reading 54 be switched so as to fit the movement of the distal end portion 12d, and become a reading corresponding to the actual insertion length. For example, in the case of Figure 7 , since the distal end portion 12d is in the vicinity of the turning-back position, the insertion length display reading 54 indicates the insertion length in the vicinity of the turning-back position (100 cm to 120 cm). On the other hand, if it is the descending colon section immediately after insertion, the insertion length display reading 54 indicates the insertion length (0 to 30 cm or so).

[0051] In addition, in the case of displaying the movement locus with a two-dimensional curve, as shown in Figure 8 , on the movement locus display screen 50, a forward movement display curve 56 indicating the movement locus of the forward movement and a return movement display curve 57 indicating the movement locus of the return movement are displayed. Further, the solid line portion LM of the forward movement display curve 56 and the return movement display curve 57 indicates the position where the distal end portion 12d is currently located, and the broken line portion LN indicates the portion where the movement of the distal end portion 12d is completed or the predetermined portion to be moved.

[0052] In addition, in the movement locus display screen 50, the movement locus can be displayed by classifying into a plurality of sections determined in accordance with a plurality of portions. If the inside of the body is the large intestine, it is preferable to display by classifying into three sections of the descending colon, the transverse colon, and the ascending colon determined in accordance with the splenic flexure (first portion) and the hepatic flexure (second portion). Specifically, in the case of indicating the movement locus with the straight line 51 for the forward movement display or the straight line 52 for the return movement display, as shown in Figure 9 (A), using the first portion display marker 58a, 58b indicating the splenic flexure 45 and the second portion display marker 59a, 59b indicating the hepatic flexure 46, the section 47a of the descending colon, the section 48a of the transverse colon, the section 49a of the ascending colon in the forward movement and the section 47b of the descending colon, the section 48b of the transverse colon, the section 49b of the ascending colon in the return movement are indicated. Further, the above sections 47a, 47b of the descending colon, the sections 48a, 48b of the transverse colon, the sections 49a, 49b of the ascending colon are preferably displayed by changing the colors respectively. Further, since the shape of the large intestine differs between the forward movement and the return movement, the first portion display marker 58a and the second portion display marker 59a on the forward movement side and the first portion display marker 58b and the second portion display marker 59b on the return movement side are displayed slightly shifted on the straight line.

[0053] In addition, as shown inFigure 9 (B) shown, preferably, instead of using the first site display markers 58a, 58b and the second site display markers 59a, 59b indicating the liver curvature 46, the cutouts 60a, 60b, 60c, 60d are provided between the sections to understand the descending colon section 47a, the transverse colon section 48a, the ascending colon section 49a in the outgoing route and the descending colon section 47b, the transverse colon section 48b, the ascending colon section 49b in the returning route.

[0054] In addition, as shown in Figure 9 (C), it is also possible to display both the first site display markers 58a, 58b and the second site display markers 59a, 59b and the cutouts 60a, 60b, 60c, 60d to understand the descending colon section 47a, the transverse colon section 48a, the ascending colon section 49a in the outgoing route and the descending colon section 47b, the transverse colon section 48b, the ascending colon section 49b in the returning route.

[0055] Further, in Figure 7 or Figure 8 , the solid line portions LY, LM indicate the current position of the distal end portion 12d of the endoscope, and the broken line portions LX, LN indicate the portion where the movement of the distal end portion 12d is completed or the predetermined portion to be moved, but other methods can be used. For example, as shown in Figure 10 (A), Figure 10 (B), in the movement trajectory display screen 50, only the current position of the distal end portion 12d and the portion where the movement is completed are indicated by the solid line LP, and the predetermined portion to be moved is not displayed. In this case, in order to display which section the distal end portion 12d is currently located in, at least one of the first site display markers 58a, 58b and the second site display markers 59a, 59b and the cutouts 60a, 60b, 60c, 60d can be used to understand the sections 47a, 47b, 48a, 48b, 49a, 49b, and the messages Ml, M2 can be used to display the section where the distal end portion 12d is currently located.

[0056] For example, Figure 10 (A) indicates the movement trajectory at time t, and the display of the message Ml indicates that the current position is in the "transverse colon". In addition, Figure 10 (B) indicates the movement trajectory at time t+α later than time t, and the display of the message M2 indicates that the current position is in the "ascending colon". Further, the current position of the distal end portion 12d of the endoscope is displayed by the distal end of the movement trajectory, i.e., the distal end portion position 61. The distal end portion position 61 is displayed with a portion cut off when the distal end portion 12d is located in the middle of each section, and is displayed without a cut when the distal end portion 12d is located at the boundary of each section.

[0057] The second recognition processing section 34 recognizes a region of interest from the endoscope image. Further, the region of interest is, for example, a region including a lesion portion represented by cancer, a treated scar, a surgical scar, a bleeding portion, a benign tumor portion, an inflammation portion (including a portion having a change such as bleeding or atrophy in addition to so-called inflammation), a burn scar caused by heating or a marked portion marked based on coloring by a coloring agent, a fluorescent agent, or the like, or a biopsy performed portion on which a biopsy (so-called biopsy) is performed. That is, a region including a lesion, a region in which a lesion can occur, a region on which a certain treatment such as biopsy is performed, a treatment instrument such as a clip or a forceps, or a dark portion region (a region in which observation light is difficult to reach because it is the back of a wrinkle (crease), a lumen inner side) or the like in which detailed observation is required regardless of whether a lesion occurs or not can become a region of interest. The second recognition processing section 34 detects a region including at least any one of a lesion portion, a treated scar, a surgical scar, a bleeding portion, a benign tumor portion, an inflammation portion, a marked portion, or a biopsy performed portion as a region of interest. Further, the second recognition processing section 34, like the first recognition processing section 32, preferably uses a learned model that has been subjected to machine learning using an endoscope image as an input image and a correct answer (presence or absence of a lesion, or the like).

[0058] The category classification section 35 classifies the category of the region of interest recognized by the second recognition processing section 34. The category to be classified is any one or a combination of a photographing portion, presence or absence or a kind of a lesion as one of the regions of interest, a use state of a treatment instrument, a scattering state of a coloring agent, or the like. For example, as the category of the region of interest, there are "Hyper Plastie" or "Neo Plastie" or the like.

[0059] As described above, in a case where the region of interest is recognized by the second recognition processing section 34, in the movement trajectory display screen 50, it is preferable to display the position of the region of interest on the movement trajectory. Specifically, in a case where the movement trajectory is expressed by both a straight line and a curve, as illustrated in FIG. 6, the position of the region of interest is displayed as a region of interest detection point BP with respect to a straight line 51 for the outward movement display, a straight line 52 for the return movement display, or a mark 53 for the return display. In addition, the position of the region of interest is displayed as a region of interest detection point BP with respect to a curve 56 for the outward movement display or a curve 57 for the return movement display. Figure 11

[0060] In addition, in a case where the category of the region of interest is classified, in the movement trajectory display screen 50, the position of the region of interest can also be displayed on the movement trajectory in a different display manner according to the classification result of the category. Specifically, in a case where the movement trajectory is expressed by both a straight line and a curve, as illustrated in FIG. 7, the position of the region of interest is displayed as a region of interest detection point BP with respect to a straight line 51 for the outward movement display, a straight line 52 for the return movement display, or a mark 53 for the return display. In addition, the position of the region of interest is displayed as a region of interest detection point BP with respect to a curve 56 for the outward movement display or a curve 57 for the return movement display. Figure 12 ​As shown, for the straight lines 51, 52, the attention region detection point BPA of category A is displayed at the position where the attention region of category A is recognized, and the attention region detection point BPB of category B (different from category A) is displayed at the position where the attention region of category B is recognized. For example, the color of the attention region detection point BPA is preferably set to yellow, and the color of the attention region detection point BPB is set to green. Further, category A is, for example, "Neo Plastic", and category B is, for example, "Hyper Plastic". Furthermore, for the curved moving track, in the case where the categories are different, the attention region detection points BPA, BPB are displayed using different display modes.

[0061] Further, in the case where the categories of the attention regions are classified, in the moving track display screen 50, for the moving track, the category information including the category of the attention region can also be displayed. Specifically, in the straight lines 51, 52, the attention region detection point BPA of category A is displayed at the position where the attention region of category A is recognized, and the category information CA that the attention region is category A and the size including the attention region is displayed to the side of the attention region detection point BPA. Further, the attention region detection point BPB of category B is displayed at the position where the attention region of category B is recognized, and the category information CB that the attention region is category B and the size including the attention region is displayed to the side of the attention region detection point BPB. Furthermore, the images GA, GB of the attention regions can also be included in the category information CA, CB and displayed.

[0062] Further, in the moving track display screen 50, all sections determined by the plurality of sites (for example, if the body is the large intestine, all sections of the descending colon, transverse colon, and ascending colon) are displayed, but in the case where the examination result is displayed after the examination ends in the case where the report is made using the endoscope image, the moving track can also be constituted by a plurality of sub-section moving tracks provided in each sub-section, and in the moving track display screen, a sub-section moving track display screen displaying each sub-section moving track is displayed.

[0063] For example, in the case where the body is the large intestine and the splenic flexure (first site) and the hepatic flexure (second site) are recognized, as shown in FIG. 6A, the attention region detection point BPA of category A is displayed at the position where the attention region of category A is recognized, and the attention region detection point BPB of category B is displayed at the position where the attention region of category B is recognized. Figure 13As shown, the moving locus of the descending colon section is displayed by the section-by-section moving locus display screen 70, the moving locus of the transverse colon section is displayed by the section-by-section moving locus display screen 71, and the moving locus of the ascending colon section is displayed by the section-by-section moving locus display screen 72. By displaying in three sections in this way, the amount of information of the moving locus or the region of interest for each display screen is reduced, and thus, the visibility of the user is improved. Further, in a case where the information of all sections is displayed by one screen, when the number of regions of interest is large, many marks (region of interest detection points BPA, etc.) are sometimes attached to one moving locus, and the user can have difficulty in seeing.

[0064] The section-by-section moving locus display screen 70 has a straight line section 70a in which the moving locus is represented by a straight line and a curved line section 70b in which the moving locus is represented by a curved line. In the straight line section 70a, only the portion in which the region of interest is recognized in the moving locus is preferably displayed. Further, in the curved line section 70b, the descending colon section is represented by a solid line portion LS, and the other sections are represented by a broken line portion LT. As for the other, the display method of the region of interest, etc. is the same as in the above case (refer to Figure 11 、 Figure 12 ).

[0065] Further, as for the section-by-section moving locus display screens 71 and 72, the same as the section-by-section moving locus display screen 70, the straight line sections 71a and 72a and the curved line sections 71b and 72b are respectively provided, and further, the moving locus or the region of interest, etc. is displayed by the same method as the section-by-section moving locus display screen 70.

[0066] Next, the display of the moving locus is described along the flowchart of Figure 14 The moving condition acquisition section 30 provided in the processor device 14 acquires the moving condition of the endoscope moving in the body. The moving locus calculation section 31 calculates the moving locus representing the locus of the movement of the endoscope using the moving condition of the endoscope. The first recognition processing section 32 recognizes whether or not the distal end section 12d reaches the turnaround position based on the endoscope image obtained by the endoscope 12.

[0067] In a case where it is recognized that the distal end section 12d does not reach the turnaround position, the first recognition processing section 32 recognizes that the moving locus corresponds to the outbound. On the other hand, in a case where it is recognized that the distal end section 12d reaches the turnaround position, the first recognition processing section 32 recognizes that the moving locus corresponds to the inbound. The display control section 33 displays the moving locus display screen 50 in which the moving locus is represented by being divided into the outbound or the inbound on the display 15.

[0068] In the above-described embodiments, the hardware structure of the processing units that execute various processes of the movement situation acquisition unit 30, the movement trajectory calculation unit 31, the first recognition processing unit 32, the display control unit 33, the second recognition processing unit 34, the category classification unit 35, and the like is various processors as described below. Among the various processors, there are a general-purpose processor, that is, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an FPGA (Field Programmable Gate Array), and the like, which execute software (programs) as various processing units to function, a programmable logic device (PLD), which is a processor whose circuit structure can be changed after manufacturing, and a dedicated circuit, which is a processor having a circuit structure designed specifically for executing various processes.

[0069] One processing unit can be constituted by one of these various processors, or can be constituted by a combination of two or more processors of the same kind or different kinds (for example, a combination of a plurality of FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU, and the like). In addition, a plurality of processing units can be constituted by one processor. As an example in which a plurality of processing units is constituted by one processor, first, there is a manner in which one processor is constituted by a combination of one or more CPUs and software, which functions as a plurality of processing units, using a computer such as a client or a server as a representative. Next, there is a manner in which one processor is used, which functions as a plurality of processing units, using a system on chip (SoC) or the like as a representative. In this way, the various processing units are constituted by one or more of the above-described various processors as a hardware structure.

[0070] Furthermore, more specifically, the hardware structure of these various processors is a circuitry in which circuit elements such as semiconductor elements are combined. In addition, the hardware structure of the storage unit is a storage device such as an HDD (hard disc drive) or an SSD (solid state drive).

[0071] Symbol Explanation

[0072] 10 endoscope system

[0073] 12 endoscope

[0074] 12a insertion section

[0075] 12b operation section

[0076] 12c curved portion

[0077] 12d front end portion

[0078] 13 light source device

[0079] 14 processor device

[0080] 15 display

[0081] 16 user interface

[0082] 18a, 18b operation switch

[0083] 19 zoom operation portion

[0084] 30 movement state acquisition portion

[0085] 31 movement trajectory calculation portion

[0086] 32 first recognition processing portion

[0087] 33 display control portion

[0088] 34 second recognition processing portion

[0089] 35 category classification portion

[0090] 40 marker

[0091] 41 marker detection sensor

[0092] 42 movement amount measurement sensor

[0093] 43 ileocecal portion

[0094] 44 anus

[0095] 45 splenic flexure

[0096] 46 hepatic flexure

[0097] 47 descending colon

[0098] 47a interval of descending colon in outgoing course

[0099] 47b interval of descending colon in return course

[0100] 48 transverse colon

[0101] 48a interval of transverse colon in outgoing course

[0102] 48b interval of transverse colon in return course

[0103] 49 ascending colon

[0104] 49a interval of ascending colon in outgoing course

[0105] 49b section of ascending colon in backhauled

[0106] 50 moving locus display screen

[0107] 51 straight line for forward display

[0108] 52 straight line for backhaul display

[0109] 53 mark for turn-back display

[0110] 54 reading for insertion length display

[0111] 56 curve for forward display

[0112] 57 curve for backhaul display

[0113] 58a, 58b mark for first site display

[0114] 59a, 59b mark for second site display

[0115] 60a, 60b, 60c, 60d cut slit

[0116] 61 tip position

[0117] 70, 71, 72 moving locus display screen for sub-section

[0118] 70a, 71a, 72a linear region

[0119] 70b, 71b, 72b curved region

[0120] LY, LM, LS, LP solid line portion

[0121] LX, LN, LT dotted line portion

[0122] M1, M2 message

[0123] BPA, BPB attention region detection point

[0124] CA, CB category information

[0125] P(N), P(N+1) endoscope image

[0126] GA, GB image of attention region

Claims

1. A processor device having a processor, wherein the processor, acquires a movement state of an endoscope moving in a body, calculates a movement trajectory representing a trajectory of the endoscope movement using the movement state of the endoscope, in a case where a distal end portion of the endoscope is reciprocally moved to and from a turnaround position in the body, performs, using a learning completion model that has been machine-learned using endoscope images, recognition of the distal end portion reaching the turnaround position and recognition of which one of a forward movement or a return movement the movement trajectory corresponds to, in the recognition of which one of the forward movement or the return movement, a plurality of sites including at least a first site and a second site and a plurality of intervals determined from the plurality of sites are recognized between an insertion port of the endoscope and the turnaround position, and the recognition of which one of the forward movement or the return movement the movement trajectory corresponds to is performed based on which one of the plurality of intervals the distal end portion is located and the sites recognized in the plurality of sites, displays, on a display, a movement trajectory display screen representing the movement trajectory divided into the forward movement or the return movement.

2. The processor device according to claim 1, wherein a forward movement display straight line representing a movement trajectory of the forward movement and a return movement display straight line representing a movement trajectory of the return movement are displayed on the movement trajectory display screen.

3. The processor device according to claim 2, wherein the forward movement display straight line and the return movement display straight line are connected by a turnaround display mark representing the turnaround position.

4. The processor device according to claim 2, wherein an insertion length display reading representing an insertion length of the endoscope is displayed on the movement trajectory display screen with respect to the forward movement display straight line or the return movement display straight line.

5. The processor device according to claim 3, wherein an insertion length display reading representing an insertion length of the endoscope is displayed on the movement trajectory display screen with respect to the forward movement display straight line or the return movement display straight line.

6. The processor device according to claim 1, wherein a forward movement display curve representing a movement trajectory of the forward movement and a return movement display curve representing a movement trajectory of the return movement are displayed on the movement trajectory display screen.

7. The processor device according to any one of claims 1 to 6, wherein the movement trajectory is displayed classified into the plurality of intervals on the movement trajectory display screen.

8. The processor device according to claim 7, wherein a message is displayed on the movement trajectory display screen indicating which interval the distal end portion is currently located in.

9. The processor device according to any one of claims 1 to 6, wherein the processor recognizes a region of interest from the endoscope images, a position of the region of interest is displayed on the movement trajectory display screen on the movement trajectory.

10. The processor device according to claim 7, wherein the processor recognizes a region of interest from the endoscope images, In the movement locus display screen, the position of the region of interest is displayed on the movement locus.

11. The processor apparatus according to claim 8, wherein the processor identifies a region of interest from the endoscope image, In the movement locus display screen, the position of the region of interest is displayed on the movement locus.

12. The processor apparatus according to claim 9, wherein the processor classifies a category of the region of interest, In the movement locus display screen, the position of the region of interest is displayed on the movement locus in a different display manner according to the classification result of the category.

13. The processor apparatus according to claim 12, wherein In the movement locus display screen, category information including the category of the region of interest is displayed for the movement locus.

14. The processor apparatus according to any one of claims 1 to 6, wherein the movement locus is constituted by a plurality of sub-interval movement loci respectively provided for the intervals, the movement locus display screen respectively displays sub-interval movement locus display screens that display the respective sub-interval movement loci.

15. The processor apparatus according to any one of claims 1 to 6, wherein the movement condition is a movement amount of a distal end portion of the endoscope, the movement amount of the distal end portion is calculated based on at least an insertion length of the endoscope.

16. An endoscope system comprising a processor and a display, wherein the processor, acquires a movement condition of an endoscope that moves in a body, calculates a movement locus that represents a locus of movement of the endoscope using the movement condition of the endoscope, in a case where a distal end portion of the endoscope is reciprocally moved to and from a turnaround position in the body, performs, using a learning-completed model that has been machine-learned using an endoscope image obtained by the endoscope, recognition of arrival of the distal end portion at the turnaround position and recognition of which one of a forward movement or a return movement the movement locus corresponds to, in the recognition of which one of the forward movement or the return movement the movement locus corresponds to, a plurality of sites including at least a first site and a second site and a plurality of intervals determined based on the plurality of sites are recognized between an insertion port of the endoscope and the turnaround position, and the recognition of which one of the forward movement or the return movement the movement locus corresponds to is performed based on which one of the plurality of intervals the distal end portion is located in and the sites recognized in the plurality of sites, the display displays a movement locus display screen that represents the movement locus divided into the forward movement or the return movement.

17. A non-transitory computer-readable medium storing a computer executable program for causing a computer to function as a processor apparatus, the computer executable program causing the computer to perform: a function of acquiring a movement condition of an endoscope that moves in a body; a function of calculating a movement locus that represents a locus of movement of the endoscope using the movement condition of the endoscope; In a case where the distal end portion of the endoscope is reciprocally moved between a folded-back position of the distal end portion in the body and a position other than the folded-back position, based on an endoscope image obtained by the endoscope, using a learning-completed model that has been subjected to machine learning using the endoscope image, a function of performing recognition of the distal end portion reaching the folded-back position and recognition of which one of a going course or a returning course the movement trajectory corresponds to, in the recognition of which one of the going course or the returning course, a plurality of sites including at least a first site and a second site and a plurality of intervals determined based on the plurality of sites are recognized between a site of insertion of the endoscope and the folded-back position, and the recognition of which one of the going course or the returning course the movement trajectory corresponds to is performed based on which one of the plurality of intervals the distal end portion is positioned and the site recognized among the plurality of sites; and a function of displaying, on a display, a movement trajectory display screen in which the movement trajectory is divided into the going course or the returning course and represented.

Images