Processor device, operating method for processor device, program for processor device, and endoscope system

Abstract

The present invention provides a processor device, an operation method for a processor device, a program for a processor device, and an endoscope system with which it is possible to accurately display information relating to the current position in a case in which the distal end part of an endoscope reciprocatingly moves at a turn-back position in the body. A movement trajectory calculation unit 31 calculates, using the movement status of an endoscope 12, a movement trajectory indicating the trajectory of movement of the endoscope 12. A first recognition processing unit 32 performs a recognition of the distal end part 12d of the endoscope reaching a turn-back position, and a recognition of whether the movement trajectory corresponds to a forward movement or return movement, from an endoscopic image obtained by the endoscope 12. A display control unit 33 displays, on a display 15, a movement trajectory display screen 50 in which the movement trajectory is represented differently between forward movement and backward movement.

Description

[Technical Field]

[0001] The present invention relates to a processor device, an operation method for the processor device, a program for the processor device, and an endoscope system when the tip of an endoscope moves inside a body. [Background technology]

[0002] In the current medical field, an endoscope system including a light source device, an endoscope, and a processor device is used. M are becoming widespread. In an endoscope system, an endoscopic image obtained by an endoscope is displayed on a display, allowing the user to grasp the state inside the body. Furthermore, as shown in Patent Document 1, when an image of the current position is displayed on the display, a progress bar indicating the insertion length is also displayed along with the portion corresponding to the current position. This allows the user to grasp the position inside the body of the image currently displayed on the display. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-93326 Summary of the Invention [Problem to be solved by the invention]

[0004] In an endoscopic system, when observing the digestive tract inside the body, such as the stomach or large intestine, the tip of the endoscope is moved back and forth at a turning position. For example, when observing the large intestine, the endoscope is inserted into the large intestine while straightening or folding it, and then moved to the turning position. When the tip of the endoscope reaches the turning position, the endoscope is removed. Therefore, the insertion length of the endoscope and the shape of the large intestine differ when the endoscope is inserted and when it is removed. In this regard, the progress bar in Patent Document 1 does not take into account deformation of the large intestine during insertion and removal, making it difficult to accurately display the current position.

[0005] The present invention aims to provide a processor device, an operating method for the processor device, a program for the processor device, and an endoscopic system that can accurately display information regarding the current position when the tip of the endoscope moves back and forth at a turning point inside the body. [Means for solving the problem]

[0006] The present invention relates to a processor device having a processor, in which the processor acquires the movement status of an endoscope moving within the body, and uses the movement status of the endoscope to calculate a movement trajectory indicating the path along which the endoscope moved.When the tip of the endoscope moves back and forth at a turning point within the body, the processor recognizes from the endoscopic image obtained by the endoscope that the tip has reached the turning point and whether the movement trajectory corresponds to an outward or return journey, and displays a movement trajectory display screen on the display that shows the movement trajectory divided into an outward or return journey.

[0007] The movement trajectory display screen preferably displays an outgoing movement display line representing the outgoing movement trajectory, and a return movement display line representing the return movement trajectory. The outgoing movement display line and the return movement display line are preferably connected by a turn-back display mark representing a turn-back position. The movement trajectory display screen preferably displays an insertion length display scale representing the insertion length of the endoscope relative to the outgoing movement display line or the return movement display line. The movement trajectory display screen preferably displays an outgoing movement display curve representing the outgoing movement trajectory, and a return movement display curve representing the return movement trajectory.

[0008] The processor preferably recognizes a plurality of regions, including at least a first region and a second region, between the insertion port of the endoscope and the turning position, and classifies and displays the movement trajectory into a plurality of sections defined by the plurality of regions on the movement trajectory display screen.The movement trajectory display screen preferably displays a message indicating the section in which the tip is currently located.

[0009] It is preferable that the processor recognizes a region of interest from the endoscopic image and displays the position of the region of interest on the movement trajectory on the movement trajectory display screen.It is preferable that the processor classifies the region of interest into categories and displays the position of the region of interest on the movement trajectory on the movement trajectory display screen in a display manner that differs depending on the category classification result.It is preferable that the movement trajectory display screen displays category information including the category of the region of interest for the movement trajectory.

[0010] The processor preferably recognizes a plurality of regions including at least a first region and a second region between the insertion port of the endoscope and the turning position, a plurality of sections are determined from the plurality of regions, the movement trajectory is composed of a plurality of section-specific movement trajectories provided for each section, and the movement trajectory display screen preferably displays a section-specific movement trajectory display screen that displays each section-specific movement trajectory. The movement status is the amount of movement of the tip of the endoscope, and the amount of movement of the tip is preferably calculated based on at least the insertion length of the endoscope.

[0011] The present invention provides an endoscopic system equipped with a processor and a display, in which the processor acquires the movement status of an endoscope moving within the body, and uses the movement status of the endoscope to calculate a movement trajectory showing the path along which the endoscope moved.When the tip of the endoscope moves back and forth at a turning point within the body, the processor recognizes from the endoscopic image obtained by the endoscope that the tip has reached the turning point and whether the movement trajectory corresponds to an outward or return journey, and displays a movement trajectory display screen on the display that shows the movement trajectory divided into an outward or return journey.

[0012] The present invention relates to a method for operating a processor device having a processor, in which the processor acquires the movement status of an endoscope moving within the body, calculates a movement trajectory showing the path along which the endoscope moved using the movement status of the endoscope, and, when the tip of the endoscope moves back and forth at a turning position within the body, recognizes from the endoscopic image obtained by the endoscope that the tip has reached the turning position and whether the movement trajectory corresponds to an outward or return journey, and displays a movement trajectory display screen on the display that shows the movement trajectory divided into an outward or return journey.

[0013] The program for the processor device of the present invention causes a computer to perform the following functions: acquire the movement status of an endoscope moving within the body; calculate a movement trajectory indicating the path taken by the endoscope using the movement status of the endoscope; recognize that the tip of the endoscope has reached a turning position from an endoscopic image obtained by the endoscope when the tip of the endoscope moves back and forth at a turning position within the body, and recognize whether the movement trajectory corresponds to an outward or return journey; and display a movement trajectory display screen on a display that shows the movement trajectory divided into an outward or return journey. [Effects of the Invention]

[0014] According to the present invention, when the distal end of the endoscope moves back and forth at a turning point inside the body, information about the current position can be accurately displayed. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram of an endoscope system. [Figure 2] FIG. 2 is a functional block diagram of a processor device. [Figure 3] FIG. 2 is an explanatory diagram showing the movement direction of the tip of the endoscope. [Figure 4] FIG. 10 is an image diagram of two frames of endoscopic images acquired at different timings. [Figure 5] 10 is an explanatory diagram showing a marker and a marker detection sensor for measuring the insertion depth of the endoscope. FIG. [Figure 6] FIG. 1 is a schematic diagram of the large intestine. [Figure 7] FIG. 10 is an image diagram of a movement trajectory display screen in which the movement trajectory is divided into outward and return paths and displayed as straight lines. [Figure 8] FIG. 10 is an image diagram of a movement trajectory display screen that displays the movement trajectory as a curved line, divided into an outward path and a return path. [Figure 9] FIG. 1 is an explanatory diagram showing a movement trajectory classified into three sections ((A), (B), (C)) and displayed, which are a current position, a movement already made, and a movement planned. [Figure 10] FIG. 1 is an explanatory diagram showing a movement trajectory that is divided into two sections ((A) and (B)) and shows a current position and a movement trajectory that has been moved. [Figure 11] FIG. 10 is an image diagram of a movement trajectory display screen in which the movement trajectory is displayed as a straight line, divided into outward and return paths, and the position of the attention area is displayed on the movement trajectory. [Figure 12] FIG. 10 is an image diagram of a movement trajectory display screen in which the movement trajectory is divided into outward and return paths and displayed as straight lines and curves, and the position of the region of interest is displayed on the movement trajectory. [Figure 13] FIG. 10 is an image diagram showing a section-by-section movement trajectory display screen corresponding to three sections. [Figure 14] 10 is a flowchart showing a series of steps in a method for displaying a movement trajectory. DETAILED DESCRIPTION OF THE INVENTION

[0016] As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, and a user interface 16. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The endoscope 12 includes an insertion section 12a that is inserted into the body of an observation subject, an operation section 12b provided at the base end of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip side of the insertion section 12a. The bending section 12c is bent by operating the operation section 12b. The tip section 12d is directed in a desired direction by the bending operation of the bending section 12c.

[0017] The operation unit 12b is provided with operation switches 18a and 18b that are used for various operations by the user. 18 b is assigned with operation commands for executing various operations via the user interface 16. In addition, the operation unit 12b is assigned with operation commands for executing various operations via the user interface 16. Image A zoom operation unit 19 is provided for use in zooming in or out.

[0018] The light source device 13 generates illumination light for illuminating the observation object and supplies the generated illumination light to the endoscope 12. The endoscope 12 irradiates the observation object with the illumination light from the light source device 13 and captures an image of the observation object illuminated by the illumination light. The endoscope 12 transmits an endoscopic image obtained by capturing an image of the observation object to the processor device 14.

[0019] The processor device 14 is electrically connected to the display 15 and the user interface 16. The processor device 14 performs various image processing such as color adjustment processing or structure enhancement processing on the image from the endoscope 12. The image after the various image processing is sent to the display 15. The display 15 outputs and displays the image of the observation target and information accompanying the image of the observation target. The user interface 16 has a keyboard, mouse, touchpad, microphone, etc., and has the function of accepting input operations such as function settings.

[0020] An extended processor device (not shown) for executing AI processing such as detecting lesions using AI (Artificial Intelligence) may be connected to the processor device 14. In this case, an extended display (not shown) separate from the display 15 may be connected to the extended processor device to display images processed by the extended processor device. Therefore, the "processor device" of the present invention corresponds to the processor device 14 as well as the extended processor device. Similarly, the "display" of the present invention corresponds to the display 15 as well as the extended display.

[0021] 2, the processor device 14 includes a movement status acquisition unit 30, a movement trajectory calculation unit 31, a first recognition processing unit 32, a display control unit 33, a second recognition processing unit 34, and a category classification unit 35. The processor device 14 has processor device programs for executing various processes stored in a program memory (not shown). The processor device 14 is provided with a central control unit (not shown) made up of a processor. The central control unit executes the processor device programs stored in the program memory, thereby realizing the functions of the movement status acquisition unit 30, the movement trajectory calculation unit 31, the first recognition processing unit 32, the display control unit 33, the second recognition processing unit 34, and the category classification unit 35.

[0022] The movement status acquisition unit 30 acquires the movement status of the endoscope moving inside the body. Specifically, it is preferable that the movement status acquisition unit 30 acquires the movement status based on the movement amount of the tip portion 12d of the endoscope. As shown in Fig. 3, it is preferable that the movement amount of the tip portion 12d be expressed by the movement amount in the up / down / left / right directions (X-axis direction or Y-axis direction) of the tip portion 12d relative to the movement direction (Z-axis direction) of the tip portion 12d, and the rotation amount (θ (when the Z-axis is the center of rotation)) of the tip portion 12d (parameters of X, Y, Z, θ).

[0023] As shown in FIG. 4, the movement status acquisition unit 30 preferably calculates the amount of movement of the tip portion 12d by comparing at least two frames of endoscopic images (endoscopic image P(N) in the Nth frame and endoscopic image P(N+1) in the N+1th frame) acquired at different times by the endoscope. Comparing the endoscopic images P(N) and P(N+1) reveals that the region X has moved by a specific distance. To calculate the amount of movement of the tip portion 12d using the endoscopic images, it is preferable to use a registration process between the endoscopic image P(N) and the endoscopic image P(N+1).

[0024] The alignment process includes a translation process for translating the endoscopic image P(N) or P(N+1) and a scaling process for enlarging or reducing the endoscopic image P(N) or P(N+1). The translation process makes it possible to calculate the amount of movement of the tip portion 12d in the X-axis direction or Y-axis direction, and the amount of rotation θ of the tip portion 12d around the Z-axis as the center of rotation. The scaling process also makes it possible to calculate the amount of movement of the tip portion 12d in the movement direction Z when it is inserted or removed.

[0025] The movement status acquiring unit 30 may acquire the amount of movement of the tip 12d based on the insertion length of the insertion unit 12a inserted into the body. In this case, as shown in Fig. 5, the insertion unit 12a has markers 40 such as magnets provided at regular intervals, and the markers are positioned relative to the insertion opening of the endoscope 12 (the anus 44 (see Fig. 6) when the inside of the body is the large intestine). 40 The insertion length of the insertion section 12a is calculated based on the result of the marker detection sensor 41 detecting the marker 40 due to movement of the insertion section 12a (movement in the Z-axis direction). Note that the amount of movement of the tip section 12d in the X-axis or Y-axis direction is preferably calculated using a movement amount measuring sensor 42 provided in the operation section 12b. The movement amount measuring sensor 42 calculates the amount of movement of the tip section 12d in the X-axis or Y-axis direction from the amount of operation of the bending section 12c. Note that the amount of movement of the tip section 12d of the endoscope may also be measured using a magnetic sensor such as a coronabi.

[0026] The movement trajectory calculation unit 31 calculates a movement trajectory indicating the path along which the endoscope 12 has moved, using the movement status of the endoscope 12. Specifically, the movement trajectory can be calculated by accumulating the movement status of the endoscope 12. When the movement status of the endoscope 12 is expressed by the movement amount (X, Y, Z, θ) of the tip portion 12d, the movement trajectory can be expressed as a three-dimensional curve by using all four parameters (X, Y, Z, θ). Furthermore, the movement trajectory can also be expressed as a two-dimensional curve by projecting the three-dimensional curve (see FIG. 8). Note that the movement trajectory can be expressed as a straight line by using only Z of the four parameters (X, Y, Z, θ) (see FIG. 7).

[0027] When the tip 12d of the endoscope reciprocates at a turning position inside the body, the first recognition processing unit 32 recognizes from the endoscopic image obtained by the endoscope 12 that the tip 12d has reached the turning position and recognizes whether the movement trajectory corresponds to the outward or return path. Specifically, as shown in FIG. 6, when the inside of the body is the large intestine, the turning position corresponds to the ileocecal region 43. The first recognition processing unit 32 performs processing to recognize the ileocecal region 43 from the endoscopic image. Therefore, until the first recognition processing unit 32 recognizes the ileocecal region 43 (the ileocecal region 43 has not been recognized), the first recognition processing unit 32 recognizes that the movement trajectory corresponds to the outward path. On the other hand, after the first recognition processing unit 32 recognizes the ileocecal region 43 (the ileocecal region 43 has been recognized), the first recognition processing unit 32 recognizes that the movement trajectory corresponds to the outward path. trajectory It is preferable that the first recognition processing unit 32 is a trained model that is trained by machine learning using the endoscopic image that is the input image and the correct answer (site, position, etc.).

[0028] Furthermore, the first recognition processing unit 32 recognizes multiple regions, including at least a first region or a second region, between the insertion opening and the turning position of the endoscope 12. Specifically, if the inside of the body is the large intestine, the insertion opening of the endoscope 12 corresponds to the anus 44, and the turning position corresponds to the ileocecal flexure 43. Furthermore, the first region on the anal side corresponds to the splenic flexure 45, and the second region on the ileocecal flexure side corresponds to the hepatic flexure 46. By recognizing multiple regions in this manner, it becomes possible to recognize multiple sections defined by the multiple regions. If the inside of the body is the large intestine, by recognizing the splenic flexure 45 and the hepatic flexure 46, it is possible to recognize three regions: the descending colon 47, the transverse colon 48, and the ascending colon 49.

[0029] If neither the splenic flexure 45 nor the hepatic flexure 46 is recognized, it is recognized that the movement trajectory is in the section of the descending colon 47 on the outward journey. If the splenic flexure 45 is recognized but the hepatic flexure 46 is not recognized, it is recognized that the movement trajectory is in the section of the transverse colon 48 on the outward journey. If the hepatic flexure 46 is recognized, it is recognized that the movement trajectory is in the section of the ascending colon 49 on the outward journey or the return journey. If the hepatic flexure 46 is recognized again after the hepatic flexure 46 is recognized, it is recognized that the movement trajectory is in the transverse colon 48 on the return journey. If the splenic flexure 45 is recognized after the hepatic flexure 46 is recognized, it is recognized that the movement trajectory is in the descending colon 47 on the return journey. Section of Recognize that there is.

[0030] The display control unit 33 displays a movement trajectory display screen on the display 15, which displays the movement trajectory separately for the outward path and the return path. This visualizes the trajectory of the tip 12d separately for insertion (outward path) and removal (return path) of the endoscope 12, thereby reducing the influence of changes in the shape of the body. Specifically, when the movement trajectory is displayed as a straight line, the display control unit 33 displays an outward path display straight line 51 representing the movement trajectory of the outward path and a return path display straight line 52 representing the movement trajectory of the return path on the movement trajectory display screen 50, as shown in FIG.

[0031] Also, a straight line for indicating the outward journey 5 1 and the return route display line 52 are connected by a turn-back display mark 53 that indicates the turn-back position. 53 However, it may have other shapes (for example, a straight line (see FIG. 9(A) and the like)). In addition, among the outward route display straight line 51, the return route display straight line 52, and the turn-back display mark 53, the solid line portion L Y indicates the current location of the tip 12d, and the dotted line portion L X indicates the portion to which the tip 12d has already moved or is to move in the future.

[0032] Furthermore, the movement trajectory display screen 50 may display an insertion length display scale 54 that indicates the insertion length of the endoscope 12 relative to the forward path display line 51 or the backward path display line 52. The insertion length display scale 54 is preferably disposed to the side of the forward path display line 51 or the backward path display line 52, and the display of the insertion length display scale 54 is preferably switched (scrolled) to show the scale corresponding to the actual insertion length in accordance with the movement of the distal end portion 12d of the endoscope. For example, in the case of FIG. 7, since the distal end portion 12d is near the turn-back position, the insertion length display scale 54 shows the insertion length near the turn-back position (100 cm to 120 cm). On the other hand, The tip 12d In the case of the descending colon section immediately after insertion, the insertion length display scale 54 indicates the insertion length (approximately 0 to 30 cm).

[0033] Furthermore, when the movement trajectory is represented by a two-dimensional curve, an outward movement display curve 56 representing the movement trajectory of the outward movement, and a return movement display curve 57 representing the movement trajectory of the return movement, are displayed on the movement trajectory display screen 50, as shown in Fig. 8. Of the outward movement display curve 56 and the return movement display curve 57, the solid line portion LM represents the location where the tip 12d is currently located, and the dotted line portion LN represents the portion where the tip 12d has already moved or is scheduled to move.

[0034] Furthermore, on the movement trajectory display screen 50, the movement trajectory may be displayed by classifying it into a plurality of sections defined by a plurality of regions. If the inside of the body is the large intestine, it is preferable to display it by classifying it into three sections, namely, the descending colon, the transverse colon, and the ascending colon, defined by the splenic flexure (first region) and the hepatic flexure (second region). Specifically, when the movement trajectory is displayed by an outward path display line 51 or a return path display line 52, as shown in FIG. 9(A), first region display markers 58a and 58b representing the splenic flexure 45 and second region display markers 59a and 59b representing the hepatic flexure 46 are used to represent a descending colon section 47a, a transverse colon section 48a, and an ascending colon section 49a on the outward path, and a descending colon section 47b, a transverse colon section 48b, and an ascending colon section 49b on the return path. It is preferable that the descending colon sections 47a, b, the transverse colon sections 48a, b, and the ascending colon sections 49a, b be displayed in different colors. Since the shape of the large intestine differs between the outbound and return journeys, the first region display marker 58a and the second region display marker 59a on the outbound journey side and the first region display marker 58b and the second region display marker 59b on the return journey side are displayed with some deviation on a straight line.

[0035] Furthermore, as shown in Figure 9(B), it is preferable to provide gaps 60a, b, c, and d between each section so that the descending colon section 47a, the transverse colon section 48a, and the ascending colon section 49a on the outbound journey, and the descending colon section 47b, the transverse colon section 48b, and the ascending colon section 49b on the return journey can be identified without using first region display markers 58a and b and second region display markers 59a and b representing the hepatic flexure 46.

[0036] As shown in FIG. 9C, the first region display markers 58a and 58b and the second region display markers 59a and 59b Group of and breaks 60a, b, c, d Group of and 49a, the descending colon section 47a, the transverse colon section 48a, and the ascending colon section 49a on the outbound journey, and the descending colon section 47b, the transverse colon section 48b, and the ascending colon section 49b on the return journey may be displayed so as to be clearly visible.

[0037] In addition, in FIG. 7 or 8, the current position of the distal end portion 12d of the endoscope is indicated by a solid line portion L Y , LM, and the portion where the tip portion 12d has already moved or is about to move is indicated by the dotted line portion L X , LN, but may be represented in other ways. For example, as shown in Fig. 10(A) and (B), on the movement trajectory display screen 50, only the current position of the tip 12d and the portion that has already moved are represented by a solid line LP, and the portion that is to move is not displayed. In this case, in order to display in which section the tip 12d is currently located, first region display markers 58a, b and second region display markers 59a, b Group of and breaks 60a, b, c, d Group of At least one of the above may be used to identify the sections 47a, 47b, 48a, 48b, 49a, 49b, and the section in which the tip portion 12d is currently located may be displayed by messages M1 and M2.

[0038] For example, Fig. 10(A) shows the movement trajectory at time t, with message M1 indicating that the current position is in the "transverse colon." Fig. 10(B) shows the movement trajectory at time t+α, which is later than time t, with message M2 indicating that the current position is in the "ascending colon." The current position of tip 12d of the endoscope is displayed as tip position 61, which is the tip of the movement trajectory. Tip position 61 is displayed with a portion cut out when tip 12d is located in the middle of each section, and is displayed without being cut out when tip 12d is located at the boundary between each section.

[0039] The second recognition processor 34 recognizes a region of interest from the endoscopic image. The region of interest may be, for example, a lesion such as cancer, a treatment scar, a surgical scar, a bleeding site, a benign tumor, an inflamed area (including areas with changes such as bleeding or atrophy in addition to inflammation), a cauterization scar or a marking made by coloring with a coloring agent or fluorescent agent, or a biopsy site where a biopsy test (biopsy) was performed. In other words, the region of interest may be a region containing a lesion, a region with a possibility of a lesion, a region where some kind of treatment such as a biopsy has been performed, a treatment tool such as a clip or forceps, or a dark area (such as the back of a fold or an area deep in the lumen where observation light cannot reach), which requires detailed observation regardless of the possibility of a lesion. The second recognition processor 34 detects a region including at least one of a lesion, a treatment scar, a surgical scar, a bleeding site, a benign tumor, an inflamed area, a marking, or a biopsy site as a region of interest. Like the first recognition processing unit 32, the second recognition processing unit 34 is preferably a trained model that is machine-learned using the endoscopic image that is the input image and the correct answer (presence or absence of a lesion, etc.).

[0040] The category classification unit 35 classifies the category of the region of interest recognized by the second recognition processing unit 34. The category to be classified is any one of the following: the imaging site, the presence or absence and type of a lesion which is one of the regions of interest, the use state of a treatment tool, the state of dye spraying, etc., or a combination thereof. For example, the category of the region of interest may be "Hyper Plastic (hyperplasia)" or "Neo Plastic (tumor)".

[0041] As described above, when an attention area is recognized by the second recognition processing unit 34, it is preferable to display the position of the attention area on the movement trajectory on the movement trajectory display screen 50. Specifically, when the movement trajectory is displayed with both straight and curved lines, as shown in Fig. 11, an attention area detection point BP is displayed at the position where the attention area is recognized relative to the outbound path display line 51, the return path display line 52, or the turn-back display mark 53. Furthermore, an attention area detection point BP is displayed at the position where the attention area is recognized relative to the outbound path display curve 56 or the return path display curve 57.

[0042] Furthermore, when the attention area is classified into categories, the position of the attention area may be displayed on the movement trajectory in a different display mode depending on the classification result of the category on the movement trajectory display screen 50. Specifically, the movement trajectory may be displayed as a straight line. and When both curves are displayed, as shown in FIG. 12, a category A attention area detection point BPA is displayed at the position where the attention area of ​​category A is recognized relative to an outward route display straight line 51 and a return route display straight line 52, and a category B attention area detection point BPB is displayed at the position where the attention area of ​​category B (different from category A) is recognized. For example, it is preferable to use yellow as the color of the attention area detection point BPA and green as the color of the attention area detection point BPB. Also, category A is, for example, "Neo Plastic" and category B is, for example, "Hyper Plastic". Note that the movement locus of the curve to In this case, when the categories are different, the attention area detection points BPA and BPB are used in different display modes.

[0043] Furthermore, when the areas of interest are classified into categories, category information including the category of the area of ​​interest may be displayed for the movement trajectory display screen 50. Specifically, on the outbound path display straight line 51 and the return path display straight line 52, an area of ​​interest detection point BPA for category A is displayed at a position where an area of ​​interest of category A is recognized, and category information CA including the fact that the area of ​​interest is category A and the size of the area of ​​interest is displayed to the side of the area of ​​interest detection point BPA. Furthermore, an area of ​​interest detection point BPB for category B is displayed at a position where an area of ​​interest of category B is recognized, and category information CB including the fact that the area of ​​interest is category B and the size of the area of ​​interest is displayed to the side of the area of ​​interest detection point BPB. Note that the category information CA and CB may also include images GA and GB of the area of ​​interest, respectively.

[0044] The movement trajectory display screen 50 displays all sections defined by multiple parts (for example, if the inside of the body is the large intestine, all sections of the descending colon, transverse colon, and ascending colon). However, when displaying the examination results after the examination is completed, such as when creating a report using endoscopic images, the movement trajectory may be composed of multiple section-specific movement trajectories, each for a different section, and the movement trajectory display screen may display a section-specific movement trajectory display screen that displays the movement trajectory for each section.

[0045] For example, if the inside of the body is the large intestine and the splenic flexure (first region) and the hepatic flexure (second region) are recognized, as shown in FIG. 13 , the movement trajectory for the descending colon section is displayed on a section-specific movement trajectory display screen 70, the movement trajectory for the transverse colon section is displayed on a section-specific movement trajectory display screen 71, and the movement trajectory for the ascending colon section is displayed on a section-specific movement trajectory display screen 72. By displaying the movement trajectory divided into three sections in this way, the amount of information on the movement trajectory and attention area per display screen is reduced, improving the user's visibility. Note that if the information for all sections is displayed on one screen, if there are a large number of attention areas, one movement trajectory may be marked with many marks (attention area detection points BPA, etc.), making it difficult for the user to see.

[0046] The section-by-section movement trajectory display screen 70 includes a straight-line area section 70a that displays the movement trajectory as a straight line, and a curved-line area section 70b that displays the movement trajectory as a curve. In the straight-line area section 70a, it is preferable to display only the portion of the movement trajectory that is recognized as a region of interest. In the curved-line area section 70b, the descending colon section is displayed by a solid-line section LS, and other sections are displayed by a dotted-line section LT. The method of displaying the region of interest is otherwise the same as in the above cases (see FIGS. 11 and 12).

[0047] Similarly to the section-specific movement trajectory display screen 70, the section-specific movement trajectory display screen 71 and the section-specific movement trajectory display screen 72 each have straight line area sections 71a, 72a and curved line area sections 71b, 72b, and display the movement trajectory or areas of interest in the same manner as the section-specific movement trajectory display screen 70.

[0048] Next, the display of the movement trajectory will be described with reference to the flowchart in Fig. 14. A movement status acquisition unit 30 provided in the processor device 14 acquires the movement status of the endoscope moving inside the body. A movement trajectory calculation unit 31 calculates a movement trajectory indicating the path traveled by the endoscope using the movement status of the endoscope. A first recognition processing unit 32 recognizes, from the endoscopic image obtained by the endoscope 12, whether the tip 12d has reached the turning back position.

[0049] When it is recognized that the tip end 12d has not reached the turn-back position, the first recognition processing unit 32 recognizes that the movement trajectory corresponds to the outward journey. On the other hand, when it is recognized that the tip end 12d has reached the turn-back position, the first recognition processing unit 32 recognizes that the movement trajectory corresponds to the return journey. The display control unit 33 displays on the display 15 a movement trajectory display screen 50 that shows the movement trajectory divided into the outward journey and the return journey.

[0050] In the above embodiment, the hardware structure of processing units that perform various processes, such as the movement status acquisition unit 30, the movement trajectory calculation unit 31, the first recognition processing unit 32, the display control unit 33, the second recognition processing unit 34, and the category classification unit 35, is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a GPU (Graphical Processing Unit), a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacturing, such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for performing various processes.

[0051] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Also, multiple processing units may be configured with a single processor. Examples of multiple processing units configured with a single processor include, first, a configuration in which one processor is configured with a combination of one or more CPUs and software, as typified by client or server computers, and this processor functions as multiple processing units. Second, a configuration in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by a system-on-chip (SoC). In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0052] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit formed by combining circuit elements such as semiconductor elements, and the hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). [Explanation of symbols]

[0053] 10 Endoscopy System 12 Endoscopy 12a Insertion part 12b Operation section 12c curved section 12d Tip 13 Light source device 14 Processor unit 15 Display 16 User Interface 18a, b Operation switch 19 Zoom control section 30 Movement status acquisition unit 31 Movement trajectory calculation section 32 First recognition processing unit 33 Display control unit 34 Second recognition processing section 35 Category Classification Department 40 markers 41 Marker detection sensor 42 Movement measurement sensor 43 Ileocecal area 44 Anal 45 Splenic flexure 46 Liver flexure 47 Descending colon 47a Descending colon section on outward journey 47b Descending colon section on the return journey 48 Transverse colon 48a Transverse colon section on outbound journey 48b Transverse colon section on the return journey 49 Ascending colon 49a Ascending colon section on outward journey 49b Ascending colon section on the return journey 50 Movement trajectory display screen 51 Line for outward route indication 52 Return journey display straight line 53 Wrap-around mark 54 Insertion depth indicator 56 Outward journey curve 57 Return route display curve 58a, 58b First body part display marker 59a, 59b Second body part display marker 60a, b, c, d cuts 61 Tip position 70, 71, 72 Section-specific movement track display screen 70a, 71a, 72a Straight line area 70b, 71b, 72b curve area section L Y , LM, LS, LP solid line parts L X , LN, LT dotted line area M1, M2 messages BPA, BPB attention area detection points CA, CB category information P(N), P(N+1) endoscopic images GA, GB Image of the area of ​​interest

Claims

1. In a processor device having a processor, The processor: Acquire the movement status of the endoscope moving inside the body, calculating a movement trajectory indicating a path along which the endoscope has moved using the movement status of the endoscope; When the distal end of the endoscope moves back and forth at a turning position inside the body, it is recognized from an endoscopic image obtained by the endoscope that the distal end has reached the turning position, and it is recognized whether the movement trajectory corresponds to an outward path or a return path; displaying a movement trajectory display screen on a display, the movement trajectory being divided into the outbound path and the return path; an outward path display line representing the outward path movement trajectory and a return path display line representing the return path movement trajectory are displayed spaced apart from each other on the movement trajectory display screen; a processor device in which the end of the forward route indication straight line and the end of the return route indication straight line are connected by a turn-back indication mark that indicates the turn-back position;

2. 2. The processor device according to claim 1, wherein an insertion length display scale indicating the insertion length of the endoscope is displayed on the movement trajectory display screen relative to the forward path display line or the return path display line.

3. 2. The processor device according to claim 1, wherein the movement locus display screen displays an outward movement display curve representing the movement locus of the outward movement and a return movement display curve representing the movement locus of the return movement.

4. A processor device having a processor, The processor: Acquire the movement status of the endoscope moving inside the body, calculating a movement trajectory indicating a path along which the endoscope has moved using the movement status of the endoscope; When the distal end of the endoscope moves back and forth at a turning position inside the body, it is recognized from an endoscopic image obtained by the endoscope that the distal end has reached the turning position, and it is recognized whether the movement trajectory corresponds to an outward path or a return path; displaying a movement trajectory display screen on a display, the movement trajectory being divided into the outbound path and the return path; the processor recognizes a plurality of regions including at least a first region and a second region between the insertion port of the endoscope and the folding back position; The processor device displays the movement trajectory on the movement trajectory display screen by classifying the movement trajectory into a plurality of sections defined by the plurality of parts.

5. 5. The processor device according to claim 4, wherein the movement locus display screen displays a message indicating the section in which the tip is currently located.

6. The processor recognizes a region of interest from the endoscopic image; 6. The processor device according to claim 1, wherein the position of the region of interest is displayed on the movement trajectory on the movement trajectory display screen.

7. 7. The processor device according to claim 6, wherein the processor classifies categories of the attention areas, and displays the positions of the attention areas on the movement trajectory on the movement trajectory display screen in different display modes depending on the classification results of the categories.

8. 8. The processor device according to claim 7, wherein the movement trajectory display screen displays category information including a category of the region of interest for the movement trajectory.

9. A processor device having a processor, The processor: Acquire the movement status of the endoscope moving inside the body, calculating a movement trajectory indicating a path along which the endoscope has moved using the movement status of the endoscope; When the distal end of the endoscope moves back and forth at a turning position inside the body, it is recognized from an endoscopic image obtained by the endoscope that the distal end has reached the turning position, and it is recognized whether the movement trajectory corresponds to an outward path or a return path; displaying a movement trajectory display screen on a display, the movement trajectory being divided into the outbound path and the return path; the processor recognizes a plurality of regions including at least a first region and a second region between the insertion port of the endoscope and the folding back position; A plurality of sections are defined from the plurality of parts, The movement trajectory is composed of a plurality of section-specific movement trajectories provided for each of the sections, The movement trajectory display screen is a processor device that displays a movement trajectory display screen for each section, which displays the movement trajectory for each section.

10. the movement status is a movement amount of the distal end portion of the endoscope, 10. The processor device according to claim 1, wherein the amount of movement of the tip portion is calculated based on at least the insertion length of the endoscope.

11. An endoscope system comprising a processor and a display, The processor: Acquire the movement status of the endoscope moving inside the body, Using the movement status of the endoscope, a movement trajectory indicating the path along which the endoscope has moved is calculated, and when the distal end of the endoscope moves back and forth at a turning position within the body, it is recognized from an endoscopic image obtained by the endoscope that the distal end has reached the turning position and whether the movement trajectory corresponds to an outward or return path; displaying a movement trajectory display screen on the display, the movement trajectory being divided into the outbound path and the return path; an outward path display line representing the outward path movement trajectory and a return path display line representing the return path movement trajectory are displayed spaced apart from each other on the movement trajectory display screen; An endoscope system in which the end of the forward path display straight line and the end of the return path display straight line are connected by a turn-back display mark that indicates the turn-back position.

12. A method for operating a processor device having a processor, comprising: The processor: Acquire the movement status of the endoscope moving inside the body, Using the movement status of the endoscope, a movement trajectory indicating the path along which the endoscope has moved is calculated, and when the distal end of the endoscope moves back and forth at a turning position within the body, it is recognized from an endoscopic image obtained by the endoscope that the distal end has reached the turning position and whether the movement trajectory corresponds to an outward or return path; displaying a movement trajectory display screen on a display, the movement trajectory being divided into the outbound path and the return path; an outward path display line representing the outward path movement trajectory and a return path display line representing the return path movement trajectory are displayed spaced apart from each other on the movement trajectory display screen; A method for operating a processor device, wherein the end of the forward path indication straight line and the end of the return path indication straight line are connected by a turn-back indication mark that indicates the turn-back position.

13. A function of acquiring the movement status of the endoscope moving within the body; a function of calculating a movement trajectory indicating the path along which the endoscope has moved using the movement status of the endoscope; a function of recognizing, from an endoscopic image obtained by the endoscope, that the tip of the endoscope has reached the turning position and recognizing whether the movement trajectory corresponds to an outward or return path when the tip of the endoscope moves back and forth at the turning position inside the body; and a function of displaying a movement trajectory display screen on a display, the movement trajectory being divided into the outward journey and the return journey, an outward path display line representing the outward path movement trajectory and a return path display line representing the return path movement trajectory are displayed spaced apart from each other on the movement trajectory display screen; A program for a processor device, wherein an end of the outward path display straight line and an end of the return path display straight line are connected by a turn-back display mark that indicates the turn-back position.

Images