Diagnosis assistance device, endoscope device, display information generation method, display information generation program, and non-volatile storage medium

Abstract

A diagnosis assistance device comprises a processor. The processor acquires position information pertaining to an object being detected within a body, acquires position information pertaining to a landmark within the body, and outputs direction information pertaining to the detection position for the object being detected with respect to the landmark.

Description

Diagnosis support device, endoscope device, display information generation method, display information generation program, and non-volatile storage medium

[0001] The present invention relates to a diagnosis support device, an endoscopic device, a display information generation method, a display information generation program, and a non-volatile storage medium that are capable of displaying information for identifying an observation site when observing an organ in which the observation site is difficult to identify.

[0002] Conventionally, endoscope devices have been widely used in the medical and industrial fields. Endoscope devices used in the medical field have a long, thin insertion portion that is inserted into the body, and are widely used for observing organs, performing medical treatments using treatment tools, performing surgical operations under endoscopic observation, and the like.

[0003] For example, International Publication No. 2021 / 149137 discloses a technology for recording information such as the location of lesions discovered during endoscopic observation and imaging conditions on a 2D organ model.

[0004] This technology allows the tip of the endoscope to be temporarily moved to another position after a lesion is discovered, and then the lesion can be located again when re-observing or re-examining at a later date, facilitating re-observation and re-examination.

[0005] However, when observing organs that are nearly spherical, such as the bladder or uterus, users can easily lose their sense of direction, and even when the location of a lesion is indicated on the organ model, it can be difficult to know in which direction to move the tip of the endoscope.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a diagnostic support device, an endoscopic device, a display information generation method, a display information generation program, and a non-volatile storage medium that are capable of presenting information indicating in which direction the tip of the endoscope should be advanced, even when observing an organ whose direction is easily lost.

[0007] A diagnostic support device according to one aspect of the present invention includes a processor that acquires position information of a detection target within a body, acquires position information of landmarks within the body, and outputs directional information of the detection position of the detection target relative to the landmarks.

[0008] Another aspect of the diagnostic support device of the present invention includes a first position information acquisition unit that acquires position information of a detection target within the body, a second position information acquisition unit that acquires position information of landmarks within the body, an organ model information acquisition unit that acquires information of an organ model in which a diagram of an organ is depicted and the landmarks are further depicted within the organ, an alignment unit that determines the left-right, up-down, and right-left directions when displaying the organ model and then aligns the landmarks on the organ model with landmarks within the body, a first output unit that outputs information of the organ model that reflects the detection position of the detection target, and a second output unit that outputs directional information of the detection position relative to the landmarks on the organ model.

[0009] An endoscopic device according to one aspect of the present invention includes the above-described diagnostic support device, an endoscope that directly or indirectly transmits endoscopic image data to the diagnostic support device, and a display that receives a signal output from the diagnostic support device and displays an organ model.

[0010] A display information generation method of one embodiment of the present invention is a display information generation method for displaying the direction toward a detection target on an observation target model, wherein a first position information acquisition unit acquires position information of the detection target inside the body, a second position information acquisition unit acquires position information of landmarks inside the body, an organ model information acquisition unit acquires information on an organ model in which a diagram simulating an organ is drawn and the landmarks are further drawn inside the organ, an alignment unit determines the left / right, up / down, and up / down directions when displaying the organ model and then aligns the landmarks on the organ model with the landmarks inside the body, a first output unit outputs information on the organ model that reflects the detection position of the detection target, and a second output unit outputs direction information of the detection position relative to the landmarks on the organ model.

[0011] A display information generation program of one embodiment of the present invention is a display information generation program for displaying the direction toward a detection target on an observation target model, and causes a processor to execute the following processes: acquire position information of the detection target inside the body; acquire position information of landmarks inside the body; acquire information on an organ model in which a diagram simulating an organ is depicted and the landmarks are further depicted inside the organ; determine the left-right, up-down, and right-left directions when displaying the organ model; align the landmarks on the organ model with the landmarks inside the body; output information on the organ model that reflects the detection position of the detection target; and output direction information of the detection position relative to the landmarks on the organ model.

[0012] A non-volatile storage medium of one embodiment of the present invention is a non-volatile storage medium that stores a display information generation program for displaying the direction toward a detection target on an observation target model, the program causing a processor to execute the following processes: acquire position information of the detection target inside the body; acquire position information of landmarks inside the body; acquire information about an organ model in which a diagram of an organ is depicted and the landmarks are depicted inside the organ; determine the left-right, up-down, and right-left directions when displaying the organ model; align the landmarks on the organ model with the landmarks inside the body; output information about the organ model that reflects the detection position of the detection target; and output direction information of the detection position relative to the landmarks on the organ model.

[0013] FIG. 1 is a diagram illustrating an example of the functional configuration of an endoscope system according to a first embodiment; FIG. 2 is a diagram illustrating an example of a virtual model of a bladder; FIG. 3 is a diagram illustrating an example of a display screen displayed on a monitor according to the first embodiment; FIG. 4 is a flowchart illustrating an example of the flow of display information generation processing according to the first embodiment; FIG. 5 is a diagram illustrating an example of the functional configuration of an endoscope system according to a second embodiment; FIG. 6 is a diagram illustrating an example of the functional configuration of an endoscope system according to a third embodiment; FIG. 7 is a diagram illustrating an example of a display screen displayed on a monitor according to the third embodiment; and FIG. 8 is a flowchart illustrating an example of the flow of display information generation processing according to the third embodiment.

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, identical or corresponding elements are appropriately designated by the same reference numerals. It should be noted that the drawings are schematic, and the length relationships, length ratios, and quantities of elements within a single drawing may differ from reality in order to simplify the description. Furthermore, there may be portions in which the length relationships and ratios differ between multiple drawings.

[0015] Fig. 1 is a diagram showing an example of the functional configuration of an endoscope system according to a first embodiment. As shown in Fig. 1, an endoscope system 100 includes an endoscope 1, a video processor 2, a light source device 3, and a monitor 4. The endoscope system according to this embodiment is made up of the endoscope 1, the video processor 2, and the monitor 4. In this embodiment, the video processor 2 and the light source device 3 are separate entities, but the video processor 2 and the light source device 3 may be integrated into one unit.

[0016] The endoscope 1 includes an imaging lens 11 , an imaging element 12 , an A / D converter 13 , a light guide 14 , and an illumination lens 15 .

[0017] The imaging lens 11 includes one or more lenses and forms an optical image of the subject on the imaging element 12. The optical image formed on the imaging element 12 includes images of one or more parts of the subject's multiple parts.

[0018] The imaging element 12 photoelectrically converts (captures) an optical image of the subject and generates an analog imaging signal related to the endoscopic image. The imaging element 12 is configured using, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging lens 11 and the imaging element 12 constitute an imaging system that captures images and acquires endoscopic images.

[0019] The A / D converter 13 converts the analog imaging signal into a digital imaging signal. The endoscopic image corresponding to the digital imaging signal is transmitted from the endoscope 1 to the video processor 2. Although the example in which the A / D converter 13 is provided within the endoscope 1 has been shown here, the A / D converter 13 may also be provided within the video processor 2. Furthermore, the imaging element 12 may be a digital imaging element having the functionality of the A / D converter 13.

[0020] The light guide 14 transmits illumination light supplied from the light source device 3 to the illumination lens 15. The illumination lens 15 irradiates the subject with the illumination light transmitted by the light guide 14. The illumination lens 15 and the imaging system (imaging lens 11 and imaging element 12) are disposed at the tip of the insertion portion of the endoscope 1 (hereinafter referred to as the endoscope tip).

[0021] The video processor 2 constituting the diagnosis support device includes an image processing unit 21, a lesion detection unit 22, a landmark detection unit 23, a storage unit 24, an image synthesis unit 25, and a control unit 26. The control unit 26 includes a position calculation unit 27, an inter-landmark distance calculation unit 28, and a virtual model acquisition unit 29.

[0022] The image processing unit 21 performs image processing for image adjustment (image construction) on the endoscopic image input from the A / D conversion unit 13 of the endoscope 1. The image processing performed by the image processing unit 21 includes, for example, demosaicing, gain adjustment, white balance adjustment, gamma correction, noise reduction, contrast enhancement, color change, etc. Parameters for some of the processing performed by the image processing unit 21, for example, contrast enhancement, color change, etc., may be set by the user.

[0023] The image processing unit 21 outputs the processed endoscopic image to the lesion detection unit 22, the landmark detection unit 23, the image synthesis unit 25, and the control unit 26.

[0024] The lesion detection unit 22 detects lesions from the input endoscopic images and outputs the detection results to the image synthesis unit 25 and the control unit 26. The means by which the lesion detection unit 22 detects lesions is not particularly limited, and for example, the lesion detection unit 22 may detect lesions using a machine learning model trained to detect lesions from endoscopic images. The machine learning model is generated using any known machine learning method. Examples of machine learning that can be used include a convolutional neural network (CNN), a recurrent neural network (RNN), and a long short-term memory (LSTM), which are types of deep learning. The lesion detection unit 22 is not limited to detecting lesions using a machine learning model, and may also detect lesions using, for example, template matching or feature point matching.

[0025] The landmark detection unit 23 detects landmarks from the input endoscopic image and outputs the detection result to the control unit 26. The landmark detection unit 23 constitutes a second position information acquisition unit that acquires position information of landmarks inside the body. The means by which the landmark detection unit 23 detects landmarks is not particularly limited, and for example, the landmarks are detected using a machine learning model trained to detect landmarks from endoscopic images. Note that the landmark detection unit 23 is not limited to detecting landmarks using a machine learning model, and may detect landmarks using, for example, template matching or feature point matching.

[0026] A landmark is, for example, a distinctive part that serves as a landmark within an organ. Normally, the bladder is spherical and has no distinctive shape. However, the bladder has openings such as the urethral opening, the right ureteral opening, and the left ureteral opening. The urethral opening, the right ureteral opening, and the left ureteral opening have distinctive shapes because they are open, and furthermore, they do not move within the bladder, so they can serve as landmarks when observing the bladder. If the organ being observed is the uterus, the fallopian tube openings serve as landmarks.

[0027] Therefore, the landmark detection unit 23 detects the urethral opening, the right ureteral opening, and the left ureteral opening as landmarks from the input endoscopic image, and outputs the detection results to the control unit 26. In the following description, the right ureteral opening will be referred to as the right ureteral opening, and the left ureteral opening will be referred to as the left ureteral opening.

[0028] The control unit 26 comprehensively controls the entire endoscope system 100. For example, the control unit 26 stores information on distances between landmarks (described later) in the storage unit 24. When a lesion is detected by the lesion detection unit 22, the control unit 26 stores position information of the tip of the endoscope inside the body, specifically, information on the direction and distance from the landmark (described later), in the storage unit 24. Furthermore, the control unit 26 generates a control signal for controlling the light intensity of the light source 32 based on the brightness of the endoscopic image input from the image processing unit 21, and outputs the control signal to the light intensity control unit 31.

[0029] The control unit 26 is configured as a processor having hardware. For example, the control unit 26 is configured as an ASIC (Application Specific Integrated Circuit) including a CPU (Central Processing Unit) or an FPGA (Field Programmable Gate Array). However, at least a part of the control unit 26 may be configured as a dedicated electronic circuit.

[0030] The position calculation unit 27 constituting the first position information acquisition unit acquires position information of a detection target inside the body. The detection target inside the body is, for example, the tip of the endoscope. That is, the position calculation unit 27 acquires position information of the tip of the endoscope inside the body. The position calculation unit 27 calculates direction information and distance information from the landmark as current position information of the tip of the endoscope relative to the landmark.

[0031] Specifically, the position calculation unit 27 calculates (acquires) current position information of the endoscope tip relative to the landmark using the optical flow method on the endoscopic images sequentially input from the image processing unit 21. The optical flow method represents the positional deviation of an object or the like between consecutive frames as a vector, and calculates direction information and distance information from the landmark based on the direction and length of this vector. The position calculation unit 27, which constitutes the second output unit, outputs the calculated direction information and distance information from the landmark to the image synthesis unit 25.

[0032] The landmark distance calculation unit 28 calculates the distance between landmarks based on the detection results of the landmark detection unit 23. In order for the landmark distance calculation unit 28 to calculate the distance between landmarks, the landmark detection unit 23 needs to detect at least two landmarks from among the urethral opening, the right ureteral opening, and the left ureteral opening, which serve as landmarks. For example, when the landmark detection unit 23 detects the right ureteral opening and the left ureteral opening, the landmark distance calculation unit 28 calculates the distance between the right ureteral opening and the left ureteral opening and stores the calculation result in the storage unit 24.

[0033] The virtual model acquisition unit 29, which constitutes the organ model information acquisition unit, acquires organ model information (virtual model) that depicts a diagram of an organ (hereinafter simply referred to as an organ) and further depicts landmarks within the organ. The virtual model acquisition unit 29 also constitutes an alignment unit that determines the left-right, up-down, and up-down directions when displaying the virtual model, and then aligns the landmarks on the virtual model with landmarks within the body.

[0034] The virtual model acquisition unit 29 reflects, specifically, pastes, into the virtual model of the bladder the current position of the tip of the endoscope relative to the landmarks acquired by the position calculation unit 27. The virtual model acquisition unit 29 constitutes a first output unit that outputs the virtual model reflecting the current position of the tip of the endoscope to the image synthesis unit 25.

[0035] 2 is a diagram showing an example of a virtual bladder model. The virtual bladder model 41 shows a virtual bladder corresponding to the inner wall of the bladder observed by the endoscope 1. If the bladder is assumed to be approximately spherical, the hemisphere on the anterior wall side (ventral side) of the bladder is indicated by a circle 42, and the hemisphere on the posterior wall side (dorsal side) of the bladder is indicated by a circle 43.

[0036] A urethral orifice 44 is depicted as an opening at the center of the joint between the circle 42 on the anterior wall of the bladder and the circle 43 on the posterior wall of the bladder. A right ureteral orifice 45 and a left ureteral orifice 46 are depicted as openings at predetermined positions on the circle 43 on the posterior wall of the bladder (for example, positions 3 / 8 of the way from the bottom and 1 / 4 of the way to the left and right of the circle 43).

[0037] Furthermore, the current position of the tip of the endoscope relative to the landmark is attached to the virtual model 41 by the virtual model acquisition unit 29. In the example of Fig. 2, the current position of the tip of the endoscope relative to the landmark is reflected in the virtual model 41 as a black star (reference numeral 47).

[0038] The image synthesis unit 25 synthesizes the endoscopic image output from the image processing unit 21, the direction information and distance information from the landmark output from the position calculation unit 27, and the virtual model 41 output from the virtual model acquisition unit 29, and outputs the result to the monitor 4.

[0039] 3 is a diagram showing an example of a display screen displayed on the monitor in the first embodiment. Through the synthesis process of the image synthesis unit 25, the endoscopic image 51 output from the image processing unit 21, the direction information 61 and distance information 62 from the landmark (e.g., the right ureteral orifice 45) output from the position calculation unit 27, and the virtual model 41 output from the virtual model acquisition unit 29 are synthesized and displayed on the monitor 4.

[0040] Here, when the right ureteral orifice 45 is used as the landmark, the direction information 61 from the landmark is represented by the clockwise intersection angle between a first line L1 drawn up and down on the virtual model 41 to cross the right ureteral orifice 45, and a second line L2 drawn to cross both the right ureteral orifice 45 and the current position of the tip of the endoscope.

[0041] 3, the endoscopic image 51, the direction information 61 from the landmark, the distance information 62, and the virtual model 41 are displayed on the monitor 4, but it is sufficient if at least the direction information 61 from the landmark is displayed on the monitor 4. By checking the direction information 61 from the landmark, the user can understand which direction from the landmark they are observing.

[0042] The present invention can also use computer-aided detection (CADe) to detect the position of a lesion. When CADe is used in combination, when a lesion 52 is detected by the lesion detection unit 22, the detected position of the lesion 52 may be indicated by a marker such as a bounding box, as exemplified by reference numeral 53 in FIG. 3 .

[0043] The light source device 3 includes a light amount control unit 31 and a light source 32. The light amount control unit 31 controls the light amount of the light source 32 based on a control signal from the control unit 26 of the video processor 2.

[0044] The light source 32 is a light-emitting device such as an LED (Light Emitting Diode) light source, a laser light source, or a xenon light source. The light source 32 emits illumination light using the light-emitting device based on the control of the light quantity control unit 31. The light source 32 is configured by combining one or more types of light-emitting devices such as an LED light source, a laser light source, or a xenon light source. However, the light-emitting device is not limited to the examples given here, and any known technology can be used as appropriate.

[0045] Illumination light emitted from the light source 32 is incident on the incident end of the light guide 14. The light guide 14 transmits the illumination light incident from the incident end to the exit end. The transmitted illumination light is emitted from the exit end of the light guide 14 and is irradiated onto the subject by the illumination lens 15.

[0046] Next, a description will be given of the display information generation process of the thus configured endoscope system 100. Fig. 4 is a flowchart showing an example of the flow of the display information generation process of the first embodiment. Note that the display information generation process shown in Fig. 4 is constantly executed during the endoscopic examination and ends when the endoscopic examination is completed.

[0047] First, the control unit 26 determines whether or not at least two landmarks have been detected (S1). The process of S1 is performed by the control unit 26 based on the detection result of the landmark detection unit 23.

[0048] If the control unit 26 determines that at least two landmarks have not been detected (S1: NO), the control unit 26 returns to the process of S1 and repeats the same process. On the other hand, if the control unit 26 determines that at least two landmarks have been detected (S1: YES), the control unit 26 calculates the distance between the landmarks (S2). The process of S2 is calculated by the landmark distance calculation unit 28 of the control unit 26. For example, if the landmark detection unit 23 detects the right ureteral orifice 45 and the left ureteral orifice 46 as landmarks, the landmark distance calculation unit 28 calculates the distance between the right ureteral orifice 45 and the left ureteral orifice 46. Next, the control unit 26 stores the calculated distance between the landmarks in the storage unit 24 (S3).

[0049] Next, the control unit 26 calculates direction information and distance information of the current observation position from the landmark and displays it on the monitor 4 (S4). The direction information and distance information of the current observation position from the landmark are calculated by the position calculation unit 27. The calculated direction information and distance information are synthesized with the endoscopic image by the image synthesis unit 25 and displayed on the monitor 4.

[0050] Next, the control unit 26 determines whether or not a lesion has been detected (S5). The process of S5 is performed by the control unit 26 based on the detection result of the lesion detection unit 22.

[0051] If the control unit 26 determines that a lesion has not been detected (S5: NO), it returns to the process of S4 and repeats the same process. On the other hand, if the control unit 26 determines that a lesion has been detected (S5: YES), it stores information about the direction and distance of the current observation position from the landmark in the storage unit 24 (S6). Thereafter, it returns to the process of S4 and repeats the same process.

[0052] The current observation position when a lesion is detected is the position of the lesion relative to the landmark. That is, in the process of S6, position information of the lesion relative to the landmark is stored in the storage unit 24.

[0053] Although details will be described in the third embodiment, the information on the distance between landmarks stored in the memory unit 24 by the processing in S3 and the position information of the lesion based on the landmark stored in the memory unit 24 by the processing in S6 will be used during the next examination.

[0054] As described above, in this embodiment, the current observation position based on a landmark, specifically, the direction information and distance information from the landmark, is calculated and displayed on the monitor 4. Therefore, even when observing an organ whose observation position is easily lost, such as a spherical bladder, the current observation position can be made clear.

[0055] Second Embodiment Next, a second embodiment will be described. Fig. 5 is a diagram showing an example of the functional configuration of an endoscope system according to the second embodiment.

[0056] The endoscope system 100A includes an endoscope 1A and a video processor 2A instead of the endoscope 1 and the video processor 2 in Fig. 1. The endoscope 1A has a position sensor 16 added to it compared to the endoscope 1 in Fig. 1. The video processor 2A includes a control unit 26A and a position calculation unit 27A instead of the control unit 26 and the position calculation unit 27 of the video processor 2 in Fig. 1.

[0057] The position sensor 16 is a six-axis sensor consisting of a three-axis acceleration sensor and a three-axis gyro sensor. The position sensor 16 detects acceleration and angular velocity along three mutually orthogonal axes (X-axis, Y-axis, and Z-axis) and outputs the detection results to the control unit 26. The position calculation unit 27A calculates direction information and distance information of the current observation position from the landmark based on the detection results of the position sensor 16 and outputs the information to the image synthesis unit 25.

[0058] In the first embodiment, the direction information and distance information of the current observation position from the landmark are calculated using the optical flow method. In contrast, in the second embodiment, the direction information and distance information of the current observation position from the landmark are calculated based on the detection results of the six-axis position sensor 16. The other configurations are the same as those of the first embodiment.

[0059] The method for calculating the direction information and distance information of the current observation position from the landmark is not limited to the optical flow method or the method using the position sensor 16.

[0060] The direction information may be calculated using, for example, the detection results of a magnetic coil sensor or three-dimensional SLAM (Simultaneous Localization and Mapping). Furthermore, during a bladder examination, the patient is examined in a supine position. Typically, during a bladder examination, the bladder is filled with irrigation fluid, and when bubbles are generated, the bubbles move from the dorsal side to the ventral side. Therefore, the direction information may be calculated based on the direction of bubble movement. Furthermore, since the patient is examined in a supine position, the positions of the right ureteral orifice 45 and the left ureteral orifice 46 are approximately fixed. Therefore, the direction information may be calculated based on the line between the right ureteral orifice 45 and the left ureteral orifice 46.

[0061] The distance information may be calculated using distance estimation with a monocular camera or distance measurement with a stereo camera. The distance information may also be calculated using the detection results of a magnetic coil sensor, 3D SLAM, or distance measurement light, such as TOF (Time of Flight). Furthermore, the distance information may be calculated based on the diameter of a blood vessel or the size of a part of an instrument. Furthermore, the distance information may be calculated based on the diameter of the right ureteral orifice 45 or the left ureteral orifice 46, or the distance between the right ureteral orifice 45 and the left ureteral orifice 46.

[0062] Third Embodiment Next, a third embodiment will be described. Fig. 6 is a diagram showing an example of the functional configuration of an endoscope system according to the third embodiment. An endoscope system 100B includes a video processor 2B instead of the video processor 2 in Fig. 1 .

[0063] The video processor 2B includes a control unit 26B instead of the control unit 26 of the video processor 2 in Fig. 1. The control unit 26B includes a direction / distance information correction unit 30 in addition to the control unit 26 in Fig. 1. The other configurations are the same as those in the first embodiment.

[0064] In the third embodiment, as in the first embodiment, direction information and distance information from the landmark are calculated using the optical flow method, but direction information and distance information from the landmark may also be calculated using the detection results of the position sensor 16 in the second embodiment or other methods.

[0065] Generally, bladder cancer has a high probability of recurrence at the same location, so the location information of the lesion stored in the storage unit 24 in the process of S6 in Fig. 4 of the first embodiment, specifically, the direction information and distance information from the landmark, serves as a guide for the next endoscopic observation.

[0066] Normally, when observing the inside of the bladder, the field of view is secured while irrigation fluid is flowing, and the bladder expands and contracts depending on the amount of fluid during observation, causing the bladder size to change. If the size of the bladder changes, the position of the lesion also changes, so even if the position information of the lesion recorded in the previous examination is displayed on monitor 4 as is, there is a possibility that the position of the lesion from the previous examination cannot be identified during a re-examination.

[0067] Therefore, in this embodiment, the distance between landmarks calculated during the previous examination is compared with the distance between landmarks calculated during the current examination, and the position information of the lesion detected during the previous examination (information on the direction and distance from the landmark) is corrected according to the comparison result. As a result, position information of the lesion that takes into account the expansion and contraction state of the bladder (intracorporeal deformation) is displayed on monitor 4, making it possible to identify the position of the lesion detected during the previous examination.

[0068] At the time of re-examination, the direction / distance information correction unit 30 reads out the distance between the landmarks stored in the memory unit 24. This distance between the landmarks was calculated during the previous examination and is stored in the memory unit 24 by the process of S3 in Fig. 4. The direction / distance information correction unit 30 compares the distance between the landmarks during the previous examination stored in the memory unit 24 with the distance between the landmarks during the current examination calculated by the landmark distance calculation unit 28.

[0069] Then, in accordance with the comparison result, the direction / distance information corrector 30 corrects the position information of the lesion detected during the previous examination, which is stored in the memory 24, specifically, the direction information and distance information from the landmark. The direction / distance information corrector 30 outputs the corrected direction information and distance information from the landmark detected during the previous examination to the image combiner 25.

[0070] The virtual model acquisition unit 29 reflects the previous lesion position corrected by the direction / distance information correction unit 30 in addition to the current observation position in the virtual model, and outputs it to the image synthesis unit 25.

[0071] The image synthesis unit 25 synthesizes the endoscopic image output from the image processing unit 21, the virtual model output from the virtual model acquisition unit 29, information on the current observation position output from the position calculation unit 27, and information on the previous lesion position output from the direction / distance information correction unit 30, and outputs the synthesized image to the monitor 4.

[0072] 7 is a diagram showing an example of a display screen displayed on the monitor in the third embodiment. Through the synthesis process of the image synthesis unit 25, the endoscopic image 51 output from the image processing unit 21, direction information 61 and distance information 62 from a landmark (e.g., the right ureteral opening 45) output from the position calculation unit 27, the virtual model 41A output from the virtual model acquisition unit 29, and position information of the lesion during the previous examination corrected by the direction / distance information correction unit 30 (specifically, direction information 63 and distance information 64 from the landmark), are synthesized and displayed on the monitor 4.

[0073] The virtual model 41A reflects the current observation position as a filled-in star mark 47, and also reflects the lesion position detected in the previous examination as corrected by the direction / distance information correction unit 30 as a hollow star mark 48.

[0074] Furthermore, the monitor 4 displays the current observation position information (direction information 61 and distance information 62) based on the landmark as the base point, as well as the position information of the lesion detected in the previous examination (direction information 63 and distance information 64). The position information of the lesion detected in the previous examination has been corrected by the direction / distance information corrector 30 in accordance with the expansion and contraction of the bladder. Therefore, the user can direct the tip of the endoscope to the position of the lesion detected in the previous examination based on the current observation position information and the corrected position information of the lesion.

[0075] Next, a display information generation process of the endoscope system 100B configured as described above will be described. Fig. 8 is a flowchart showing an example of the flow of the display information generation process of the third embodiment. Note that the display information generation process shown in Fig. 8 is constantly executed during the endoscopic examination and ends when the endoscopic examination is completed.

[0076] First, the control unit 26B determines whether or not at least two landmarks have been detected (S11). The process of S11 is performed by the control unit 26B based on the detection result of the landmark detection unit 23.

[0077] If the control unit 26B determines that at least two landmarks have not been detected (S11: NO), the control unit 26B returns to the process of S11 and repeats the same process. On the other hand, if the control unit 26B determines that at least two landmarks have been detected (S11: YES), the control unit 26B calculates the distance between the landmarks (S12). The process of S12 is calculated by the landmark distance calculation unit 28 of the control unit 26B. For example, if the right ureteral orifice 45 and the left ureteral orifice 46 are detected as landmarks, the distance between the right ureteral orifice 45 and the left ureteral orifice 46 is calculated.

[0078] Next, the control unit 26B compares the distance between the landmarks calculated during the current examination with the distance between the landmarks calculated during the previous examination (S13). The distance between the landmarks calculated during the previous examination is stored in the memory unit 24. Therefore, the control unit 26B reads the distance between the landmarks calculated during the previous examination from the memory unit 24 and compares the read distance between the landmarks with the distance between the landmarks calculated during the current examination. Note that the two landmarks used when calculating the distance between the landmarks in the process of S12 are the same landmarks (e.g., the right ureteral orifice 45 and the left ureteral orifice 46) as the two landmarks used when calculating the distance between the landmarks stored in the memory unit 24.

[0079] Next, the control unit 26B corrects the position information (direction information and distance information from the landmark) of the lesion detected during the previous examination based on the comparison result (S14). This correction process is performed by the direction / distance information corrector 30 of the control unit 26B. The position information of the lesion detected during the previous examination is stored in the memory unit 24 by the process of S6 in FIG. 4.

[0080] Next, the control unit 26B calculates the direction information and distance information of the current observation position from the landmark (S15). This calculation process is performed by the position calculation unit 27 of the control unit 26B.

[0081] Next, the control unit 26B displays on the monitor 4 the corrected position information of the lesion detected during the previous examination, and the direction information and distance information of the current observation position from the calculated landmark (S16). Thereafter, the process returns to S15 and repeats the same process.

[0082] Through the above processing, in addition to direction information 61 and distance information 62 of the current observation position from the landmark, direction information 63 and distance information 64 from the landmark are displayed on the monitor 4 as corrected position information of the lesion detected during the previous examination.

[0083] As described above, in the display information generation process of the third embodiment, the previous (past) position information of the lesion is corrected according to the current state of the bladder, specifically, the state of expansion and contraction of the bladder, and displayed on the monitor 4. As a result, the user can accurately guide the tip of the endoscope to the position of the lesion discovered in the previous observation.

[0084] It should be noted that the steps in the flowcharts in this specification may be executed in a different order, may be executed multiple times simultaneously, or may be executed in a different order each time, as long as this does not contradict the nature of the steps.

[0085] The present invention is not limited to the above-described embodiments, and it goes without saying that various modifications, combinations, and applications are possible within the scope of the invention without departing from the spirit of the invention.

[0086] Furthermore, among the technologies described herein, many of the controls and functions, mainly those described in the flowcharts, can be set by a program, and the above-described controls and functions can be realized by a computer reading and executing the program. The program can be recorded or stored, in whole or in part, as a computer program product on a portable medium such as a flexible disk, CD-ROM, or nonvolatile memory, or on a storage medium such as a hard disk or volatile memory, and can be distributed or provided at the time of product shipment, via a portable medium, or via a communication line. A user can easily realize the display information generation program of this embodiment by downloading the program via a communication network and installing it on a computer, or by installing it on a computer from a recording medium.

Claims

1. The diagnosis support device has a processor, which acquires position information of a detection target within the body, acquires position information of landmarks within the body, and outputs directional information of the detection position of the detection target relative to the landmarks.

2. In the diagnostic support device described in claim 1, the processor acquires information about an organ model in which a diagram of an organ is depicted and in which the landmarks are depicted within the organ, determines the left-right and up-down directions when displaying the organ model, aligns the landmarks of the organ model with landmarks within the body, outputs information about the organ model that reflects the detected position of the detection target, and outputs directional information about the detected position relative to the landmarks on the organ model.

3. A diagnosis support device according to claim 2, wherein the organ is a bladder or a uterus.

4. A diagnosis support device according to claim 3, wherein, when the organ is the bladder, the landmark is a ureteral orifice or a urethral orifice, and when the organ is the uterus, the landmark is a fallopian tube orifice.

5. A diagnosis support device according to claim 1, wherein the processor outputs distance information of the detected position relative to the landmark.

6. In the diagnostic support device described in claim 2, the processor acquires information on the observation position of the endoscope inserted into the body, reflects the observation position in the organ model, and outputs directional information of the observation position relative to the landmark on the organ model.

7. A diagnosis support device according to claim 6, wherein the processor outputs distance information of the observation position relative to the landmark.

8. In the diagnostic support device described in claim 1, the direction information is indicated by the clockwise intersection angle between a first line drawn up and down on the organ model so as to cross the landmark, and a second line drawn on the organ model so as to cross both the landmark and the detection target, after determining the left-right, up-down and down directions when displaying the organ model.

9. A diagnosis support device according to claim 1, wherein the processor updates the direction information in accordance with deformations within the body.

10. A diagnostic support device comprising: a first position information acquisition unit that acquires position information of a detection target within the body; a second position information acquisition unit that acquires position information of landmarks within the body; an organ model information acquisition unit that acquires information about an organ model in which a diagram of an organ is depicted and the landmarks are further depicted within the organ; an alignment unit that determines the left-right, up-down, and right-left directions when displaying the organ model and then aligns the landmarks on the organ model with landmarks within the body; a first output unit that outputs information about the organ model that reflects the detection position of the detection target; and a second output unit that outputs directional information of the detection position relative to the landmarks on the organ model.

11. An endoscopic device comprises: the diagnostic support device according to claim 1; an endoscope that directly or indirectly transmits endoscopic image data to the diagnostic support device; and a display that receives a signal output from the diagnostic support device and displays an organ model.

12. A display information generation method for displaying the direction toward a detection target on an observation target model includes: a first position information acquisition unit acquires position information of the detection target inside the body; a second position information acquisition unit acquires position information of landmarks inside the body; an organ model information acquisition unit acquires information on an organ model in which a diagram simulating an organ is drawn and the landmarks are further drawn inside the organ; an alignment unit determines the left / right, up / down and left / right directions when displaying the organ model, and then aligns the landmarks on the organ model with the landmarks inside the body; a first output unit outputs information on the organ model reflecting the detection position of the detection target; and a second output unit outputs direction information of the detection position relative to the landmarks on the organ model.

13. A display information generation program for displaying the direction toward a detection target on an observation target model causes a processor to perform the following processes: acquire position information of the detection target inside the body; acquire position information of landmarks inside the body; acquire information on an organ model in which a diagram of an organ is drawn and the landmarks are further drawn inside the organ; determine the left-right, up-down, and right-left directions when displaying the organ model; align the landmarks on the organ model with the landmarks inside the body; output information on the organ model that reflects the detection position of the detection target; and output direction information of the detection position relative to the landmarks on the organ model.

14. In a non-volatile storage medium storing a display information generation program for displaying the direction toward a detection target on an observation target model, the program causes a processor to execute the following processes: acquire position information of the detection target inside the body; acquire position information of landmarks inside the body; acquire information on an organ model in which a diagram of an organ is depicted and the landmarks are further depicted inside the organ; determine the left-right, up-down, and right-left directions when displaying the organ model; align the landmarks on the organ model with the landmarks inside the body; output information on the organ model that reflects the detection position of the detection target; and output direction information of the detection position relative to the landmarks on the organ model.

Images