Endoscope system, processor device, and operation method thereof
The endoscope system addresses the challenge of seamlessly switching and displaying multiple image types by using a processor device to control illumination and display frame rates, ensuring smooth observation of images of interest in endoscopic examinations.
Patent Information
- Application Number
- JP2021123622
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-07-28
- Publication Date
- 2024-11-13
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing endoscopy systems struggle to simultaneously and smoothly observe multiple types of images obtained using different illumination lights, as the focus often shifts between normal light images for understanding structures and special light images for detailed examination, requiring seamless transitions between these views.
An endoscope system with a processor device that includes a light source processor and an image processing processor, capable of automatically switching between different illumination periods and types, and controlling the display of multiple images with adjustable frame rates and attention sections to ensure smooth observation of images of interest.
Enables seamless switching and display of multiple image types, allowing for focused observation on particular images without stutter, enhancing the efficiency and clarity of endoscopic examinations.
Smart Images

Figure 2024160185000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an endoscope system for acquiring a plurality of types of images and displaying different images among images having duplicate sections, a processor device, and an operating method thereof. [Background technology]
[0002] In endoscopic examinations, a subject is illuminated with a plurality of illumination lights having different wavelength ranges depending on the purpose of diagnosis. For example, in addition to normal light that allows the mucous membrane of the digestive tract to be observed in natural colors, in order to obtain information on the surface layer such as superficial blood vessels and information on the deep layer such as deep blood vessels, short-wavelength light that has a depth of penetration to the surface layer and medium-wavelength light that has a depth of penetration to the deep layer are switched for illumination, and a normal light image obtained by illumination with normal light and a surface image obtained by illumination with short-wavelength light and a deep image obtained by illumination with medium-wavelength light are obtained as special light images.
[0003] A technology is known in which illumination is switched between a first illumination light that emphasizes superficial blood vessels and a second illumination light that emphasizes deep blood vessels, and a first special observation image obtained by illumination with the first illumination light and a second special observation image obtained by illumination with the second illumination light are switched and displayed sequentially (Patent Document 1). [Prior art documents] [Patent documents]
[0004] [Patent Document 1] International Publication No. 2020 / 158165 Summary of the Invention [Problem to be solved by the invention]
[0005] There is a demand for independent and simultaneous observation of images obtained using multiple types of illumination light. Even if multiple types of images are acquired, the type of image that one wishes to focus on varies depending on the situation. For example, when advancing an endoscope inside the digestive tract or performing screening, it is better to be able to focus on normal light images, which have natural light and brightness and make it easier to grasp the structure inside the lumen. On the other hand, when one wishes to closely examine and classify the structure of a lesion, it is better to be able to focus on special light images. Thus, when there is an image that one wishes to focus on in particular, there is a demand for smooth observation of that image without stuttering.
[0006] The present invention aims to provide an endoscopic system, a processor device, and an operating method thereof that enable smooth observation of images of particular interest when switching between multiple lights for illumination and displaying multiple images obtained by each light. [Means for solving the problem]
[0007] The processor device of the present invention is a processor device including a light source processor and an image processing processor, in which the light source processor performs control to automatically switch between a first illumination period in which an image of an object illuminated with a first illumination light is captured, and a second illumination period in which an image of an object illuminated with at least one type of second illumination light having a spectrum different from that of the first illumination light is captured, and the image processing processor acquires a first illumination light image as a medical image in the first illumination period, acquires a second illumination light image as a medical image in the second illumination period, generates a display image for displaying the first illumination light image and / or the second illumination light image, and performs control to automatically switch between a first illumination period and a second illumination light period in which an image of an object is captured. The display image has one attention section and at least one sub-attention section, the attention section display frame rate, which is the number of medical images displayed in the attention section per unit time, is higher than the sub-attention section display frame rate, which is the number of medical images displayed in the sub-attention section per unit time, the medical images displayed in the attention section are first illumination light images or second illumination light images, the number of display images displayed per unit time on the display that displays the display images is the overall display frame rate, and the attention section display frame rate and the sub-attention section display frame rate can be set arbitrarily within the range of the overall display frame rate.
[0008] It is preferable that the image processing processor sets the attention section imaging frame rate, which is the number of attention medical images acquired per unit time, for the attention section display frame rate, to be greater than or less than the attention section display frame rate as medical images to be displayed in the attention section, and acquires the attention medical images based on the attention section imaging frame rate, and sets the medical images to be displayed in the sub-attention section as quasi-attention medical images, and sets the quasi-attention section imaging frame rate, which is the number of quasi-attention medical images acquired per unit time, for the quasi-attention section display frame rate, to be greater than or less than the quasi-attention section display frame rate, and acquires the quasi-attention medical images based on the quasi-attention section imaging frame rate.
[0009] It is preferable that the image processing processor uses the medical image of interest to generate an interpolated frame image to be displayed in the sub-region of interest when the region of interest display frame rate is higher than the region of interest imaging frame rate, or uses the sub-region of interest medical image to generate an interpolated frame image to be displayed in the sub-region of interest when the region of interest display frame rate is higher than the sub-region of interest imaging frame rate.
[0010] The interpolated frame image is preferably generated by an averaging method using at least two or more medical images of interest or quasi-medical images of interest.
[0011] The interpolated frame image is preferably generated by a motion vector method using at least two or more medical images of interest or semi-medical images of interest.
[0012] The interpolated frame images are preferably generated by replicating the medical image of interest or the sub-medical image of interest.
[0013] It is preferable that the image processing processor selects a medical image of interest to be displayed in the region of interest when the region of interest display frame rate is lower than the region of interest imaging frame rate, or selects a sub-medical image of interest to be displayed in the sub-region of interest when the sub-region of interest display frame rate is lower than the sub-region of interest imaging frame rate.
[0014] It is preferable that the image processing processor detects a region of interest, which is an area on which the user should focus, from the featured medical image and / or quasi-featured medical image and identifies the region of interest from the featured medical image and / or quasi-featured medical image, or detects a region of interest from the featured medical image and / or quasi-featured medical image and identifies the region of interest.
[0015] It is preferable that the image processing processor controls the display of a detection result, which is the result of detecting the area of interest, a discrimination result, which is the result of discriminating the area of interest, or a detection result and a discrimination result in the area of interest and / or sub-area of interest.
[0016] It is preferable that the sub-interest medical image displayed in the sub-interest section can be switched between display and non-display.It is preferable that the size of the interest section and the sub-interest section in the display image can be changed arbitrarily.
[0017] It is preferable that the image processing processor be operable by user operation to switch between a mono-light emission mode in which a medical image is obtained by capturing an image of a subject illuminated with either the first illumination light or at least one type of second illumination light, and a multi-light emission mode in which a medical image is obtained by automatically switching between the first illumination period and the second illumination period.
[0018] A method of operating a processor device of the present invention includes the steps of: performing control to automatically switch between a first illumination period in which an image of a subject illuminated with a first illumination light is captured, and a second illumination period in which an image of a subject illuminated with at least one type of second illumination light having a spectrum different from that of the first illumination light; acquiring a first illumination light image as a medical image in the first illumination period; acquiring a second illumination light image as a medical image in the second illumination period; and generating a display image for displaying the first illumination light image and / or the second illumination light image, wherein the display image has one attention section and at least one or more sub-attention sections, a attention section display frame rate which is the number of medical images displayed in the attention section per unit time is higher than a sub-attention section display frame rate which is the number of medical images displayed in the sub-attention section per unit time, and the medical images displayed in the attention section are the first illumination light image or the second illumination light image, and the number of display images displayed per unit time on a display for displaying the display image is an overall display frame rate, and the attention section display frame rate and the sub-attention section display frame rate can be set arbitrarily within the range of the overall display frame rate.
[0019] An endoscope system of the present invention includes the above-mentioned processor device, a light source device, and an endoscope. Effect of the Invention
[0020] According to the present invention, it is possible to provide an endoscopic system, a processor device, and an operating method thereof that enable smooth observation of images of particular interest when multiple lights are switched for illumination and multiple images obtained by each light are switched and displayed. [Brief description of the drawings]
[0021] [Figure 1] FIG. 1 is an explanatory diagram of a configuration of an endoscope system. [Diagram 2] 2 is a block diagram showing the functions of the endoscope system. FIG. [Diagram 3] 13 is a graph showing the spectrum of a first illumination light. [Figure 4] 13 is a graph showing a spectrum SP of the first spectrum light of the second illumination. [Diagram 5] 13 is a graph showing a spectrum SQ of the second spectrum light of the second illumination. [Figure 6] 13 is a graph showing a spectrum SR of the second illumination third spectrum light. [Figure 7] 13 is a graph showing a spectrum SS of the second illumination, fourth spectrum light. [Figure 8] FIG. 11 is an explanatory diagram showing a first light emission pattern in a multiple light emission mode. [Figure 9] FIG. 11 is an explanatory diagram showing a second light emission pattern in a multiple light emission mode. [Figure 10] FIG. 13 is an explanatory diagram showing a third light emission pattern in the multi-light emission mode. [Figure 11] FIG. 11 is an image diagram showing an example of a display image. [Figure 12] FIG. 2 is a block diagram showing functions of a display control unit. [Figure 13] FIG. 13 is an image diagram showing an example of a display image setting screen. [Figure 14] FIG. 11 is an image diagram showing a display image in which the focus area and the sub-focus area are the same size. [Figure 15] FIG. 13 is an image diagram showing an example of a partition display frame rate setting screen. [Figure 16]FIG. 13 is an image diagram showing a display image when a first illumination light image is displayed in a region of interest and a second illumination light image is displayed in a sub-region of interest. [Figure 17] FIG. 13 is an image diagram showing an example of a section image capture frame rate setting screen. [Figure 18] FIG. 13 is an explanatory diagram showing an averaging method. [Figure 19] FIG. 13 is an explanatory diagram showing a motion vector method. [Figure 20] FIG. 1 is an explanatory diagram showing a simple copy method. [Figure 21] FIG. 13 is an explanatory diagram showing selection of a target medical image or a semi-target medical image to be displayed. [Figure 22] FIG. 2 is a block diagram showing the functions of a lesion recognition unit. [Diagram 23] FIG. 13 is an explanatory diagram showing that when a medical image is input to a detection classification, a detection result is output. [Figure 24] 10 is an explanatory diagram showing how a medical image is input to a classification for differentiation and then a differentiation result is output; FIG. [Diagram 25] FIG. 11 is an image diagram showing a display image when displaying a detection result. [Figure 26] FIG. 13 is an image diagram showing a display image when displaying a discrimination result. [Figure 27] 11 is an image diagram showing a display image when a still image and character information are displayed. FIG. [Figure 28] 13 is an explanatory diagram of an example of how a display image is displayed when an extended processor device is provided; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0022] 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, and a user interface 16. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14.
[0023] The endoscope 12 has an insertion section 12a, an operating section 12b, a bending section 12c, and a tip section 12d. The insertion section 12a is inserted into the body of the subject. The operating section 12b is provided at the base end portion of the insertion section 12a. The bending section 12c and the tip section 12d are provided on the tip side of the insertion section 12a. The bending section 12c is bent by operating an angle knob 12e of the operating section 12b. The tip section 12d is directed in a desired direction by the bending operation of the bending section 12c. A forceps channel (not shown) for inserting a treatment tool or the like is provided from the insertion section 12a to the tip section 12d. The treatment tool is inserted into the forceps channel from the forceps port 12j.
[0024] An optical system for forming an image of a subject and an optical system for irradiating illumination light onto the subject are provided inside the endoscope 12. The operation unit 12b is provided with an angle knob 12e, a mode changeover switch 12f, a still image acquisition instruction switch 12h, and a zoom operation unit 12i. The mode changeover switch 12f is used to change the observation mode. The still image acquisition instruction switch 12h is used to instruct acquisition of a still image of the subject. The zoom operation unit 12i is used to operate the zoom lens 42.
[0025] The light source device 13 generates illumination light. The display 15 displays medical images. The medical images include a first illumination light image and a second illumination light image that are captured using different illumination lights, as well as a target medical image and a quasi-target medical image that have different display sections. The user interface 16 has a keyboard, mouse, microphone, tablet, touch pen, etc., and accepts input operations such as function settings. The processor device 14 performs system control of the endoscope system 10, and further performs image processing, etc. on image signals transmitted from the endoscope 12.
[0026] In FIG. 2, the light source device 13 includes a light source section 20, a light source processor 21 that controls the light source section 20, and an optical path coupling section 22. The light source section 20 has a plurality of semiconductor light sources, which are turned on or off. When the plurality of semiconductor light sources are turned on, the light emission amount of each semiconductor light source is controlled to emit illumination light that illuminates the subject. The light source section 20 has four color LEDs, a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode) 20d. The light source section 20 or the light source processor 21 may be built into the endoscope 12. The light source processor 21 may be built into the processor device 14.
[0027] The endoscope system 10 has a mono-emission mode and a multi-emission mode, and the mode can be switched via the central control unit 50 by operating the mode changeover switch 12f. The mono-emission mode is a mode in which illumination light of the same spectrum is continuously irradiated to illuminate the subject to be observed. The multi-emission mode is a mode in which multiple illumination lights of different spectrums are irradiated while being switched according to a specific pattern to illuminate the subject. The illumination light includes a first illumination light and a second illumination light having a spectrum different from that of the first illumination light. The first illumination light is normal light (broadband light such as white light) used to give brightness to the entire subject for screening observation. The second illumination light is multiple types of special light used to highlight specific structures such as glandular ducts and blood vessels of the mucous membrane that is the subject. In the mono-emission mode, it is preferable that the first illumination light and the second illumination light can be switched.
[0028] When emitting the first illumination light, as shown in Fig. 3, the V-LED 20a emits purple light V with a central wavelength of 405±10 nm and a wavelength range of 380 to 420 nm. The B-LED 20b emits blue light B with a central wavelength of 450±10 nm and a wavelength range of 420 to 500 nm. The G-LED 20c emits green light G with a wavelength range of 480 to 600 nm. The R-LED 20d emits red light R with a central wavelength of 620 to 630 nm and a wavelength range of 600 to 650 nm.
[0029] Examples of the second illumination light include second illumination first spectrum light that emphasizes superficial blood vessels, second illumination second spectrum light that emphasizes very superficial blood vessels that are shallower than superficial blood vessels, second illumination third spectrum light for generating an oxygen saturation image that utilizes the difference in absorption coefficients between oxyhemoglobin and deoxyhemoglobin, and second illumination fourth spectrum light for generating a color difference enhanced image that enhances the color difference between multiple subject ranges.
[0030] When the second illumination emits the first spectrum light, as shown in FIG. 4, the light is emitted in a spectrum SP in which the violet light V has a higher peak intensity than the other colors of blue light B, green light G, and red light R. When the second illumination emits the second spectrum light, as shown in FIG. 5, the light is emitted in a spectrum SQ including the violet light V, which is a light in a wavelength range in which there is a difference in the absorption coefficient between oxygenated hemoglobin and reduced hemoglobin. When the second illumination emits the third spectrum light, as shown in FIG. 6, the light is emitted in a spectrum SR including the blue-green light B (peak wavelength is, for example, 470 to 480 nm), which is a light in a wavelength range in which there is a difference in the absorption coefficient between oxygenated hemoglobin and reduced hemoglobin. When the second illumination emits the fourth spectrum light, as shown in FIG. 7, the light is emitted in a spectrum SS in which the peak intensities of the violet light V and blue light B are higher than the peak intensities of the green light G and red light R. Note that the type of the second illumination light is not limited to this.
[0031] The light source processor 21 independently controls the amount of light of four colors: purple light V, blue light B, green light G, and red light R, and emits first illumination light and second illumination light (e.g., second illumination first spectrum light, second illumination second spectrum light, second illumination third spectrum light, and second illumination fourth spectrum light) by changing the amount of light.
[0032] In the mono emission mode, illumination light of the same spectrum is continuously emitted for each frame. For example, a first illumination light image with natural coloring is displayed on the display 15 by illuminating a subject with a first illumination light for each frame and capturing an image. A second illumination light image with a specific structure emphasized is displayed on the display 15 by illuminating a subject with a second illumination light for each frame and capturing an image. Note that a frame refers to a unit of time that includes at least the period from the timing of light emission to the completion of image signal readout by the imaging sensor 43.
[0033] In the case of the multi-emission mode, control is performed to change the light intensities of the purple light V, blue light B, green light G, and red light R for each specific frame F according to a specific emission pattern. Examples of emission patterns are given below. For example, in the first emission pattern, as shown in FIG. 8, in a first illumination period Pe1 in which the subject is illuminated with the first illumination light L1, two frames of the first illumination light L1 are emitted, and in a second illumination period Pe2 in which the subject is illuminated with the second illumination light L2, one frame of the second illumination light (second illumination first spectrum light L2SP) is emitted in sequence. In the figure, the arrow indicates the direction in which time progresses.
[0034] 9, in the second emission pattern, the first illumination light L1 is emitted for two frames in the first illumination period Pe1, the second illumination light L2 (second illumination first spectrum light L2SP) is emitted for one frame in the second illumination period Pe2, and then the first illumination light L1 is emitted for two frames in the first illumination period Pe1, and the second illumination light L2 (second illumination first spectrum light L2SQ) is emitted for one frame in the second illumination period Pe2. In the second emission pattern, unlike the first emission pattern, the spectrum of the second illumination light emitted after the first illumination light is changed every time.
[0035] In the third emission pattern, as shown in Fig. 10, the first illumination light L1 is emitted for one frame in the first illumination period Pe1, and the second illumination light L2 is emitted for four frames in the second illumination period Pe2, and this pattern is repeated. At this time, in the second illumination period Pe2, the second illumination light L2 is automatically switched so that the second illumination light L2 having a different spectrum is emitted for each frame, such as second illumination first spectrum light L2SP, second illumination second spectrum light L2SQ, second illumination third spectrum light L2SR, and second illumination fourth spectrum light L2SS.
[0036] In the multi-flash mode, the illumination light is automatically switched, so that images of the subject illuminated with different illumination lights can be obtained at approximately the same position in the digestive tract. In the multi-flash mode, the light source processor 21 adjusts the light amount of each light source according to the shooting frame rate setting and the light emission pattern setting, which will be described later, to perform shooting. The light emission pattern is not limited to the above three patterns, and can be set arbitrarily, as will be described later.
[0037] In the mono-flash mode and the multi-flash mode, if the user wishes to capture a medical image as a still image, the still image capture instruction switch 12h is operated, whereby a signal regarding a still image capture instruction is sent to the endoscope 12, the light source device 13, and the processor device 14.
[0038] The light emitted by each of the LEDs 20a to 20d (see FIG. 2) is incident on the light guide 23 via an optical path combining unit 22 constituted by a mirror, a lens, etc. The light guide 23 propagates the light from the optical path combining unit 22 to the tip 12d of the endoscope 12.
[0039] The tip 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination optical system 30a has an illumination lens 31, and illumination light propagated by the light guide 23 is irradiated onto the subject via the illumination lens 31. On the other hand, when the light source unit 20 is built into the tip 12d of the endoscope 12, the illumination light is emitted toward the subject via the illumination lens of the illumination optical system without passing through the light guide. The imaging optical system 30b has an objective lens 41 and an imaging sensor 43. Light from the subject irradiated with illumination light is incident on the imaging sensor 43 via the objective lens 41 and the zoom lens 42. As a result, an image of the subject is formed on the imaging sensor 43. The zoom lens 42 is a lens for enlarging the subject, and is moved between the telephoto end and the wide end by operating the zoom operation unit 12i.
[0040] The imaging sensor 43 is a primary color sensor and includes three types of pixels: B pixels (blue pixels) having blue color filters, G pixels (green pixels) having green color filters, and R pixels (red pixels) having red color filters.
[0041] Moreover, the imaging sensor 43 is preferably a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The imaging processor 44 controls the imaging sensor 43. Specifically, the imaging processor 44 reads out the signal of the imaging sensor 43, thereby outputting an image signal from the imaging sensor 43. The output image signal is transmitted to the medical image acquisition unit 60 of the processor device 14.
[0042] The medical image acquisition unit 60 performs various signal processing such as defect correction processing, offset processing, demosaic processing, matrix processing, white balance adjustment, gamma conversion processing, and YC conversion processing on the received color image. Next, image processing including color conversion processing such as 3×3 matrix processing, tone conversion processing, and 3D LUT (Look Up Table) processing, color enhancement processing, and structure enhancement processing such as spatial frequency enhancement is performed to acquire a first illumination light image for the first illumination light and a second illumination light image for the second illumination light.
[0043] Furthermore, when the second illumination light is the second illumination third spectrum light, the medical image acquisition section 60 generates an oxygen saturation image as the second illumination light image based on the image signal ratio from the imaging sensor 43. Furthermore, when the second illumination light is the second illumination fourth spectrum light, the medical image acquisition section 60 generates a color difference expanded image that has been subjected to processing to expand the color difference as the second illumination light image based on the image signal ratio from the imaging sensor 43.
[0044] The processor device 14 includes a central control unit 50, a medical image acquisition unit 60, a display control unit 70, an overall display frame rate recognition unit 90, a partition display frame rate setting unit 100, a partition shooting frame rate setting unit 110, an interpolated frame image generation unit 120, and a lesion recognition unit 130 (see FIG. 2).
[0045] In the processor device 14, a central control unit 50 consisting of a light source processor 21 and an image processing processor (not shown) operates programs in a program memory to realize the functions of a medical image acquisition unit 60, a display control unit 70, an overall display frame rate recognition unit 90, a partition display frame rate setting unit 100, a partition shooting frame rate setting unit 110, an interpolated frame image generation unit 120, and a lesion recognition unit 130.
[0046] The first illumination light image and / or the second illumination light image acquired by the medical image acquisition unit 60 is transmitted to the display control unit 70. The display control unit 70 generates a display image 71 as shown in FIG. 11, which displays the first illumination light image and / or the second illumination light image, and causes the display 15 to display the image. The display image 71 has one attention section 72 and at least one or more sub-attention sections 73. In the example of FIG. 11, the display image 71 has the attention section 72 and one sub-attention section 73. The attention section 72 is a section that displays the type of medical image that a user such as a doctor wants to pay the most attention to. The sub-attention section 73 is a section that displays a type of medical image other than the medical image displayed in the attention section.
[0047] The medical image to be displayed in the attention section can be selected to be either the first illumination light image or the second illumination light image. The type of medical image to be displayed in the attention section 72 and the sub-attention section 73 can be set via a display image setting section 80 of the display control section 70 shown in Fig. 12. The user can set the type of medical image to be displayed in the attention section 72 and the sub-attention section 73, and the number of sub-attention sections 73, using tabs 82 on a display image setting screen 81 shown in Fig. 13. To make the diagram easier to understand, only one tab 82 is labeled with a reference symbol.
[0048] 13, the number of sub-areas of interest 73 is 1. The type of medical image to be displayed in the area of interest 72 is a first illumination light image. The type of medical image to be displayed in the sub-area of interest 73 is a second illumination light image, and in this case, the second illumination light image is captured using the second illumination first spectrum light.
[0049] It is preferable that the medical image displayed in the sub-attention section (sub-attention medical image) can be switched between display and non-display. The setting of display or non-display of the sub-attention medical image displayed in the sub-attention section 73 is performed, for example, on the display image setting screen 81. In the specific example shown in FIG. 13, the display of the sub-attention medical image displayed in the sub-attention section is set to "on." The display or non-display of the sub-attention medical image may be switched on the display image 71.
[0050] It is preferable that the size of the attention section 72 and the sub-attention section 73 in the display image 71 can be changed arbitrarily. The layout such as the size and arrangement of the attention section 72 and the sub-attention section 73 may be set by selecting a layout setting button 83 on the display image setting screen 81. For example, the attention section 72 may be made larger than the sub-attention section 73 (see FIG. 11), or the attention section 72 and the sub-attention section 73 may be made the same size as each other as shown in FIG. 14. The size and arrangement of the attention section 72 and the sub-attention section 73 may be selected from a prepared layout template, or may be set by the user directly operating the display example 84 of the display image displayed on the display image setting screen 81 by pinching in, pinching out, dragging with two fingers, or the like.
[0051] The display frame rates of the focus area and the sub-focus area can each be set within the range of the overall display frame rate of the display 15. First, the overall display frame rate recognition unit 90 recognizes the frame rate of the display 15 connected to the processor device 14 as the overall display frame rate and transmits it to the area display frame rate setting unit 100. The overall display frame rate is the number of display images that the display 15 displays per unit time. The focus area display frame rate and the sub-focus area display frame rate may be set within the range of the refresh rate of the display 15. The refresh rate refers to the number of display images that can be processed for display per unit time.
[0052] The section display frame rate setting unit 100 sets the section of interest display frame rate and the sub-section of interest display frame rate within the range of the total display frame rate. The section of interest display frame rate is the number of medical images displayed in the section of interest per unit time. The sub-section of interest display frame rate is the number of medical images displayed in the sub-section of interest per unit time.
[0053] In order to obtain a smooth moving image without stuttering by displaying as many medical images as possible in the attention section, the attention section display frame rate is set higher than the sub-attention section table frame rate. Note that the sub-attention section table frame rate can be set higher than the attention section display frame rate, or the sub-attention section table frame rate and the attention section display frame rate can be set to the same value. Hereinafter, the term "section display frame rate" will be used to refer to the attention section display frame rate and the sub-attention section display frame rate collectively without distinction.
[0054] The partition display frame rate is set via a partition display frame rate setting screen 101. In the partition display frame rate setting screen 101 shown in Fig. 15, the overall display frame rate is displayed in an overall display frame rate display field 102. The user can set the partition display frame rate by inputting numerical values within the range displayed in the overall display frame rate display field 102 into a focus partition display frame rate setting field 103 and a sub-focus partition display frame rate setting field 104.
[0055] In the example of FIG. 15, the overall display frame rate is 60 fps (frames per second). In this case, the attention section display frame rate of the attention section and the sub-attention section display frame rates of the sub-attention sections indicated by "sub-attention section (1)" and "sub-attention section (2)" can each be set within a range not exceeding 60 fps. In the example of FIG. 15, the attention section display frame rate is set to 60 fps, and the sub-attention section display frame rate is set to 15 fps. Methods of inputting data into the attention section display frame rate setting field 103 and the sub-attention section display frame rate setting field 104 include, but are not limited to, user input using a keyboard or voice, input using tabs, automatic input of a predetermined value, etc.
[0056] The display control unit 70 displays a medical image (a first illumination light image or a second illumination light image) in each of the attention section 72 and the sub-attention section 73 of the display image 71 in accordance with the settings of the attention section display frame rate and the sub-attention section display frame rate of the section display frame rate setting unit 100. In the example of the display image 71 shown in Fig. 16, the first illumination light image is displayed in the attention section 72, and the second illumination light image (shown by diagonal lines) is displayed in the sub-attention section 73.
[0057] The configuration that allows the display frame rate of the region of interest and the display frame rate of the sub-region of interest to be set separately is particularly effective in the multi-emission mode in which multiple types of medical images can be obtained in sequence. With the above configuration, when multiple types of light are switched for illumination and multiple images obtained by each illumination light are switched for display, the type of image that is particularly of interest is displayed in the region of interest, and the frame rate of the region of interest is set higher than the frame rate of the sub-region of interest, so that the image of interest can be observed smoothly without stuttering.
[0058] The section imaging frame rate setting unit 110 sets the section of interest imaging frame rate and the sub-section of interest imaging frame rate. The overall imaging frame rate is an imaging frame rate determined by the performance of the imaging sensor 43, and is the number of medical images that can be acquired per unit time. If the medical images displayed in the section of interest are the medical images of interest, and the medical images displayed in the sub-section of interest are the medical images of sub-interest, the section of interest imaging frame rate is the number of medical images of interest acquired per unit time. Also, the sub-section of interest imaging frame rate is the number of medical images of sub-interest acquired per unit time. The section of interest imaging frame rate can be set to be equal to or greater than the section of interest display frame rate. Similarly, the sub-section of interest imaging frame rate can be set to be equal to or greater than the sub-section of interest display frame rate. The sum of the section of interest imaging frame rate and the sub-section of interest imaging frame rate is the overall imaging frame rate.
[0059] Hereinafter, the term "section imaging frame rate" will be used to refer collectively to the section of interest imaging frame rate and the sub-section of interest imaging frame rate without distinction. The section imaging frame rate is set via a section imaging frame rate setting screen 111. The entire imaging frame rate is displayed in an entire imaging frame rate display field 112 of the section imaging frame rate setting screen 111 illustrated in Fig. 17. The user can set the section imaging frame rate by inputting values within the range displayed in the entire imaging frame rate display field 112 into a section of interest imaging frame rate setting field 113 and a sub-section of interest imaging frame rate setting field 114.
[0060] In the example of Figure 17, the overall imaging frame rate is 60 fps, the imaging frame rate of the area of interest (the medical image of interest to be displayed in the area of interest) is set to 30 fps, and the imaging frame rate of each of the sub-areas of interest (the medical image of interest to be displayed in the area of interest) indicated as "sub-area of interest (1)" and "sub-area of interest (2)" is set to 15 fps.
[0061] Furthermore, the type of illumination light used to acquire the target medical image and the quasi-target medical image can be set on the section imaging frame rate setting screen 111. The type of illumination light used to acquire the target medical image and the quasi-target medical image can be set on each of the tabs 82. In the example of FIG. 17, it is set that the target medical image uses the first illumination light, the first quasi-target medical image uses the first spectrum light of the second illumination, and the second quasi-target medical image uses the second spectrum light of the second illumination. That is, the target medical image is a first illumination light image, the first quasi-target medical image is a second illumination light image illuminated with the first spectrum light of the second illumination, and the second quasi-target medical image is a second illumination light image illuminated with the second spectrum light of the second illumination. Furthermore, on the section imaging frame rate setting screen 111, the light emission pattern of the illumination light used to acquire the target medical image and the quasi-target medical image can be set in the light emission pattern setting field 115. In the example of FIG. 17, the light emission pattern is set to "1, 1, 2, 1, 1, 3." This indicates that the light emission pattern of "first illumination light, first illumination light, second illumination first spectrum light, first illumination light, first illumination light, second illumination second spectrum light" is repeated and emitted sequentially in each frame F. The setting of the light emission pattern set on the section imaging frame rate setting screen 111 is transmitted to the light source processor 21 via the central control unit 50.
[0062] Table 1 shows specific examples of the overall display frame rate, section display frame rate, overall imaging frame rate, and section imaging frame rate. In Table 1, "Overall" under "Display Frame Rate" refers to the "Overall Display Frame Rate", "Section of Interest" refers to the "Section of Interest Display Frame Rate", and "Sub-Section of Interest" refers to the "Sub-Section of Interest Display Frame Rate". Also, under "Imaging Frame Rate", "Overall" refers to the "Overall Imaging Frame Rate", "Section of Interest" refers to the "Section of Interest Imaging Frame Rate", and "Sub-Section of Interest" refers to the "Sub-Section of Interest Imaging Frame Rate". The unit of the column with only numerical values is fps. Also, in Examples 1 to 6 of Table 1, the medical image of interest is the first illumination light image, and the sub-medical image of interest is the second illumination light image.
[0063] [Table 1]
[0064] Example 1 in Table 1 is an example in which the overall display frame rate (60 fps) is higher than the overall imaging frame rate (45 fps) and the interest section imaging frame rate (30 fps) is higher than the sub-interest section imaging frame rate (15 fps).
[0065] Example 2 in Table 1 is an example in which the overall display frame rate (60 fps) is higher than the overall imaging frame rate (45 fps) and the attention section imaging frame rate (15 fps) is lower than the sub-attention section imaging frame rate (30 fps). "15 fps x 2" in the "Notes on sub-attention section" column indicates that there are two types of second illumination light, and second illumination light images for each type of second illumination light are acquired at 15 fps. In Example 2, second illumination light images captured at a sub-attention section imaging frame rate of 15 fps are displayed in two sub-attention sections. For example, a second illumination light image captured at a sub-attention section imaging frame rate of 15 fps using second illumination first spectrum light is displayed in one sub-attention section (sub-attention section display frame rate is 15 fps). In the other sub-region of interest (region display frame rate is 15 fps), a second illumination light image captured using second illumination second spectrum light at a sub-region of interest imaging frame rate of 15 fps is displayed.
[0066] Example 3 in Table 1 is an example in which the overall display frame rate (60 fps) is lower than the overall imaging frame rate (90 fps) and the section of interest imaging frame rate (60 fps) is higher than the sub-section of interest imaging frame rate (30 fps). "15 fps x 2" in the "Notes on sub-section of interest" column indicates that there are two types of second illumination light, and second illumination light images for each type of second illumination light are acquired at 15 fps. The relationship between the section imaging frame rate and the section frame rate of the sub-section of interest medical image is the same as in Example 2.
[0067] Example 4 in Table 1 is an example in which the overall display frame rate (60 fps) is lower than the overall imaging frame rate (90 fps), and the section of interest imaging frame rate (30 fps) is lower than the sub-section of interest imaging frame rate (60 fps). "15 fps x 4" in the "Notes on sub-section of interest" column indicates that there are four types of second illumination light, and the second illumination light images for each type of second illumination light are acquired at 15 fps. The relationship between the section imaging frame rate and the section frame rate of the sub-section of interest medical image is similar to the example in Example 2 in which the second illumination light images captured using two types of second illumination light are displayed in two sub-sections of interest. That is, the second illumination light images captured using four types of second illumination light at a sub-section of interest imaging frame rate of 15 fps are displayed in four sub-sections of interest (sub-section of interest display frame rate is 15 fps).
[0068] Example 5 in Table 1 is an example in which the sub-attention section display frame rate (45 fps) is higher than the attention section image capture frame rate (15 fps), and an example in which an interpolated frame image, which will be described later, is required for the sub-attention section.
[0069] Example 6 in Table 1 is an example in which the segment of interest display frame rate (60 fps) is lower than the segment of interest imaging frame rate (90 fps), and is an example in which it is necessary to select a segment of interest medical image to be displayed in the segment of interest, as described later.
[0070] As in the first to sixth embodiments, regardless of the relationship between the overall imaging frame rate and the overall display frame rate, an image of particular interest can be set as a focus area and the frame rate of the focus area can be made higher than the frame rate of the sub-focus area, thereby enabling smooth observation of the image of particular interest.
[0071] The interpolated frame image generating unit 120 preferably generates an interpolated frame image to be displayed in the section of interest using the medical image of interest. Specifically, as in Examples 1, 2, 4, and 5, the interpolated frame image is generated when the section of interest display frame rate is higher than the section of interest imaging frame rate. That is, the interpolated frame image is generated when the captured medical image of interest does not satisfy the section of interest display frame rate.
[0072] In addition, it is preferable that the interpolated frame image generating unit 120 generates an interpolated frame image to be displayed in the quasi-attention section by using the quasi-attention medical image. Specifically, as in the sixth embodiment, the interpolated frame image is generated when the quasi-attention section display frame rate is higher than the quasi-attention section image capture frame rate. That is, the interpolated frame image is generated when the captured quasi-attention medical image does not satisfy the quasi-attention section display frame rate.
[0073] With the above configuration, when the section display frame rate is higher than the section imaging frame rate, an interpolated frame image is generated and displayed, so that an image of particular interest is set as the section of interest, and an image that interpolates the image of particular interest is displayed, allowing for smooth observation. Note that in the third embodiment, since the section of interest display frame rate and the section of interest imaging frame rate are 60 fps, an interpolated frame image is not generated, and the captured medical images of interest are displayed in sequence in the section of interest.
[0074] The interpolated frame image is preferably generated by an averaging method using at least two or more frames of the medical image of interest or the quasi-medical image of interest. The averaging method is a method for generating an interpolated frame image by superimposing a plurality of frames of the medical image of interest or the quasi-medical image of interest captured in the past.
[0075] The averaging method will be described using a specific example (FIG. 18) in which the medical image of interest is a first illumination light image. As shown in the example of FIG. 18, with one illumination light emission pattern Pa1, two frames of a medical image of interest 121 are captured during a first illumination period in which the first illumination light L1 is emitted (shown as image frames 1 and 2 in FIG. 18), and one frame of a quasi-medical image of interest 122 is captured during a second illumination period in which the second illumination light L2 is emitted (shown as image frame 3 in FIG. 18). In this case, the interpolated frame image generating unit 120 generates an interpolated frame image 123 (shown as an interpolated frame "1+2" in FIG. 18) by superimposing the image frames 1 and 2. The display frames are in the order of image frame 1, interpolated frame "1+2", image frame 2, and so on, and the images are displayed in the section of interest in this order. Thereafter, in a similar manner, imaging frames 4 and 5 are acquired as a medical image of interest 121, imaging frame 6 is acquired as a quasi-medical image of interest 122, and an interpolated frame image 123 is generated from the interpolated frames "2+4" in one light emission pattern.
[0076] 18, one frame of the interpolated frame image 123 is generated from two frames of the medical image of interest 121 for the sake of simplicity, but three or more frames of the medical image of interest 121 may be superimposed to generate the interpolated frame image 123. Also, as in Example 5 of Table 1, when the sub-region of interest display frame rate is higher than the sub-region of interest imaging frame rate and the interpolated frame image 123 is generated from the sub-region of interest, the interpolated frame image is generated in the same manner.
[0077] When generating the interpolated frame image 123 by superimposing the medical images of interest 121 using the averaging method, the image may be generated by a simple averaging method in which a plurality of medical images of interest 121 are superimposed at the same ratio, or by a weighted averaging method in which the images are weighted by the medical images of interest 121 and superimposed. Also, when generating the interpolated frame image 123 from a quasi-medical image of interest as in Example 5 of Table 1, the interpolated frame image is generated in a similar manner.
[0078] With the above configuration, when generating an interpolated frame image that compensates for the partition display frame rate, a natural-looking interpolated frame image can be generated by superimposing images.
[0079] Moreover, it is preferable that the interpolation frame image is generated by using at least two or more target medical images or quasi-target medical images by a motion vector method, which generates an interpolation frame image by superimposing target medical images or quasi-target medical images in consideration of a motion vector of a subject appearing between target medical images or quasi-target medical images of multiple frames captured in the past.
[0080] When generating an interpolated frame image using the motion vector method, the interpolated frame image generating unit 120 first analyzes the image signals of multiple target medical images or quasi-target medical images, searches for pixels corresponding to the same observation site between images of previous and next frames using, for example, a pattern matching method, and calculates the spatial distance (amount of motion) and direction (direction of motion) of this pixel as a motion vector. Next, the image of each frame is weighted taking into account the motion vector, and the target medical image or quasi-target medical image is superimposed.
[0081] A specific example of the motion vector method will be described. For example, as shown in Fig. 19, when three medical images of interest 124, 125, and 126 acquired in chronological order are superimposed, a motion vector 127a of subject S between medical image of interest 124 of the first frame and medical image of interest 125 of the second frame, and a motion vector 127b of subject S between medical image of interest 125 of the second frame and medical image of interest 126 of the third frame are calculated. In this case, since motion vector 127b is larger than motion vector 127a, the image of the third frame is weighted relatively heavily before superimposing medical images of interest 124, 125, and 126 to generate an interpolated frame image 128. Interpolated frame image 128 shows superimposition 128a (shown by a solid line) of subject S calculated from medical image of interest 125 of the second frame and medical image of interest 126 of the third frame. An afterimage 128b (shown by a dotted line) of subject S may also be displayed. The generated interpolated frame image 128 is displayed, for example, inserted between the second frame medical image of interest 125 and the third frame medical image of interest 126. This makes it possible to generate an interpolated frame image that appears to move smoothly in accordance with the moved image.
[0082] With the above configuration, when generating an interpolated frame image that compensates for the partition display frame rate, a natural-looking interpolated frame image can be generated by superimposing images that take into account the motion vector.
[0083] Moreover, it is preferable that the interpolation frame image is generated by a simple copy method of copying a medical image of interest or a quasi-medical image of interest. The simple copy method is a method of generating an interpolation frame image by copying at least one or more medical images of interest or quasi-medical images captured in the past. This makes it possible to generate an interpolation frame image while reducing the burden of image analysis on the processor device 14.
[0084] A specific example of the simple copy method will be described. A specific example (FIG. 20) in which the medical image of interest is a first illumination light image will be described. As in FIG. 18, with one illumination light emission pattern Pa1, two frames of the medical image of interest 121 are captured during a first illumination period in which the first illumination light L1 is emitted, and one frame of the quasi-medical image of interest 122 is captured during a second illumination period in which the second illumination light L2 is emitted. In this case, the interpolated frame image generating unit 120 generates an interpolated frame image 123 (indicated by interpolated frames "1", "2", "4", "5", and "7" in FIG. 20) by superimposing the captured frames 1 and 2. The displayed frames are in the order of captured frame 1, interpolated frame "1", captured frame 2, and so on, and the images are displayed in the section of interest in this order.
[0085] With the above configuration, when generating an interpolated frame image that compensates for the partition display frame rate, the image is replicated, thereby reducing the load on the processor and generating a natural-looking interpolated frame image.
[0086] Methods for generating the interpolated frame image include, but are not limited to, an averaging method, a moving vector method, or a simple copy method.
[0087] As in Example 6 of Table 1, when the section display frame rate is lower than the section imaging frame rate, it is preferable to select a medical image of interest to be displayed in the section of interest, or a quasi-medical image to be displayed in the quasi-section of interest. When the section display frame rate is lower than the section imaging frame rate, the number of acquired medical images of interest or quasi-medical images exceeds the number that can be displayed, so the display control unit 70 selects medical images of interest or quasi-medical images to be displayed and adjusts the number to the number that can be displayed. The selection of images to be displayed is preferably determined by the difference between the section display frame rate and the section imaging frame rate.
[0088] For example, in the case of Example 6 in Table 1, since the interest section display frame rate is 60 fps and the interest section imaging frame rate is 90 fps, a selection is made to display every two out of three frames of the acquired interest medical image. Also, since the sub-interest section display frame rate is 15 fps and the interest section imaging frame rate is 30 fps (in Example 6 in Table 1, one type of second illumination light is used to capture the sub-interest medical image at 30 fps), a selection is made to display every two out of two frames of the acquired interest medical image.
[0089] For example, as shown in Fig. 21, when selecting a featured medical image or quasi-featured medical image to be displayed, two frames of featured medical image 121 are captured during a first illumination period in which first illumination light L1 is emitted with one illumination light emission pattern Pa1, and one frame of quasi-featured medical image 122 is captured during a second illumination period in which second illumination light L2 is emitted, as in Fig. 18. In this case, display controller 70 selects, for example, one image to be displayed for every two frames of captured featured medical image 121. That is, featured medical images 121 are displayed in the order of captured frames "1", "3", "5", "7" ... as display frames.
[0090] With the above configuration, regardless of the magnitude relationship between the overall imaging frame rate and the overall display frame rate, images to be displayed can be thinned out and selected in accordance with the attention section display frame rate and the sub-attention section display frame rate.
[0091] The lesion recognition unit 130 preferably detects a region of interest, which is a region that the user should pay attention to, from the focused medical image and / or the quasi-focused medical image, and further distinguishes the region of interest. The region of interest is a site such as a lesion having a feature amount within a specific range. The feature amount is preferably the shape or color of the subject, or a value obtained from the shape, color, or the like. Items of the feature amount include, for example, blood vessel density, blood vessel shape, number of blood vessel branches, blood vessel thickness, blood vessel length, blood vessel tortuosity, blood vessel depth, duct shape, duct opening shape, duct length, duct tortuosity, and color information. The feature amount is preferably at least one of these, or a value combining two or more of these. Note that the items of the feature amount are not limited to these, and may be added as appropriate depending on the usage situation.
[0092] Detection is the process of finding a region of interest in a medical image, while differentiation is the process of identifying the nature of the region of interest (e.g., neoplastic polyp, non-neoplastic polyp, inflammation, etc.).
[0093] As shown in Fig. 22, the lesion recognition unit 130 includes a detection classifier 131 and a discrimination classifier 132. As shown in Fig. 23, the detection classifier 131 receives a medical image 130a, which is a medical image of interest and / or a quasi-medical image, from the medical image acquisition unit 60, and outputs a detection result 131a, which is a result of detecting a region of interest. As shown in Fig. 24, the discrimination classifier 132 receives a medical image 130a, which is a medical image of interest and / or a quasi-medical image, from the medical image acquisition unit 60, and outputs a discrimination result 132a, which is a result of discriminating the region of interest. Either one of the detection and discrimination of the region of interest may be performed, or both may be performed.
[0094] The detection classifier 131 and the discrimination classifier 132 are preferably learning models trained by teacher image data. The detection results and discrimination results in the teacher image data may be assigned by an experienced doctor, or may be automatically assigned by a device other than the detection classifier 131 and the discrimination classifier 132. Information output by the detection classifier 131, the discrimination classifier 132, or another learning model may be assigned to a medical image and used as teacher image data for training the detection classifier 131 and the discrimination classifier 132.
[0095] It is preferable to use deep learning for machine learning to generate a learning model, for example, a multi-layer convolutional neural network. In addition to deep learning, machine learning includes decision trees, support vector machines, random forests, regression analysis, supervised learning, semi-unsupervised learning, unsupervised learning, reinforcement learning, deep reinforcement learning, learning using neural networks, generative adversarial networks, and the like.
[0096] With the above configuration, it is possible to detect and distinguish a region of interest from a medical image of interest or a quasi-medical image of interest, thereby assisting in screening observation or observation of a detailed structure.
[0097] The detection result output by the detection classifier 131 and the discrimination result output by the discrimination classifier 132 are transmitted to the display control unit 70. The display control unit 70 preferably controls to display the detection result or the discrimination result in the attention section and / or the sub-attention section. The display control unit 70 may also control to display the detection result and the discrimination result.
[0098] A specific example of displaying the detection result will be described. For example, as shown in Fig. 25, when displaying the detection result of detecting a region of interest 135 from a quasi-targeted medical image 134 in a target section 72 on a display image 71, a frame 137 is displayed around the region of interest 135 included in a target medical image 136 displayed in the target section 72. Also, a marker 138 indicating an approximate position within the target medical image 136 is displayed. In the case of Fig. 21, the marker 138 indicates that the region of interest 135 is located at the bottom right. The method of displaying the detection result is not limited to a circular or polygonal enclosure, a warning display on the screen, or a warning sound.
[0099] A specific example of the discrimination result will be described. For example, as shown in FIG. 26, when the discrimination result obtained by discriminating the region of interest 135 from the quasi-attention medical image 134 is displayed in the attention section 72 on the display image 71, a caption 139 indicating the discrimination result is displayed. In the case of FIG. 26, since the discrimination result is "tumor", "tumor" is displayed in the caption 139. Also, the color of the marker 138 may be changed according to the discrimination result. For example, if the discrimination result is neoplastic, the marker 138 is displayed in yellow, and if it is non-neoplastic, the marker 138 is displayed in green. The method of displaying the discrimination result is not limited to a circle or polygon enclosure, a warning display on the screen, or a warning sound.
[0100] With the above configuration, the detection results or discrimination results can be displayed to assist in screening observation or observation of detailed structures.
[0101] In addition to the focus area 72 and the sub-focus area 73, the display image 71 may also be provided with a still image area 140 for displaying a still image and a text information area 141 for displaying text information, as shown in FIG.
[0102] 28, an extended processor device 150 may be provided, with the lesion recognition section 130 included in the extended processor device 150. In this case, the medical image of interest 151 and / or the quasi-medical image of interest 152 are input to the lesion recognition section 130 of the extended processor device 150, and as a result, a detection result and / or a discrimination result of the region of interest 135 is output. The detection result and / or the discrimination result are transmitted to the processor device 14, and the detection result and / or the discrimination result are displayed in the region of interest 72 or the quasi-region of interest 73. In FIG. 28, the detection result of the region of interest 135 is shown in the display image 71.
[0103] In the present embodiment, the processor device 14 is connected to the endoscope system 10, but the present invention is not limited thereto, and other medical devices such as an ultrasound imaging device or a radiography device may be used. The endoscope 12 may be a rigid or flexible endoscope. In addition, the central control unit 50, the medical image acquisition unit 60, the display control unit 70, the overall display frame rate recognition unit 90, the section display frame rate setting unit 100, the section imaging frame rate setting unit 110, the interpolated frame image generation unit 120, and the lesion recognition unit 130 of the endoscope system 10 may be partly or entirely provided in an image processing device that communicates with the processor device 14 and cooperates with the endoscope system 10. For example, they may be provided in a diagnosis support device that acquires an image captured by the endoscope 12 directly from the endoscope system 10 or indirectly from the PACS. In addition, a medical work support device connected via a network to various inspection devices such as a first inspection device, a second inspection device, ..., an Nth inspection device, including the endoscopic system 10, may be provided with some or all of the central control unit 50, medical image acquisition unit 60, display control unit 70, overall display frame rate recognition unit 90, section display frame rate setting unit 100, section shooting frame rate setting unit 110, interpolated frame image generation unit 120, and lesion recognition unit 130 of the endoscopic system 10.
[0104] In this embodiment, the hardware structure of the processing units that execute various processes, such as the central control unit 50, medical image acquisition unit 60, display control unit 70, overall display frame rate recognition unit 90, partition display frame rate setting unit 100, partition photography frame rate setting unit 110, interpolated frame image generation unit 120, and lesion recognition unit 130, is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing various processes.
[0105] One processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same type or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, as represented by a computer such as a client or server, one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form using a processor that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.
[0106] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements, etc. Also, the hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). [Explanation of symbols]
[0107] 10 Endoscope System 12 Endoscopy 12a Insertion part 12b Operation section 12c Curved section 12d Tip 12e Angle knob 12f Observation mode switch 12h Still image acquisition switch 12i Zoom control section 12j forceps mouth 13 Light source device 14 Processor unit 15 Display 16 User Interface 20 Light source section 20a V-LED 20b B-LED 20c G-LED 20d R-LED 21 Light source processor 22 Optical path coupling section 23 Light Guide 30a illumination optical system 30b Imaging optical system 31 Lighting Lens 41 Objective Lens 42 Zoom Lens 43 Image Sensor 44 Imaging Processor 50 Central Control Unit 60 Medical Image Acquisition Department 70 Display control section 71 Display images 72 Featured Areas 73 Semi-attractive area 80 Display image setting section 81 Display image setting screen 82 tabs 83 Layout setting button 84 Display example 90 Overall display frame rate recognition section 100 Section display frame rate setting section 101 Plot display frame rate setting screen 102 Overall display frame rate display field 103 Focus area display frame rate setting field 104 Semi-attention area display frame rate setting field 110 Section shooting frame rate setting unit 111 Section image capture frame rate setting screen 112 Overall image capture frame rate display field 113 Target area image capture frame rate setting field 114 Semi-interest area imaging frame rate setting field 115 Light Emitting Pattern Setting Column 120 Interpolated frame image generator 121, 124, 125, 126, 136, 151 Featured Medical Images 122, 134, 152 Semi-attractive medical images 123, 128 Interpolated Frame Images 127a, 127b Movement vector 128a Overlaying the subject 128b Afterimage 130 Lesion Recognition Department 130a Medical Imaging 131 Detection Classifier 131a Detection Results 132 Classifier for discrimination 132a Identification results 135 Areas of Interest 137 slots 138 Marker 139 Caption 140 Still Image Section 141 Text information section 150 Extended Processor Unit
Claims
1. A processor device including a light source processor and an image processing processor, The light source processor includes: performing control to automatically switch between a first illumination period in which an image of an object illuminated with a first illumination light is captured and a second illumination period in which an image of the object illuminated with at least one type of second illumination light having a spectrum different from that of the first illumination light is captured; The image processor includes: acquiring a first illumination light image as a medical image during the first illumination period; acquiring a second illumination light image as the medical image during the second illumination period; generating a display image for displaying the first illumination light image and / or the second illumination light image; The display image has one attention area and at least one sub-attention area, a focus area display frame rate, which is the number of medical images displayed in the focus area per unit time, is higher than a sub-focus area display frame rate, which is the number of medical images displayed in the sub-focus area per unit time; the medical image to be displayed in the section of interest is the first illumination light image or the second illumination light image, a total display frame rate is a number of the display images displayed per unit time on a display that displays the display images; A processor device in which the focus region display frame rate and the sub-focus region display frame rate can be set arbitrarily within the range of the total display frame rate.
2. The image processor includes: The medical image to be displayed in the attention section is set as an attention medical image, setting a region of interest imaging frame rate, which is the number of the region of interest medical images acquired per unit time, to be equal to or higher than the region of interest display frame rate; acquiring said medical image of interest based on said segment of interest imaging frame rate; The medical image to be displayed in the sub-attention section is treated as a sub-attention medical image. A sub-region of interest imaging frame rate, which is the number of the sub-region of interest medical images acquired per unit time, is set to be equal to or higher than the sub-region of interest display frame rate; The processor apparatus of claim 1 , wherein the sub-region of interest medical images are acquired based on the sub-region of interest imaging frame rate.
3. The image processor includes: If the attention area display frame rate is higher than the attention area imaging frame rate, generating an interpolated frame image to be displayed in the section of interest using the medical image of interest; Or, if the sub-attention area display frame rate is higher than the sub-attention area imaging frame rate, The processor unit according to claim 2 , wherein the sub-interest medical image is used to generate the interpolated frame image for display in the sub-interest section.
4. The processor device according to claim 3 , wherein the interpolated frame image is generated by an averaging method using at least two or more of the medical images of interest or the quasi-medical images of interest.
5. The processor device according to claim 3 , wherein the interpolated frame image is generated by using at least two or more of the medical images of interest or the quasi-medical images of interest and by a motion vector method.
6. The processor apparatus according to claim 3 , wherein the interpolated frame image is generated by duplicating the medical image of interest or the sub-medical image of interest.
7. The image processor includes: if the segment of interest display frame rate is lower than the segment of interest imaging frame rate, selecting the medical image of interest to display in the segment of interest; Or, if the sub-attention area display frame rate is lower than the sub-attention area imaging frame rate, 3. The processor unit of claim 2, further comprising: a processor for selecting said sub-interest medical image for display in said sub-interest section.
8. The image processor includes: Detecting a region of interest, which is a region to which a user should pay attention, from the medical image of interest and / or the quasi-medical image of interest; Identifying the region of interest in the medical image of interest and / or the quasi-medical image of interest, or 8. A processor device according to claim 2, further comprising: a processor for detecting the region of interest from the medical image of interest and / or the quasi-medical image of interest; and for identifying the region of interest.
9. The image processor includes: A processor device as described in claim 8, which controls the display of a detection result resulting from detecting the region of interest, a discrimination result resulting from discriminating the region of interest, or both the detection result and the discrimination result in the region of interest and / or the sub-region of interest.
10. The processor device according to claim 2 , wherein the semi-target medical image displayed in the semi-target section can be switched between display and non-display.
11. 11. The processor device according to claim 1, wherein the size of the area of interest and the sub-area of interest in the display image can be changed arbitrarily.
12. The image processor includes: a mono-emission mode in which the medical image is acquired by capturing an image of the subject illuminated with either the first illumination light or the at least one type of second illumination light; 12. The processor device according to claim 1, further comprising a multi-flash mode in which the first illumination period and the second illumination period are automatically switched to capture an image of the subject to obtain the medical image, and the multi-flash mode is switched by a user operation.
13. performing control to automatically switch between a first illumination period in which an image of an object illuminated with a first illumination light is captured and a second illumination period in which an image of the object illuminated with at least one type of second illumination light having a spectrum different from that of the first illumination light is captured; acquiring a first illumination light image as a medical image during the first illumination period; acquiring a second illumination light image as the medical image during the second illumination period; generating a display image for displaying the first illumination light image and / or the second illumination light image; The display image has one attention area and at least one sub-attention area, a focus area display frame rate, which is the number of medical images displayed in the focus area per unit time, is higher than a sub-focus area display frame rate, which is the number of medical images displayed in the sub-focus area per unit time; the medical image to be displayed in the section of interest is the first illumination light image or the second illumination light image, a total display frame rate is a number of the display images displayed per unit time on a display that displays the display images; A method for operating a processor device, wherein the focus area display frame rate and the sub-focus area display frame rate can be arbitrarily set within the range of the total display frame rate.
14. An endoscope system comprising: the processor device according to claim 1 ; a light source device; and an endoscope.
Citation Information
Patent Citations
Endoscope system
WO2020158165A1
Cited By
Medical assistance system and image display method
US20240428410A1
