Endoscope device

Abstract

An endoscope device (1) is provided that can be applied to a large number of subjects, etc. The endoscope device (1) comprises: an endoscope (10) having an imaging unit (15); an image processing circuit (31) that generates an image of the subject; a calculation circuit (32) that calculates an index by normalizing the sum of the red and green pixel values ​​of a pixel in the image of the subject by a value that is twice the value of the blue pixel of that pixel; a memory (50) that stores multiple thresholds for identifying multiple regions of the subject as three or more symptom levels and identification colors corresponding to the symptom levels; an identification circuit (33) that identifies the symptom level according to the index; an identification color acquisition circuit (34) that acquires the identification color corresponding to the symptom level; an image generation circuit (35) that generates an identification image using the identification color; and a monitor (40) that displays the identification image.

Description

Technical Field

[0001] This invention relates to an endoscopic device for imaging a subject inside the body. Background Technology

[0002] International Publication No. 2018 / 230130 discloses an endoscope device that calculates an index representing the degree of abnormality of a subject based on the colors contained in an image of the subject captured by the endoscope, and identifies and displays the index according to a threshold.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: International Publication No. 2018 / 230130 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] However, the degree of abnormality indicated by the indicators of the aforementioned endoscopic device is either abnormal or normal. Furthermore, because the thresholds used to identify the indicators are simple to set, there are concerns that the applicable subjects may be limited.

[0008] The object of this invention is to provide an endoscopic device that identifies and displays based on three or more symptom levels. Furthermore, the object of this invention is to provide an endoscopic device applicable to a large number of subjects.

[0009] means for solving problems

[0010] An endoscopic apparatus according to an embodiment of the present invention comprises: an endoscope having a camera unit for capturing images of at least one subject within a patient's body and outputting an image signal; an image processing unit for processing the image signal to generate an image of the subject; a calculation unit for calculating an index for each pixel by standardizing the sum of the red and green pixel values ​​of each pixel in a plurality of pixels of at least a portion of the image of the subject using a value that is twice the blue pixel value of each pixel; a memory for storing a plurality of thresholds for identifying a plurality of regions of the subject as one of three or more symptom levels, and identification colors corresponding to each of the plurality of symptom levels; an identification unit for identifying the symptom level of each pixel based on the index and using the plurality of thresholds; an identification color acquisition unit for acquiring the identification color corresponding to the symptom level of each pixel; an image generation unit for generating an identification image using the identification color of each pixel of the plurality of pixels; and a monitor for displaying the identification image.

[0011] Invention Effects

[0012] According to the present invention, an endoscopic device that identifies and displays based on three or more symptom levels can be provided. Furthermore, according to the present invention, an endoscopic device applicable to a large number of subjects can be provided. Attached Figure Description

[0013] Figure 1 This is a diagram illustrating the structure of an endoscope device according to an embodiment of the present invention.

[0014] Figure 2 This is a graph used to illustrate the relationship between the light absorption characteristics of blood plasma and the light emission characteristics of a light source.

[0015] Figure 3 It is a diagram used to illustrate the light absorption characteristics of the test subject.

[0016] Figure 4 This is a diagram illustrating the calculation formulas for the parameters of the endoscope device used to explain embodiments of the present invention.

[0017] Figure 5 This is a diagram illustrating the calculation formulas for the parameters of the endoscope device used to explain embodiments of the present invention.

[0018] Figure 6 This is a flowchart of the operation method of the endoscopic device according to an embodiment of the present invention.

[0019] Figure 7 This is a graph illustrating the relationship between symptom levels and thresholds in an endoscopic device according to an embodiment of the present invention.

[0020] Figure 8 This is a graph illustrating the relationship between symptom levels and threshold groups in an endoscopic device according to an embodiment of the present invention.

[0021] Figure 9 This is a graph illustrating the relationship between symptom levels and color recognition in an endoscopic device according to an embodiment of the present invention.

[0022] Figure 10 Example 1 shows the display of the monitor screen in the endoscopic device according to an embodiment of the present invention.

[0023] Figure 11 Example 2 shows the display of the monitor screen in the endoscopic device according to an embodiment of the present invention.

[0024] Figure 12 Example 3 shows the display of the monitor screen in the endoscopic device according to an embodiment of the present invention.

[0025] Figure 13 Example 4 shows the display of the monitor screen in the endoscopic device according to an embodiment of the present invention.

[0026] Figure 14 Example 5 shows the display of the monitor screen in the endoscopic device according to an embodiment of the present invention.

[0027] Figure 15 Example 6 shows the display of the monitor screen in the endoscopic device according to an embodiment of the present invention.

[0028] Figure 16 Example 7 shows the display of the monitor screen in the endoscopic device according to an embodiment of the present invention. Detailed Implementation

[0029] <Structure of the Endoscopic Device>

[0030] like Figure 1 As shown, the endoscope device 1 of the embodiment includes an endoscope 10, a light source device 20, a processor 30, a monitor 40, and a memory 50.

[0031] The endoscope 10 includes: an elongated insertion portion 11 into which a subject 90 is inserted; an operation portion 12 located at the base of the insertion portion 11; a universal cable 13 extending from the operation portion 12; and a connector 14. The operation portion 12 includes multiple buttons 12A serving as an endoscope setting unit for operating endoscope and camera functions. The insertion portion 11 of the endoscope 10, from the front end side, sequentially includes a front end portion 11A, a curved portion 11B located at the base of the front end portion 11A, and an elongated flexible tube 11C located at the base of the curved portion 11B. An imaging unit 15 serving as an imaging unit and an illumination unit 16 serving as an illumination unit are provided at the front end portion 11A.

[0032] The connector 14 of the endoscope 10 is connected to the light source device 20 and the processor 30. The illumination light L generated by the light source device 20 is guided to the illumination unit 16 at the front end 11A of the insertion part 11 to illuminate the subject 91 inside the patient 90. The imaging unit 15 has an imaging element such as a CCD. The imaging unit 15 converts the reflected light R from the subject 91 into an electrical signal and outputs the image of the subject to the processor 30.

[0033] The light source device 20 includes a light source control unit 22, a light source 23, and a combiner 24.

[0034] The light source control unit 22 is a light source control circuit that is connected to the light source 23 and controls the light source 23 according to the control signal from the processor 30.

[0035] Light source 23 has multiple light-emitting elements, such as LEDs. Light source 23 has an R element 23R, a G element 23G, and a B element 23B. The R element 23R emits red light Br in the normal frequency band. The G element 23G emits green light Bg in the normal frequency band. The B element 23B emits blue light Bb in the normal frequency band. The B element 23B not only utilizes the blue light Bb in the normal frequency band, but also uses, for example, a narrowband optical filter (not shown) to narrow the blue light, outputting narrowband blue light Nb.

[0036] The combiner 24 combines multiple lights input from the light source 23 to output illumination light L to the illumination unit 16.

[0037] The processor 30 includes an image processing unit 31, a calculation unit 32, a recognition unit 33, a color recognition acquisition unit 34, an image generation unit 35, and a setting unit 36. The processor 30, which is composed of a CPU, controls the entire endoscope device 1 and generates an endoscope image based on the camera signal input from the endoscope 10. As will be described later, an identification image is generated based on the endoscope image.

[0038] The setting unit 36, which serves as the setting circuit, includes buttons for users to input various instructions. The setting unit 36 ​​can also be a separate component from the processor 30, such as a touch panel, keyboard, foot switch, or button 12A of the endoscope 10. For example, the setting unit 36 ​​can input instructions such as bending indicators for the curved section, drive indicators for the light source device 20, the type of illumination light L illuminating the subject 91, the type of observation area of ​​the subject 91, and the image displayed on the monitor 40.

[0039] Furthermore, the structure of the image processing unit 31, the calculation unit 32, the recognition unit 33, the color recognition acquisition unit 34, and the image generation unit 35 will be described below.

[0040] At least one of the multiple structures of the processor 30 and the light source control unit 22 can be constituted by the processor 30 that operates through software (program) or the internal circuit (CPU) of the light source device, or they can be constituted by dedicated hardware circuits.

[0041] The monitor 40 is, for example, an LCD or CRT displaying color images. The monitor 40 displays images directed by the processor 30. The monitor 40, which has a touch panel function, may also be part of the setting unit 36.

[0042] The memory 50 is a RAM, ROM, or hard disk drive device that stores data such as the operating conditions of the processor 30 and programs. The memory 50 may also be the internal memory of the processor 30 that transfers and stores data from non-transitory computer-readable storage media such as CDs or DVDs. The processor 30 performs prescribed processing based on the programs and data stored in the memory 50. Furthermore, past examination data of the patient 90 stored in a separate device, such as a server, can be transferred to the memory 50 via an internet connection.

[0043] <Indicator>

[0044] The calculation unit 32 is a calculation circuit that calculates the index VI of each pixel in the multiple pixels of the subject image output by the camera unit. The index VI, which quantitatively represents the symptom level of the subject 91, is calculated using a prescribed formula.

[0045] The following explains the process of selecting the calculation formula.

[0046] Figure 2 This is a graph illustrating the relationship between the light absorption properties W of blood plasma and the wavelength of light produced by light source 23. Figure 2 The figure shows the red light Br in the ordinary frequency band, the green light Bg in the ordinary frequency band, the blue light Bb in the ordinary frequency band, the narrowband blue light Nb, the light absorption characteristics W of plasma, and the peak wavelength Wp of the light absorption coefficient of plasma.

[0047] like Figure 2 As shown, the light absorption characteristic W of plasma decreases near wavelength 415 nm, peaks near wavelength 465 nm, and approaches 0 near wavelength 550 nm.

[0048] Therefore, while blue light Bb can be in the general frequency band, it is particularly preferable to narrow the band so that the center wavelength is the same as the peak wavelength Wp of the plasma's absorbance coefficient for significant detection. For example, blue light Bb is narrowed to a center wavelength around 465 nm and used as narrowband blue light Nb. Blue light Bb can also be narrowed to a center wavelength of 460 nm to 470 nm. Furthermore, blue light Bb can also be narrowed to a center wavelength of 415 nm to 495 nm.

[0049] When irradiated with special light containing red light (Br), green light (Bg), and narrowband blue light (Nb), the plasma absorbs more blue light than red and green light, and the yellow color is more pronounced compared to ordinary light containing ordinary blue light (Bb).

[0050] then, Figure 3 The cut surface of the mucosa is shown schematically. Figure 3The diagram shows normal mucosa (N), edema (M), polyp (S), blood vessel (Bv), and illumination light (L). Here, illumination light L is short-wavelength monochromatic light such as narrow-band blue light (Nb). The pigment within the mucosa is blood plasma.

[0051] As shown in the light-receiving region L1, in the normal mucosa N, the penetration of the illumination light L is high, and the reflected light R appears pale yellow due to the mucosal pigment with a high absorption coefficient on the short wavelength side, which is shorter than the long wavelength side.

[0052] As shown in the light-receiving region L2, the penetration of illumination light L is reduced in edema M compared to normal mucosa N. More specifically, in edema M, illumination light L is scattered more on the short-wavelength side compared to the long-wavelength side due to the thickened epithelium, and is reflected instead of being absorbed by the pigments within the mucosa. Therefore, in edema M, reflected light R appears whiter than in normal mucosa N.

[0053] As shown in the light-receiving region L3, in polyp S, compared with edema M, the light penetration is further reduced, and the reflected light R appears to be whiter than that of edema M.

[0054] Figure 4 The diagram illustrates an index VI that normalizes the green pixel value Vg, red pixel value Vr, blue pixel value Vb, or the sum of the green pixel value Vg and the red pixel value Vr of the pixels contained in an endoscopic image. The pixel value V is obtained, for example, as 8-bit data (0-255).

[0055] Figure 4 This indicates the difference in index VI among normal mucosa (N), edema (M), and polyps (S) due to differences in the calculation formula of index VI. Figure 4 In the diagram, "Vg / Vb", "Vr / Vb", "Vr / Vg", and "(Vr+Vg) / 2Vb" on the X-axis represent the calculation formulas for index VI, while the Y-axis represents index VI standardized through each calculation formula.

[0056] Solid lines represent normal mucosa (N), single-dotted lines represent edema (M), and double-dotted lines represent polyps (S). Hereinafter, edema (M) and polyps (S) will be referred to as abnormal mucosa.

[0057] In the body's mucous membranes, such as the sinus mucosa, symptom levels increase in severity in the order of normal mucosa (N), edema (M), and polyps (S). There is a color difference between normal mucosa (N) and abnormal mucosa; as symptom levels worsen, the mucosal epithelium thickens, and the whiteness of the appearance increases. Therefore, the maximum calculation formula for the index VI of normal mucosa (N) and polyp (S) is "(Vr + Vg) / 2Vb".

[0058] Figure 5 The index VI, representing the normal mucosal N, is related to... Figure 4The standardized indices VIN for edema M and polyp S were derived from the same index VI. Figure 5 In the diagram, the X-axis represents the formula for calculating the standardized index VIN, which is used to calculate the index VI through normal mucosal N, and the Y-axis represents the index VIN.

[0059] like Figure 4 and Figure 5 As shown, in normal mucosa N and polyp S, the indices VI and VIN calculated by the formula "(Vr+Vg) / 2Vb" are greater than the indices VI and VIN calculated by other formulas.

[0060] That is, the indices VI and VIN, calculated by the formula “(Vr+Vg) / 2Vb”, represent the large difference between the color of normal mucosa N and the color of abnormal mucosa.

[0061] How Endoscopic Devices Work

[0062] use Figure 6 The flowchart illustrates the working method of the endoscope device 1.

[0063] <Step S10> Illuminate with light

[0064] The insertion part 11 of the endoscope 10 is inserted into the biological body of the subject 90, for example, into the nasal cavity. Illumination light L from the light source device 20 is irradiated onto the mucous membrane of the subject 91 via the illumination unit 16 at the front end 11A. The illumination light L is red light Br, green light Bg, and blue light Bb.

[0065] <Step S20> Output camera signal

[0066] The camera unit 15 of the front end 11A receives the reflected light R from the subject 91, converts it into an electrical signal, and outputs the camera signal to the processor 30.

[0067] <Step S30> Process the image

[0068] The image processing unit 31 is an image processing circuit that generates an endoscopic image of the subject by performing image processing such as gain adjustment, white balance adjustment, gamma correction, contour emphasis correction, and zoom adjustment based on the camera signal.

[0069] <Step S40> Calculate the indicators

[0070] The calculation unit 32 calculates the index VI of each pixel by normalizing the sum of the red pixel value Vr and the green pixel value Vg of that pixel using twice the blue pixel value Vb (Nb) of each pixel in the subject image. In other words, the calculation unit 32 calculates the index VI of multiple regions (pixels) of the subject.

[0071] exist Figure 4 , Figure 5 In the original text, the formula for standardizing pixel values ​​and calculating the index VI is "(Vr+Vg) / 2Vb". However, any formula that standardizes the sum of red and green pixel values ​​using twice the value of the blue pixel can be modified appropriately.

[0072] For example, the index VI can be converted to 8-bit (0-255) data, or a value can be added to the 8-bit data, or the value of k in the calculation formula "(Vr+Vg) / kVb" can be changed. The index VI is calculated using formula 1 below.

[0073] <Formula 1>

[0074] VI=32×log2[(Vr+Vg) / 2Vb]+256

[0075] Preferably, the calculation unit 32 uses any one of a plurality of calculation formulas corresponding to a plurality of subjects 91 (e.g., sinuses, digestive tract) to calculate index VI.

[0076] <Step S50> Identify symptom levels

[0077] The recognition unit 33 is a recognition circuit that identifies the symptom level of each pixel based on index VI and using multiple thresholds T.

[0078] In endoscopic device 1, the symptom levels are categorized into five levels: "normal, mild, moderate, severe, and most severe." If there are three or more symptom levels, it is easier to make a detailed assessment of the symptoms compared to the "normal / abnormal" levels.

[0079] To identify five symptom levels, four thresholds T are required: a first threshold T1 for identifying normal and mild symptoms, a second threshold T2 for identifying mild and moderate symptoms, a third threshold T3 for identifying moderate and severe symptoms, and a fourth threshold T4 for identifying severe and most severe symptoms. These thresholds T are pre-set appropriately based on the judgments of multiple assessors. Naturally, the four thresholds T, each set to a specified range, are in the order of T1 to T4.

[0080] Figure 7 An example of a threshold T is shown. The differences ΔT between multiple thresholds are preferably approximately the same. For example, the difference ΔT3 (threshold T4 - threshold T3) is preferably more than 80% and less than 120% of the difference ΔT2 (threshold T3 - threshold T2).

[0081] However, depending on the conditions, it is also possible that the first threshold difference ΔT1 between the first threshold T1 and the second threshold T2 is greater than the second threshold difference ΔT2 between the second threshold T2 and the third threshold T3, and the second threshold difference ΔT2 is greater than the third threshold difference ΔT3 between the third threshold T3 and the fourth threshold T4. Conversely, it is also possible that the first threshold difference ΔT1 is less than the second threshold difference ΔT2, and the second threshold difference ΔT2 is less than the third threshold difference ΔT3.

[0082] In addition, depending on the combined system, the camera components it carries, etc., sometimes the higher the severity of the illness, the greater the threshold difference ΔT.

[0083] Furthermore, the memory 50 stores multiple threshold groups, each corresponding to a plurality of subjects 91 and composed of multiple threshold values. The recognition unit 33 uses the threshold group corresponding to the subject 91 for recognition. The threshold group used by the recognition unit 33 can be automatically acquired or set by the setting unit 36.

[0084] like Figure 8 As shown, there are deviations (variances) among the individual thresholds T(T1-T4) included in multiple threshold groups, but preferably the deviations of each threshold T(T1-T4) are set within a specified range. For example, the maximum value of threshold T1 is 313, and the minimum value of threshold T1 is 291. In contrast, the maximum value of threshold T4 is 273, and the minimum value is 267. That is, the deviation LT1 (=6) of threshold T4 is smaller than the deviation LT2 (=22) of threshold T1.

[0085] The reason for this is the color balance calibration of the image processing unit 31. That is, the image processing unit 31 performs a white balance calibration based on the white color closest to the most severely affected area.

[0086] By changing the color used for color balance calibration or using multiple colors for calibration, the deviation of threshold T can be adjusted. For example, by using the color of the normal region (red to yellow) for color balance calibration, the deviation of threshold T1, LT2, can be reduced. Furthermore, by using a color balance calibration using an intermediate color between the color of the most severely affected area and the color of the normal region (red to yellow), the deviations of multiple thresholds T can be reduced and averaged.

[0087] <Step S60> Identify the color acquisition unit

[0088] The color acquisition unit 34 acquires the color corresponding to the symptom level of the pixel acquired by the recognition unit 33.

[0089] Figure 9 The identification color is displayed according to the symptom level. The index VI is, for example, data in the range of (0-511) obtained by adding 256 to 8-bit data. Multiple thresholds T and identification colors are stored in memory 50.

[0090] In addition, Figure 9 In the example, the color recognition and acquisition unit 34 can acquire multiple colors with different hues, but it can also acquire multiple chroma with different vividness, multiple brightness with different lightness, multiple shadow lines with different spacing, or multiple patterns with different designs.

[0091] Furthermore, endoscopic images sometimes contain pixels with erroneous pixel values ​​that do not occur during normal imaging. In the endoscope device 1, pixels with a pixel value V that is at least below a predetermined lower pixel value or above a predetermined upper pixel value among the red, green, and blue pixel values ​​are designated as first erroneous pixels. For example, pixels with a pixel value V in the range of (0-255) that is 5 or less or 250 or more are first erroneous pixels.

[0092] Furthermore, pixels of index VI that are below the specified lower threshold or above the specified upper threshold are designated as second error pixels. For example, in Figure 9 In the example shown, pixels with a lower threshold of 10 or an upper threshold of 500 are considered second-error pixels.

[0093] The color acquisition unit 34 acquires the erroneous color at the erroneous pixels (the first erroneous pixel and the second erroneous pixel). The image generation unit uses the erroneous color to generate a recognition image. The erroneous color is not limited to... Figure 9 The white / black shown could also be gray, for example. Furthermore, pixels with thresholds below the lower limit and above the upper limit could also be the same incorrect color (e.g., white). The numerical value of the criterion for judging incorrect pixels and the data of the incorrect color are stored in memory 50.

[0094] Alternatively, the processor 30 can calculate the ratio of the number of erroneous pixels to the total number of pixels (error ratio), and generate a warning if the error ratio exceeds a predetermined value. The number of erroneous pixels can be any one of the number of first erroneous pixels, the number of second erroneous pixels, or the total number of erroneous pixels. The warning can be displayed on the monitor 40, for example, as characters or a graphic.

[0095] Warnings, such as changes to threshold groups, facilitate the identification of symptom levels. Furthermore, changes to threshold groups can be performed automatically or by the user.

[0096] <Step S70> Generate recognition image

[0097] The color acquisition unit 34 acquires the color corresponding to the symptom level of each pixel. The image generation unit 35 is a color acquisition circuit that generates a recognition image using the color of each pixel of the multiple pixels.

[0098] <Step S80> is displayed

[0099] The monitor 40 displays the recognized image in various ways.

[0100] <Example 1>

[0101] Figure 10 This shows an example of the image displayed on monitor 40. Figure 10 In this image, a portion of the endoscope image 40A, which is displayed in color, is replaced by the identification image 40B. In other words, an overlapping image of the identification image 40B is displayed in the endoscope image 40A. The area displayed as the identification image 40B in the endoscope image 40A is indicated by a box.

[0102] From the viewpoints of visibility and operability, it is preferable that the area of ​​the identified image 40B is 20% or more and 70% or less of the area of ​​the entire region of the endoscopic image 40A. The area of ​​the identified image 40B can be changed, for example, by operating the setting unit 36.

[0103] Additionally, the monitor 40 displays the average value 40D of the indicators along with the color recognition overview display 40C. That is, the calculation unit 32 calculates the average value 40D of the indicators of multiple pixels, and the monitor 40 displays the average value 40D of the indicators.

[0104] Users can easily grasp the symptoms of the examinee based on the average value of the indicator, 40D.

[0105] <Example 2>

[0106] like Figure 11 As shown, a recognition image 40B is displayed on the monitor 40 instead of the endoscope image 40A. That is, the calculation unit 32 may calculate indicators based on multiple pixels of the entire area of ​​the endoscope image 40A, which is the subject image, and the image generation unit 35 may generate a recognition image that corresponds to the entire area of ​​the endoscope image 40A.

[0107] <Example 3>

[0108] like Figure 12 As shown, in the monitor 40, the identification image 40B of the area enclosed by the frame in the endoscope image 40A is displayed in a different area than the endoscope image 40A. At least one of the position and extent (area) of a portion of the area where the identification image 40B is generated within the entire area of ​​the endoscope image 40A, which is the subject image, can be appropriately selected by operating the setting unit 36.

[0109] Within narrow confines of a tube, it can sometimes be difficult to orient the center of the endoscopic image 40A (the center of the field of view of the camera unit 15) toward the area of ​​interest. However, by selecting at least one of the location and extent of the area displaying the identification image 40B, the user can easily identify the area of ​​interest.

[0110] <Example 4>

[0111] like Figure 13 As shown, the monitor 40 displays, together with the endoscope image 40A and the identification image 40B, past images of the subject 90 from past examinations, namely identification image 40BP, and the average value of the index 40DP.

[0112] Regarding the range of past recognition images 40BP selected based on recognition image 40B, it is determined by using pattern matching of the endoscope image or recognition image, the state during the examination (e.g., the position and orientation of the endoscope tip), etc.

[0113] The display screen of the endoscopic image 40A and recognition image 40B during the examination can be switched with the display screen of the past endoscopic image and recognition image 40BP, or displayed on different monitors. Preferably, the display range and display position of the past recognition image 40BP can also be appropriately changed. The past recognition images 40BP of the examinee 90 during examinations are stored, for example, on a server within the hospital and transferred to the processor 30.

[0114] Furthermore, the difference between the past recognition image 40BP and the recognition image 40B at the time of examination can be calculated and visualized. For example, by displaying only the difference of the index VI for each pixel, the increase of pixels above a specified value can clearly indicate areas where the condition has changed.

[0115] <Example 5>

[0116] Figure 14 The monitor 40's image and Figure 10 Similarly, but in image 40B, only the red region representing the most severe level of illness is displayed in color among the five color regions, while the regions from normal to severe levels are displayed in monochrome white or gray. Alternatively, the regions from normal to severe levels can also be displayed as a typical endoscopic image illuminated with white light.

[0117] The horizontal (color) areas to be displayed in color can be preset, but it is preferable that they can be appropriately set. For example, if each of the five color areas of the identification color overview display 40C is clicked once, the horizontal area changes from being displayed in color to being displayed in white; if clicked again, it changes from being displayed in white to being displayed in color. Multiple horizontal areas can also be displayed in color. Alternatively, a typical endoscopic image illuminated by white light can be displayed in the area of ​​the identification image displayed in white.

[0118] <Example 6>

[0119] Figure 15 The monitor 40's image and Figure 10 Similar, but with Figure 10 Unlike other methods, the identification image 40B is made semi-transparent by setting the transmittance and is overlaid on the endoscope image 40A. When the transmittance is 0%, only the identification image is displayed and not the endoscope image 40A. When the transmittance is 100%, only the endoscope image 40A is displayed and not the identification image 40B.

[0120] Furthermore, the transmittance of each of the five color regions in the image 40B can be set for each color. Of course, it is also possible to make only the selected color semi-transparent.

[0121] Regarding the semi-transparency setting of the recognition image 40B, for example, if the monitor 40 becomes a touch panel, if each of the five color areas of the recognition color overview display 40C is pressed and slid to the left, the transmittance of the recognition color in that horizontal area increases; if each of the five color areas of the recognition color overview display 40C is pressed and slid to the right, the transmittance of the recognition color decreases.

[0122] (Table 1)

[0123]

[0124] In Case A of Table 1 above, all identification colors (level) are set to a transmittance of 50%.

[0125] In scenario B of Table 1 above, the transmittance decreases as the severity level changes from the most severe to the moderate level, and is set to 0% at the mild and normal levels. Figure 15 This is the screen displayed on monitor 40 under scenario B.

[0126] In case C of Table 1 above, only the heaviest level has a transmittance of 50%, while the other levels are set to 0% transmittance.

[0127] Furthermore, in areas where the transmittance is set to 100%, only the endoscopic image 40A is displayed in those areas, so it is unknown in which part of the image the recognition image 40B is displayed. Therefore, it is also possible to display it as in cases 1 to 3 below.

[0128] (Scenario 1) such as Figure 14 In this case, when the recognition image 40B, which has a 100% transmittance area, overlaps only in the central part of the endoscopic image 40A, the boundary of the recognition image 40B is displayed with solid or dashed lines.

[0129] (Scenario 2) Although not illustrated, the boundary between the most severe area and the other areas is shown with solid or dashed lines when only the most severe (red) part has a transmittance of 50% and all other levels have a transmittance of 100%.

[0130] (Scenario 3) When multiple levels, such as moderate to normal, are set to 100% transmittance, the boundaries of these multiple level areas are not visible on the screen. Although not illustrated, the boundaries of the multiple level areas are shown using solid or dashed lines. For example, by showing the boundaries between the moderate and mild areas, and between the mild and normal areas, solid lines can be used to show the boundaries of the areas even when multiple levels are set to 100% transmittance.

[0131] <Example 7>

[0132] Figure 16 The monitor 40's image and Figure 13 Similarly, along with endoscopic image 40A and recognition image 40B, recognition image 40BP shows a similar range to recognition image 40B from past examinations of the subject 90. However, with Figure 13 In contrast, in the recognition image 40B, only the red area of ​​the most severe level is displayed in color among the five color regions, while the areas from normal to severe levels are displayed in white.

[0133] The most severe level is the level with the most pixels in the previous 40BP recognition image, in other words, the largest area. The level for color display of the 40B recognition image is not limited to the most severe level and can be appropriately set.

[0134] As described above, in another embodiment of the endoscope system, the camera unit 15 of the endoscope 10 captures an image of the subject 91 inside the body of the examinee 90 and outputs an image signal. The processor 30 performs image processing on the image signal to generate a subject image. By standardizing the sum of the red and green pixel values ​​of each pixel in at least a portion of the subject image using twice the value of the blue pixel of each pixel, an index of each pixel is calculated. Multiple thresholds for identifying multiple regions of the subject as any one of three or more symptom levels and identification colors corresponding to the multiple symptom levels are stored. Based on the index, the multiple thresholds are used to identify the symptom level of each pixel, and the identification color corresponding to the symptom level of each pixel is obtained. An identification image is generated using the identification colors of each pixel of the multiple pixels, and the monitor 40 displays the identification image.

[0135] Another implementation of the program causes the computer to perform the above-described processing.

[0136] In another embodiment, a non-transitory computer-readable storage medium contains a program that enables a computer to perform the above-described processing.

[0137] Furthermore, the range of values ​​described above, such as wavelength, is not limited to the range described above and can be appropriately increased or decreased. Additionally, the insertion portion 11 of the endoscope 10 can also be a rigid endoscope. The present invention is not limited to the embodiments described above, and various modifications and alterations can be made without changing the spirit of the invention.

[0138] Explanation of reference numerals in the attached figures

[0139] 1: Endoscopic device

[0140] 3: Insertion section

[0141] 9: Endoscope

[0142] 10: Endoscopy

[0143] 11: Insertion section

[0144] 11A: Front end

[0145] 11B: Bend

[0146] 11C: Flexible tubing

[0147] 12: Operations Department

[0148] 12A: Button

[0149] 13: General-purpose cables

[0150] 14: Connector

[0151] 15: Camera Unit

[0152] 16: Lighting Unit

[0153] 20: Light source device

[0154] 22: Light source control unit (light source control circuit)

[0155] 23: Light source

[0156] 23B: Component B

[0157] 23G: G-component

[0158] 23R: R component

[0159] 24: Filter combiner

[0160] 30: Processor

[0161] 31: Image Processing Unit (Image Processing Circuit)

[0162] 32: Computing Unit (Computing Circuits)

[0163] 33: Identification Unit (Identification Circuit)

[0164] 34: Color recognition and acquisition unit (color recognition and acquisition circuit)

[0165] 35: Image generation unit (image generation circuit)

[0166] 36: Setting Department

[0167] 40: Monitor

[0168] 40A: Endoscopic image

[0169] 40B, 40BP: Image recognition

[0170] 40C: Color identification overview display

[0171] 40D: Average value

[0172] 50: Memory

[0173] 90: The examinee

[0174] 91: Subject

Claims

1. An endoscope device, characterized in that, have: An endoscope having a camera unit that captures images of at least one object inside a patient's body and outputs image signals; An image processing unit performs image processing on the image signal to generate an image of the subject; The computing unit calculates the index of each pixel by normalizing the sum of the red and green pixel values ​​of each pixel using twice the value of the blue pixel of each pixel of a plurality of pixels in at least a portion of the image of the subject. The memory stores multiple thresholds for identifying multiple regions of the subject as one of three or more symptom levels, and identification colors corresponding to the multiple symptom levels respectively. The identification unit identifies the symptom level of each pixel based on the indicators and using the plurality of thresholds; A color acquisition unit acquires the color corresponding to the symptom level of each pixel; An image generation unit uses the recognition color of each pixel of the plurality of pixels to generate a recognition image; as well as A monitor that displays the identified image.

2. The endoscopic device according to claim 1, characterized in that, The symptom levels are categorized as normal, mild, moderate, severe, and most severe.

3. The endoscopic device according to claim 1, characterized in that, The multiple thresholds are each set within a specified range.

4. The endoscopic device according to claim 1, characterized in that, The differences between the multiple thresholds are approximately the same.

5. The endoscopic device according to claim 2, characterized in that, The first threshold difference between the first threshold used to identify the normal and the mild symptoms and the second threshold used to identify the mild and the moderate symptoms is greater than the second threshold difference between the second threshold and the third threshold used to identify the moderate and the severe symptoms, and the second threshold difference is greater than the third threshold difference between the third threshold and the fourth threshold used to identify the severe and the most severe symptoms.

6. The endoscopic device according to claim 2, characterized in that, The first threshold difference between the first threshold used to identify the normal and the mild symptoms and the second threshold used to identify the mild and the moderate symptoms is less than the second threshold difference between the second threshold and the third threshold used to identify the moderate and the severe symptoms, and the second threshold difference is less than the third threshold difference between the third threshold and the fourth threshold used to identify the severe and the most severe symptoms.

7. The endoscopic device according to claim 2, characterized in that, The memory stores multiple threshold groups, each containing multiple thresholds corresponding to multiple subjects.

8. The endoscopic device according to claim 7, characterized in that, The deviation of the thresholds used to identify the severe and most severe cases in the plurality of threshold groups is less than the deviation of the thresholds used to identify the normal and mild cases.

9. The endoscopic device according to claim 8, characterized in that, The image processing unit performs white balance adjustment using white as a color balance calibration.

10. The endoscopic device according to claim 7, characterized in that, The deviation of the threshold is controlled by the color used in the color balance calibration.

11. The endoscopic device according to claim 10, characterized in that, The image processing unit performs color balance calibration by adjusting the intermediate color between the color of the normal area and the color of the most severe area.

12. The endoscopic device according to claim 1, characterized in that, The calculation unit uses any one of a plurality of calculation formulas corresponding to the plurality of subjects to calculate the index.

13. The endoscopic device according to claim 1, characterized in that, The memory stores error colors as the identification colors for the first and second error pixels. At least one of the red, green, and blue pixel values ​​of the first error pixel is a pixel value below a predetermined lower limit or above a predetermined upper limit. The index of the second error pixel is either below a predetermined lower threshold or above a predetermined upper threshold. The image generation unit uses the incorrect color on the first erroneous pixel and the second erroneous pixel to generate the recognition image.

14. The endoscopic device according to claim 13, characterized in that, A warning is output if the number of the first erroneous pixels, the number of the second erroneous pixels, or the total number of the first erroneous pixels and the second erroneous pixels is greater than a predetermined ratio relative to the total number of pixels.

15. The endoscopic device according to claim 1, characterized in that, The past image is displayed on the monitor along with the recognition image, wherein the past image is the recognition image of the subject during a past examination.

16. The endoscopic device according to claim 15, characterized in that, In the past image, an identification color is selected from the identification colors of each of the plurality of pixels, and only the pixels of the selected identification color are displayed as the identification image, while the other pixels are displayed in white or as an endoscopic image.

17. The endoscopic device according to claim 1, characterized in that, At least one identification color is selected from the identification colors of each of the plurality of pixels, and only the pixels of the selected identification color are displayed in color, while the other pixels are displayed in white.

18. The endoscopic device according to claim 1, characterized in that, The recognition image is made semi-transparent and displayed on the monitor overlaid on the subject image.

19. The endoscopic device according to claim 17, characterized in that, The identified image is made semi-transparent with different transmittance for each of the plurality of identified colors.

20. The endoscopic device according to claim 17, characterized in that, Only the identification color selected from the plurality of identification colors is made semi-transparent.

Images