Processor for endoscope and endoscope system

Abstract

An endoscope treatment instrument includes an image processing unit for calculating a severity of a lesion indicating a degree of progression of a lesion of a living tissue with one value. The image processing unit includes a feature amount calculation section configured to calculate, for each pixel, a first pixel evaluation value indicating a degree of a first feature representing an appearance of a lesion site from an image; a typical value calculation section configured to calculate a first typical evaluation value by aggregating the first pixel evaluation values; and a combination section configured to adjust, at least according to color component information of the image, an influence degree of the first typical evaluation value indicating that the severity changes with a change in the first typical evaluation value, or to adjust, at least according to a second typical evaluation value obtained by aggregating second pixel evaluation values indicating a degree of a second feature different from the first feature, or to calculate the severity of the lesion by using the adjusted influence degree, at least according to the first typical evaluation value.

Description

TECHNICAL FIELD

[0001] The present application relates to an endoscope processor that performs image processing on an image of a living tissue in a body cavity, and an endoscope system. BACKGROUND

[0002] The severity of a lesion site in a living tissue ranges from inflammation in which a mucosal layer of the living tissue is thinned, the surface is rough, and the color is reddish, to an ulcer in which a partial defect is present in the mucosal layer and a layer below the mucosal layer. For example, in an ulcer site in which ulcerative colitis occurs, the ulcer site is whitish due to the presence of white fur and pus-like mucus, and in an inflammation site, the color is reddish due to the presence of an edematous portion and a portion that is prone to bleeding. Such a lesion site can be photographed and observed by an endoscope system.

[0003] However, in order for a surgeon to be able to recognize a healthy site and a lesion site from a color difference included in an endoscope image, a long period of training under the guidance of a skilled person is required. In addition, even a skilled surgeon is not so easy to recognize a lesion site from a subtle difference in color, and careful operation is required. Therefore, it is preferable that the endoscope system provide an evaluation result for quantifying the degree of progression of a lesion site objectively.

[0004] In contrast to this, an endoscope system capable of evaluating the degree of progression of a lesion site in a living tissue with high precision is known (International Publication No. 2019 / 159770). SUMMARY

[0005] PROBLEMS TO BE SOLVED BY THE INVENTION

[0006] The above-described endoscope system includes an image processing unit that calculates a severity of a lesion for quantifying a degree of progression of a lesion site in a living tissue from a living tissue image obtained by photographing the living tissue in a body cavity. The image processing unit includes a feature value calculation section, a typical value calculation section, and a consolidation section.

[0007] The feature value calculation section calculates, for each pixel, a plurality of pixel evaluation values for displaying a degree of each of a plurality of features related to a color component displayed by a lesion site or a color component and a shape, the plurality of pixel evaluation values being able to distinguish each of a plurality of appearance features from a feature of a healthy site in the living tissue according to the color component or the shape displayed by the lesion site, and being a plurality of pixel evaluation values corresponding to the plurality of appearance features.

[0008] The typical value calculation section calculates a typical evaluation value corresponding to each of the plurality of features of the photographed living tissue by consolidating each of the calculated pixel evaluation values for each of the plurality of features.

[0009] The integrating section calculates a value integrated by performing an operation on at least two of the calculated typical evaluation values as a severity of the lesion.

[0010] Thus, the progression of the lesion at the living tissue lesion site can be evaluated with high accuracy.

[0011] The operation using at least two of the typical evaluation values in the integrating section of the above-described endoscope system is a process of performing addition and subtraction on a product obtained by multiplying at least two of the typical evaluation values by a weight coefficient, and in the operation, the type of the operation is changed depending on whether a specific typical evaluation value exceeds a predetermined threshold value. However, in the case where the type of the operation is changed depending on whether a specific typical evaluation value exceeds a predetermined threshold value, when the type of the lesion as an object changes, the above-described threshold value also changes, and thus it is not preferable to make the above-described threshold value consistent with each other depending on the type of the lesion, because this makes the system complicated. Further, when the typical evaluation value fluctuates around the above-described threshold value, the severity as a result of the operation also easily becomes unstable and fluctuates greatly. Thus, it is preferable not to use a parameter such as the above-described threshold value that has been used conventionally to evaluate the progression of the lesion at the lesion site with high accuracy.

[0012] Further, in order to evaluate the progression of the lesion at the lesion site of the living tissue more stably and with higher accuracy than conventionally, it is preferable that the typical evaluation value, which is a value obtained by aggregating the pixel evaluation values of each pixel into one, be stably and appropriately expressed.

[0013] Further, when the endoscope is inserted into a long tubular organ such as the large intestine, the color of the mucosa and the appearance of the blood vessels are different between the portion on the entrance side where the endoscope is inserted and the portion on the inside in the longitudinal direction, and even at the same healthy site, the calculated typical evaluation values are different. Thus, even when the progression of the lesion is the same, the typical evaluation values are different from each other between the portion on the entrance side and the portion on the inside in the longitudinal direction. As such, even when the progression of the lesion is the same, the typical evaluation values change depending on the imaging position. Further, at each imaging position in the longitudinal direction of the long tubular organ, when the severity is calculated using a calculation formula according to the typical evaluation value, when the same calculation formula is used to calculate the severity regardless of the imaging position, even if the organ is actually healthy, there is a case where the typical evaluation value changes and the value of the severity becomes larger depending on the imaging position. In this case, an erroneous determination that there is a lesion site easily occurs at the portion where the value of the severity is large. As such, because the calculated typical evaluation value or the value of the severity changes depending on the imaging position, there is a problem that the evaluation result of the actual progression of the lesion deviates. The evaluation result of the actual progression of the lesion is, for example, a subjective evaluation result of a doctor (for example, a MAYO endoscopic score) or a histological evaluation result.

[0014] Therefore, an object of the present application is to provide an endoscope processor and an endoscope system capable of evaluating the progression degree of a lesion site of a living tissue with high accuracy compared to the past.

[0015] Technical solution for solving the problem

[0016] One embodiment of the present application is an endoscope processor. The endoscope processor includes an image processing unit configured to obtain a severity degree of a lesion of a living tissue, which is represented by one value, from an image of a lesion site of the living tissue obtained by an electronic endoscope configured in such a manner that the living tissue in a body cavity is imaged, and using at least information of the image.

[0017] The image processing unit has:

[0018] a feature quantity calculation section configured to calculate, from the image, a first pixel evaluation value related to a first color component or a shape of the lesion site and indicating a degree of a first feature for each pixel, wherein the first pixel evaluation value is able to distinguish a first feature of an appearance of the lesion site from a feature of a healthy site of the living tissue according to the first color component displayed by the lesion site or the shape of the lesion site, and is a pixel evaluation value corresponding to the first feature;

[0019] a typical value calculation section configured to calculate a first typical evaluation value of the first feature of the living tissue by aggregating the first pixel evaluation values of the pixels in the image;

[0020] an integration section configured to adjust an influence degree of the first typical evaluation value indicating that the severity degree changes with a change in the first typical evaluation value according to second color component information of the image different from the first color component, or to adjust according to a second typical evaluation value obtained by aggregating second pixel evaluation values indicating a degree of a second feature different from the first feature, or to calculate the severity degree of the lesion at least according to the first typical evaluation value.

[0021] Preferably, the severity degree is calculated from a polynomial having at least the first typical evaluation value as an explanatory variable,

[0022] the influence degree is a coefficient related to the first typical evaluation value in the polynomial.

[0023] Preferably, the polynomial is a regression equation related to the severity, which is obtained by performing multiple regression analysis in advance using a reference image in which an evaluation value determined as an evaluation result related to the degree of progression of the lesion, the first pixel evaluation value, and a value of information indicating the second color component or the second pixel evaluation value as explanatory variables, the evaluation value in the reference image as a value of the severity, and a value of the severity as a target variable.

[0024] Preferably, when the first typical evaluation value is set as X and the second typical evaluation value is set as Y, the integration section calculates the severity according to the severity = α1 · X + β1 · Y + γ1 · X · Y + d1 by using coefficients α1, β1, γ1, and a constant d1.

[0025] Preferably, when the first typical evaluation value is set as X and a value of information indicating the second color component is set as Z, the integration section calculates the severity according to the severity = α2 · X + β2 · X · Z + d2 by using coefficients α2, β2, and a constant d2.

[0026] Preferably, the degree of influence of the first typical evaluation value can be adjusted by using, in addition to the second typical evaluation value or the value of information indicating the second color component, a third typical evaluation value obtained by summarizing a third pixel evaluation value indicating a degree of another third feature different from the first feature and the second feature.

[0027] Preferably, each of the first feature and the second feature is any one of a degree of a color component exhibited by the inflamed portion of the living tissue, a degree of a color component exhibited by a blood vessel region indicating a blood vessel extending in a striped shape on the living tissue, and a degree of a color component exhibited by the ulcerated portion of the living tissue.

[0028] Preferably, the first typical evaluation value is a number of aggregated intervals of intervals having a frequency of one or more in a histogram of the first pixel evaluation values of the image of the lesion portion of the living tissue.

[0029] Preferably, the first typical evaluation value is a value indicating a degree of a color component exhibited by the inflamed portion of the living tissue.

[0030] Preferably, the second color component information is a feature amount related to a color component exhibited by a residue introduced from the outside into the body lumen and present on the living tissue.

[0031] Another embodiment of the present application is a processor for an endoscope. The processor for an endoscope includes an image processing unit configured to calculate a value of a severity of a lesion from an image of a lesion site of a living tissue obtained by an electronic endoscope, using information of the image.

[0032] The image processing unit includes:

[0033] a feature amount calculation section configured to calculate, from the image, a pixel evaluation value related to the color component or the shape of the lesion site and used to represent a degree of each of the plurality of features for each pixel, wherein the pixel evaluation value is able to distinguish a plurality of appearance features exhibited by the lesion site from features of a healthy site of the living tissue according to the color component exhibited by the lesion site or the shape of the lesion site, and is an evaluation value corresponding to each of the appearance features;

[0034] a typical value calculation section configured to calculate, for one of the images, a typical evaluation value of each of the plurality of features of the living tissue by aggregating the pixel evaluation values of the pixels in the image;

[0035] a photographing position information processing section that associates information of a photographing position in a body lumen at the time of photographing the image with the image;

[0036] an integration section configured to calculate, for an evaluation target image photographed by the electronic endoscope, a value of the severity of the lesion from the typical evaluation value of each of the plurality of features, and to set a calculation process of the severity adjusted according to the information of the photographing position corresponding to the evaluation target image when calculating the value of the severity.

[0037] Preferably, the integration section calculates the value of the severity according to a relational expression that establishes a correlation between the severity and the typical evaluation value of each of the plurality of features,

[0038] and sets a value corresponding to the information of the photographing position as a coefficient related to the typical evaluation value in the relational expression.

[0039] The endoscope processor further includes a storage unit that holds, for each imaging position of the electronic endoscope, two reference values related to the typical evaluation value as values determined from the information of the imaging position of the reference image, wherein the typical evaluation value is a value set from a plurality of reference images obtained by imaging a plurality of lesion sites having different lesion progression degrees with the electronic endoscope, and in this case, preferably, the integration unit calculates the value of the severity by using a relational expression that previously establishes a correlation between the normalized evaluation value and the severity, wherein the normalized evaluation value is a value obtained by normalizing the typical evaluation value of the reference image by the two reference values corresponding to the information of the imaging position of the reference image imaged by the electronic endoscope; and the value of the severity is calculated by inputting the normalized evaluation value into the relational expression, wherein the normalized evaluation value is a value obtained by normalizing the typical evaluation value of the evaluation target image by the two reference values determined from the information of the imaging position of the evaluation target image.

[0040] The endoscope processor further includes a storage unit that holds, for each imaging position of the electronic endoscope, two reference values related to the typical evaluation value as values determined from the information of the imaging position of the reference image, wherein the typical evaluation value is a value set from a plurality of reference images obtained by imaging a plurality of lesion sites having different lesion progression degrees with the electronic endoscope, and in this case, preferably, the integration unit calculates the value of the severity by using a relational expression that previously establishes a correlation between the normalized evaluation value and the severity, wherein the normalized evaluation value is a value obtained by normalizing the typical evaluation value of the reference image by the two reference values corresponding to the information of the imaging position of the reference image imaged by the electronic endoscope; and the value of the severity is calculated by inputting the normalized evaluation value into the relational expression, wherein the normalized evaluation value is a value obtained by normalizing the typical evaluation value of the evaluation target image by the two reference values determined from the information of the imaging position of the evaluation target image.

[0041] Preferably, the relational expression is a regression equation of a polynomial extracted by performing a multiple regression analysis, so that the value of the severity calculated from the typical evaluation value of each of the plurality of features corresponds to the evaluation result of the reference image for which the evaluation result of the lesion progression degree is prepared in advance.

[0042] Preferably, the information of the imaging position is position information of one region when a region of a tubular organ in a longitudinal direction is divided into a plurality of regions.

[0043] Another embodiment of the present application is an endoscope system including: an electronic endoscope configured to image a living body in a body cavity; and the endoscope processor connected to the electronic endoscope.

[0044] Inventive Effects

[0045] According to the above-described endoscope system, the progression degree of a lesion at a living tissue lesion site can be evaluated with higher accuracy than ever.

[0046] According to the above-described endoscope processor and endoscope system, when the progression degree of a lesion is intended to be evaluated, the progression degree of the lesion can be evaluated stably and with high accuracy regardless of the imaging position of an evaluation target image. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a diagram illustrating an example of processing performed by an endoscope system in one embodiment.

[0048] Figure 2 is a block diagram showing the structure of an endoscope system in one embodiment.

[0049] Figure 3 is a diagram illustrating the structure of an image processing unit for calculating the severity in the endoscope processor in one embodiment.

[0050] Figure 4 is an example diagram illustrating a reference axis in a color space used in the endoscope processor of one embodiment.

[0051] Figure 5 is a diagram illustrating an offset angle calculation method for calculating the living tissue redness used in the endoscope processor of one embodiment.

[0052] Figure 6 is a diagram schematically illustrating an example of a living tissue image and an example of a color mapping image obtained by a conventional method.

[0053] Figure 7 is a diagram illustrating an example of a blood vessel region extraction method performed by the endoscope processor in one embodiment.

[0054] Figure 8 is a diagram showing an example of a filter coefficient when a template used in the endoscope processor of one embodiment is used as a spatial filter.

[0055] Figure 9 is a diagram illustrating an example of a typical evaluation value used in the endoscope processor of one embodiment.

[0056] Figure 10 is a diagram showing an example of the degree of influence of a typical value of living tissue redness used in the endoscope processor of one embodiment.

[0057] Figure 11FIG. 1 is a diagram showing an example of a flow of a severity calculation method performed by an endoscope processor of one embodiment.

[0058] Figure 12 FIG. 2 is a diagram for describing an example of a severity calculation process performed by an endoscope processor of one embodiment.

[0059] Figure 13 FIG. 3 is a diagram illustrating the structure of an image processing unit for calculating severity in an endoscope processor of one embodiment.

[0060] Figure 14 FIG. 4 is a diagram showing an example of a flow of a severity calculation method performed by an endoscope processor of one embodiment.

[0061] Figure 15A FIG. 5 is a diagram showing an example of a flow of a severity calculation method performed by an endoscope processor of one embodiment.

[0062] Figure 15B FIG. 6 is a diagram showing an example of a flow of a severity calculation method performed by an endoscope processor of one embodiment.

[0063] Figure 15C FIG. 7 is a diagram showing an example of a flow of a severity calculation method performed by an endoscope processor of one embodiment. DETAILED DESCRIPTION

[0064] Before the endoscope processor and the endoscope system in one embodiment of the present application are described below with reference to the accompanying drawings, the concept of the endoscope system in one embodiment is described.

[0065] (Calculation of Severity in Endoscope System)

[0066] According to the progress degree of the lesion, there are various cases from inflammation to ulcer in the lesion site in the living tissue. For example, a conventional endoscope system calculates, according to an image of the living tissue, a plurality of pixel evaluation values corresponding to a plurality of appearance features, each of which is able to distinguish each of a plurality of appearance features exhibited by the lesion site from a feature of a healthy site of the living tissue according to a color component exhibited by the lesion site or a shape of the lesion site. Further, in the conventional endoscope system, a plurality of typical evaluation values are calculated by integrating the pixel evaluation values of the respective pixels calculated according to each of the plurality of appearance features, and a value of the severity of the lesion is calculated as a value of the severity of the lesion by integrating one value obtained by performing an operation on at least two of the calculated plurality of typical evaluation values. The above operation is performed according to a formula that expresses a relationship between the severity and the plurality of typical evaluation values in advance. However, in this formula, the degree of influence of one of the typical evaluation values on the severity, i.e., the degree to which the severity changes with the typical evaluation value, is constant, and thus sometimes deviates from an evaluation result of the actual progress degree of the lesion. Further, in order to change the degree of influence of the typical evaluation value on the severity, there has been a case where the calculation formula for calculating the severity is changed according to the magnitude of the typical evaluation value with respect to one threshold value in the past.

[0067] In contrast to this, in the endoscope system according to one embodiment of the present application, when the severity of the lesion is calculated according to the plurality of typical evaluation values, the severity of the lesion is calculated at least according to a first typical evaluation value by adjusting the degree of influence of the first typical evaluation value that is one of the plurality of typical evaluation values. The adjustment of the degree of influence can be performed according to information of a second color component of an image different from a first color component of an image focused on when the first typical evaluation value is calculated, or at least according to a second typical evaluation value that indicates a degree of another second feature different from the first feature. In addition, the adjustment of the degree of influence is not limited to being performed according to the second typical evaluation value, and can be performed according to a third typical evaluation value, a fourth typical evaluation value, or the like that indicates a degree of another third feature, a fourth feature, or the like different from the first feature and the second feature.

[0068] Appearance features include the degree of a specific color component displayed by the lesion site. For example, the degree of redness exhibited by an inflamed area. In this case, the pixel evaluation value of each pixel obtained by quantifying the degree of this feature is, for example, the tissue redness obtained by quantifying the degree of redness. Furthermore, another appearance feature is the degree of color component contained in portions of a specific shape within the photographed tissue. The pixel evaluation value of each pixel obtained by quantifying the degree of this feature is, for example, the vascular redness obtained by quantifying the degree of redness contained in the vascular region extending in a striped pattern in and around the lesion site. Additionally, another appearance feature is the specific color component exhibited by the ulcer site, such as the degree of whiteness.

[0069] Appearance features also include shape-related features, such as the degree to which the surface of the lesion is uneven or the degree to which a predetermined shape is formed, such as surface shaping.

[0070] Figure 1 This diagram illustrates an example of processing performed by an endoscope system in one embodiment. Figure 1 In the example shown, at least the typical assessment values ​​include a typical value for the redness of the living tissue to represent the degree of inflammation, a typical value for the redness of the blood vessels to represent the degree of redness in the vascular region, and a typical value for the whiteness of the living tissue to represent the degree of ulceration. The calculation methods for these typical assessment values ​​will be described later.

[0071] On the other hand, as preprocessing, multiple reference images are prepared with pre-determined assessment results of the lesion's progression (severity), and multiple typical assessment values ​​are pre-calculated for each of these reference images. A prediction formula is calculated using these multiple typical assessment values ​​so that the predicted severity, after combining multiple typical assessment values ​​including these three typical assessment values, most closely corresponds to the assessment results of the lesion's progression (severity) determined in each reference image. This prediction formula is extracted and set before calculating the severity in the endoscopic system. Furthermore, the assessment results of the lesion's progression (severity) can be, for example, a physician's subjective assessment result (e.g., MAYO endoscopic score), a histological assessment result, or a pathological diagnostic score (Geboes score, RHI (Reactive Hyperemia Index)). For example, in the RHI (Reactive Hyperemia Index), 34 levels from 0 to 33 are used for assessment.

[0072] Next, multiple typical assessment values ​​are calculated, and the severity is calculated using a prediction formula based on these calculated typical assessment values. These typical assessment values ​​include typical values ​​of the redness of the living tissue, the redness of the blood vessels, and the whiteness of the living tissue obtained from images taken by electronic endoscopy.

[0073] At this time, in the prediction formula, the degree of influence of the first representative evaluation value, which is one of the plurality of representative evaluation values, on the severity is adjusted. The adjustment of the degree of influence can be made in accordance with information of a second color component of an image different from a first color component of an image on which attention is focused when the first representative evaluation value is calculated, or at least in accordance with a second representative evaluation value that indicates a degree of a second feature different from the first feature.

[0074] For example, the degree of influence of a representative value of redness of living tissue, which indicates a degree of inflammation, on the severity is adjusted in accordance with a representative value of redness of blood vessels. Alternatively, the degree of influence of the representative value of redness of living tissue is adjusted in accordance with color component information of blue. When the prediction formula is expressed by a polynomial that uses at least the first representative evaluation value and the second representative evaluation value, the polynomial includes, for example, a term obtained by multiplying a product of the first representative evaluation value and the second representative evaluation value by a coefficient associated with the product, in addition to a term obtained by multiplying a coefficient associated with each of the first representative evaluation value and the second representative evaluation value and then performing addition and subtraction. Thus, in this case, the degree of influence of the first representative evaluation value on the severity includes, in addition to a coefficient associated with the first representative evaluation value, at least a coefficient associated with the product of the first representative evaluation value and the second representative evaluation value and the product of the second representative evaluation value. Thus, the degree of influence of the first representative evaluation value on the severity is adjusted at least in accordance with the second representative evaluation value.

[0075] The information of the second color component is, for example, a feature amount associated with a color component exhibited by a residue that is introduced from the outside into a body lumen and exists on the living tissue.

[0076] Thus, the degree of influence of the first representative evaluation value on the severity is adjusted in accordance with the information of the second color component or in accordance with the second representative evaluation value that indicates a degree of the second feature, and thus it is possible to reduce a deviation from an actual degree of progression of a lesion (severity). Thus, compared with the past, it is possible to evaluate a degree of progression of a lesion at a lesion site of living tissue with high accuracy.

[0077] Figure 2 is a block diagram showing a configuration of an electronic endoscope system 1 in one embodiment of the present application. As shown in Figure 2 The electronic endoscope system 1 includes an electronic endoscope 100, an electronic endoscope processor 200, a monitor 300, and a printer 400.

[0078] The electronic endoscope processor 200 includes a system controller 202 and a timing controller 206. The system controller 202 executes various programs stored in the memory 204 and integrates control of the entire electronic endoscope system 1. Furthermore, the system controller 202 changes various settings of the electronic endoscope system 1 according to instructions input to the operation panel 208 by the user (surgeon or assistant). The timing controller 206 outputs clock pulses to each circuit within the electronic endoscope system 1 to adjust the operating time of each component.

[0079] The processor 200 for electronic endoscopes includes a light source unit 230 for providing illumination light to the electronic endoscope 100. Although not shown in the figure, the light source unit 230 includes a high-brightness lamp, such as a xenon lamp, metal halide lamp, mercury lamp, or halogen lamp, which emits white illumination light by receiving drive power supplied by a lamp power supply. The light source unit 230 is configured such that the illumination light emitted from the high-brightness lamp, after being focused by a condenser lens (not shown in the figure), is incident on the incident end of the optical fiber bundle LCB (Light Carrying Bundle) 102 of the electronic endoscope 100 via a dimming device (not shown in the figure).

[0080] Alternatively, the light source unit 230 may include multiple light-emitting diodes (LEDs) for emitting light in a predetermined color band. The light source unit 230 is configured to synthesize the light emitted from the LEDs using optical elements such as a dichroic mirror, use the synthesized light as illumination light, and then focus it through a condenser lens (not shown) before incident on the incident end of the LCB (light beam carrier) 102 of the electronic endoscope 100. A laser diode can be used instead of the LEDs. Compared to other light sources, LEDs and laser diodes have the advantages of low power consumption and low heat generation, thus enabling the acquisition of bright images while suppressing power consumption and heat generation. By acquiring bright images, the accuracy of assessment values ​​related to lesions described later can be improved.

[0081] In addition, Figure 2 In the example shown, the light source unit 230 is built into the electronic endoscope processor 200, but it can also be provided in the electronic endoscope system 1 as a separate device from the electronic endoscope processor 200. Furthermore, the light source unit 230 can also be provided at the front end of the electronic endoscope 100 described later. In this case, the LCB 102 for guiding the illumination light is not required.

[0082] The illumination light emitted by the light source 230 is not limited to white light, but can also be special light such as narrowband light with a specified narrow wavelength.

[0083] The illumination light incident into the LCB 102 from the incident end propagates within the LCB 102, and is emitted from the emission end of the LCB 102 arranged within the front end portion of the electronic endoscope 100, and is irradiated to the subject via the light distribution lens 104. The reflected light from the subject forms an optical image on the light-receiving surface of the solid-state image pickup element 108 via the objective lens 106.

[0084] The solid-state image pickup element 108 is, for example, a monolithic color CCD (Charge-Coupled Device) image sensor for arranging an IR (Infra Red) cut filter 108a, a color filter 108b employing a Bayer array, and the like various filters on the light-receiving surface, and generates R (Red), G (Green), B (Blue) respective primary color signals from the optical image imaged on the light-receiving surface. Instead of the monolithic color CCD image sensor, a monolithic color CMOS (Complementary Metal Oxide Semiconductor) image sensor can also be used. The CMOS image sensor generally has a tendency that the image as a whole becomes dark compared to the CCD image sensor. Therefore, when the CMOS image sensor is used, the advantageous effect of suppressing the fluctuation of the severity of the lesion in the lesion site due to the brightness of the image is more noticeable in the quantification processing for evaluating the lesion described later. Thus, the electronic endoscope 100 captures the living tissue within the body cavity using the solid-state image pickup element 108.

[0085] A driver signal processing circuit 112 is provided within the connection portion of the electronic endoscope 100. The driver signal processing circuit 112 generates image signals (luminance signal Y, color difference signals Cb, Cr) after performing predetermined signal processing such as color interpolation, matrix operation, and the like on the primary color signals input from the solid-state image pickup element 108, and outputs the generated image signals to an image processing unit 220 of an electronic endoscope processor 200. Further, the driver signal processing circuit 112 accesses a memory 114 and reads the unique information of the electronic endoscope 100. The unique information of the electronic endoscope 100 recorded in the memory 114 includes, for example, the number of pixels, the sensitivity, the operable frame rate, the model, and the like of the solid-state image pickup element 108. The driver signal processing circuit 112 outputs the unique information read from the memory 114 to a system controller 202.

[0086] The system controller 202 performs various operations based on the unique information of the electronic endoscope 100, and generates a control signal. The system controller 202 uses the generated control signal to control the operation and timing of each circuit within the electronic endoscope processor 200 so as to perform processing in accordance with the electronic endoscope 100 connected to the electronic endoscope processor 200.

[0087] The timing controller 206 supplies clock pulses to the driver signal processing circuit 112, the image processing unit 220, and the light source section 230 in accordance with timing control performed by the system controller 202. The driver signal processing circuit 112 drives and controls the solid-state imaging element 108 in accordance with the clock pulses supplied from the timing controller 206, in synchronization with the timing of the image frame rate at which processing is performed on the electronic endoscope processor 200 side.

[0088] Under the control of the system controller 202, the image processing unit 220 generates a video signal for displaying an endoscopic image or the like on the monitor, based on the image signal input from the driver signal processing circuit 112, and outputs the video signal to the monitor 300. Further, the image processing unit 220 calculates the severity of a lesion quantified using color components and shape information of an image of a lesion site of a living tissue, based on the image of the lesion site of the living tissue obtained by the electronic endoscope 100. In addition, the image processing unit 220 generates a color-mapped image in which colors are replaced, based on the redness of the living tissue and the redness of blood vessels, or the like, obtained when quantification processing is performed for calculating the severity. The image processing unit 220 generates video signals of the severity information and the color-mapped image for display on the monitor, and outputs the video signals to the monitor 300. Thereby, a surgeon can receive the severity of a lesion in a living tissue of interest through the image displayed on the display screen of the monitor 300. The image processing unit 220 outputs the color-mapped image and the severity information to the printer 400 as needed.

[0089] The electronic endoscope processor 200 is connected to the server 600 via the NIC (Network Interface Card) 210 and the network 500. The electronic endoscope processor 200 can download information related to an endoscopic examination (e.g., electronic medical record information of a patient, surgeon information) from the server 600. The downloaded information is displayed on the display screen of the monitor 300 or the operation panel 208, for example. In addition, the electronic endoscope processor 200 uploads an endoscopic examination result (endoscopic image data, examination conditions, image analysis results, findings of a surgeon, or the like) to the server 600, so that the endoscopic examination result can be saved on the server 600.

[0090] Figure 3 is a diagram illustrating the configuration of the image processing unit 220 that performs quantification processing for calculating the severity of a lesion of a living tissue, in which the degree of a lesion feature is expressed by a numerical value.

[0091] The image processing unit 220 is a portion for calculating the severity of the lesion by quantifying the degree of progression of the lesion in the living tissue from the living tissue image obtained by the electronic endoscope 100. The image processing unit 220 is provided with a preprocessing portion 220a, a feature quantity calculating portion 220b, a representative value calculating portion 220c, and a synthesizing portion 220d.

[0092] In one embodiment, the plurality of representative evaluation values are a first representative evaluation value and a second representative evaluation value, and the feature quantity calculating portion 220b calculates a living tissue redness that quantifies the degree of redness of the living tissue for each pixel as a first pixel evaluation value for calculating the first representative evaluation value, and calculates a blood vessel redness that quantifies the redness of the blood vessel region extending in a stripe shape on the living tissue as a second pixel evaluation value for calculating the second representative evaluation value. Hereinafter, the manner for calculating the living tissue redness and the blood vessel redness will be described.

[0093] The preprocessing portion 220a is a portion that performs preprocessing on the image for evaluating the redness exhibited by the living tissue. As one example, the preprocessing portion 220a performs various processes such as RGB conversion, color space conversion, setting of a reference axis, color correction, and the like.

[0094] The preprocessing portion 220a converts the image signal (luminance signal Y, color difference signals Cb, Cr) input from the driver signal processing circuit 112 into image color components (R, G, B) using predetermined matrix coefficients.

[0095] The preprocessing portion 220a also performs color space conversion for orthographically projecting the image data converted into image color components on an RG plane. Specifically, the image color components of each pixel in the RGB color space defined by RGB 3 primary colors are converted into image color components of RG. Conceptually, the image color components of each pixel in the RGB color space are plotted in the RG plane (for example, in the region of the RG plane taking the pixel values of the R component = 0 to 255 and the pixel values of the G component = 0 to 255) in accordance with the pixel values of the R, G components. Hereinafter, for ease of explanation, the point of the image color components of each pixel in the RGB color space and the point of the image color components plotted in the RG color space will be referred to as a "pixel correspondence point". The respective image color components of RGB of the RGB color space are, for example, color components of wavelengths of 620 to 750 nm, 495 to 570 nm, and 450 to 495 nm in that order. In addition, the color components constitute a color space (also including a color plane). Hue and saturation are excluded from the "color components".

[0096] The preprocessing portion 220a sets a reference axis in the RG plane required for evaluating the living tissue redness and the blood vessel redness.

[0097] In living tissue within a patient's body cavity, the R component of the image color components dominates relative to other components (G and B components) due to the influence of hemoglobin pigments, etc. When the lesion is of low progression and inflamed, the more severe the inflammation, the stronger the red (R component) becomes relative to the other colors (G and B components). However, images captured within a body cavity can exhibit color variations depending on shooting conditions that affect brightness (e.g., illumination). For example, shadowed areas untouched by illumination are black (achromatic, e.g., the R, G, and B image color components have values ​​of zero or near zero), while areas under strong illumination that are positively reflected are white (achromatic, e.g., when the R, G, and B image color components are 8-bit grayscale, the value is 255 or near 255). That is, even when photographing the same inflamed site, the stronger the illumination, the larger the pixel value of that inflamed site. Therefore, depending on the illumination, the color component values ​​of the image can sometimes be values ​​unrelated to the intensity of inflammation.

[0098] Normally, healthy areas within body cavities that are not inflamed are completely covered by mucous membranes. In contrast, inflamed areas within body cavities are not completely covered by mucous membranes. Specifically, as blood vessels dilate, blood and body fluids leak from them, causing the mucous membrane to become relatively thinner and making the blood's color easily visible. Mucous membranes are generally white but slightly yellowish, and the color projected onto an image (yellow) varies depending on its depth (mucous membrane thickness). Therefore, the depth of the mucous membrane is considered one of the indicators for assessing the degree of inflammation.

[0099] Therefore, as Figure 4 As shown, in the RG color space, the straight line passing through (50,0) and (255,76) is set as one of the reference axes, and the straight line passing through (0,0) and (255,192) is also set as one of the reference axes. For ease of explanation, the former reference axis is called "hemoglobin variation axis AX1", and the latter reference axis is called "mucosal variation axis AX2". Figure 4 This is a diagram illustrating an example of a reference axis within a color space used in one implementation.

[0100] Figure 4 The plot shown is the result of analyzing multiple reference images within the body cavity. The reference images used in the analysis include examples of inflammation at various stages, such as those with the highest degree of inflammation (the most severe level of inflammation) and those with the lowest degree of inflammation (images that can essentially be considered healthy areas). Furthermore, in Figure 4In the example shown, only a part of the plots obtained by the analysis result is shown for the sake of clarity of the drawing. The number of plots actually obtained by the analysis result is much larger than Figure 4 the number of plots shown.

[0101] As described above, the more serious the inflammation, the more the R component in the color components of the image is heavy relative to the other components (G component and B component). Therefore, in the Figure 4 In the example shown, on the boundary line between the plot-distributed region and the non-distributed region, the axis on the boundary line closer to the R axis than the G axis, and the axis on the boundary line passing through (50, 0) and (255, 76) are set as the axis with high correlation to the part with the most serious degree of inflammation, i.e., the part with the highest degree of inflammation. This axis is the hemoglobin change axis AX1. On the hemoglobin change axis AX1, the plots corresponding to the most serious degree of inflammation of the inflamed part photographed under various photographing conditions, e.g., the irradiation condition of the illumination light, are superimposed. Therefore, the hemoglobin change axis AX1 is the axis on which the pixel corresponding points plotted as the degree of inflammation of the living tissue increases are more converged.

[0102] On the other hand, the closer to the healthy part, the more the G component (or the B component) in the color components of the image is heavy relative to the R component. Therefore, in the Figure 4 In the example shown, on the boundary line between the plot-distributed region and the non-distributed region, the axis on the boundary line closer to the G axis than the R axis, and the axis on the boundary line passing through (0, 0) and (255, 192) are set as the axis with high correlation to the part with the least degree of inflammation, i.e., the part with the lowest degree of inflammation, which can be considered as the healthy part. This axis is the mucous membrane change axis AX2. On the mucous membrane change axis AX2, the plots corresponding to the part with the least degree of inflammation, i.e., the part which can be considered as the normal part, photographed under various photographing conditions, e.g., the irradiation condition of the illumination light, are superimposed. Therefore, the mucous membrane change axis AX2 is the axis on which the pixel corresponding points plotted as the degree of inflammation decreases (the closer to the healthy part) are more converged.

[0103] As a supplement, the part of the lesion site with the highest degree of lesion progression is accompanied by bleeding. On the other hand, the part with the lowest degree of lesion progression is essentially a normal healthy part, and is therefore completely covered with the mucous membrane. Therefore, it can be considered that Figure 4 The plot distribution in the RG color space shown is in the region sandwiched between the axis with the highest correlation to the color of blood (hemoglobin pigment) and the axis with the highest correlation to the color of the mucous membrane. Therefore, on the boundary line between the plot-distributed region and the non-distributed region, the boundary line closer to the R axis (large proportion of R component) corresponds to the axis indicating the inflamed part with the highest degree of inflammation (hemoglobin change axis AX1), and the boundary line closer to the G axis (large proportion of G component) corresponds to the axis indicating the inflamed part with the lowest degree of inflammation (mucous membrane change axis AX2).

[0104] After the reference axis is set in this way, a process for calculating a vital tissue redness representing a redness degree of a living tissue described later is executed for the color components of the orthographic projected image. The color correction is executed for the orthographic projected pixel data before the process for calculating the vital tissue redness is executed.

[0105] Figure 4 The illustrated reference axis is an example, and the reference axis differs depending on the disease type.

[0106] The pre-processing section 220a executes color correction for the color components of the image represented in the RG color space before calculating the inflammation evaluation value. The correction matrix coefficients are stored in a memory not shown in the figure. The pre-processing section 220a corrects the pixel data (R, G) that are the pixel corresponding points within the RG color space of each pixel using the correction matrix coefficients so that the inflammation evaluation value described later does not produce an error when the same inflamed site is imaged using different electronic endoscope systems (in other words, in order to suppress individual errors of the electronic endoscope), as shown in the following equation.

[0107]

[0108] R new : corrected pixel data (R component)

[0109] G new : corrected pixel data (G component)

[0110] M 00 ~ M 11 : correction matrix coefficients

[0111] R: pixel data before correction (R component)

[0112] G: pixel data before correction (G component)

[0113] The feature quantity calculation section 220b selects one focus pixel from the pixels, and calculates a shift angle for calculating the vital tissue redness and the inflammation degree from the color component information of the selected focus pixel. That is, the quantization process for quantifying the redness of the vital tissue is executed from the color component information of the pixel. Figure 5 is a diagram illustrating a shift angle calculation method for calculating the vital tissue redness used in one embodiment. Specifically, as shown in the figure, the feature quantity calculation section 220b calculates the shift angle for calculating the vital tissue redness from the color component information of the focus pixel. Figure 5As shown, the feature quantity calculation section 220b sets the intersection of the hemoglobin change axis AX1 and the mucosa change axis AX2 as a reference point O', and calculates an offset angle Θ at which the direction of the line segment L connecting the reference point O' and the pixel corresponding point P of the pixel of interest is offset with respect to the hemoglobin change axis AX1. In addition, the reference point O' is located at coordinates (-150, -75). An example is given in which the reference point O' is set at coordinates (-150, -75), but the present application is not limited thereto. The above reference point O' can be changed as appropriate, and for example, can be the intersection of the R axis and the G axis in the RG color space.

[0114] As the reference point O', a preferable coordinate position is, for example, a position at which the error of the evaluation result due to brightness fluctuation can be reduced. Specifically, the reference point O' is preferably set by previously finding a point at which the error between the evaluation result of a dark portion (brightness less than a predetermined value) and the evaluation result of a non-dark portion (brightness greater than or equal to a predetermined value) is minimized.

[0115] Further, for example, when the reference point O' is set between coordinates (-10, -10) and (10, 10), compared to the case in which coordinates (-150, -75) or the like are set as the reference point O', the amount of change in the angle Θ when the pixel corresponding point changes increases, and thus the resolution is improved. Thereby, a high-precision evaluation result can be obtained.

[0116] On the other hand, by setting the reference point O' between coordinates (-50, -50) and (-200, -200), the evaluation result indicating the degree of inflammation is less likely to be affected by noise.

[0117] When the image brightness of the living tissue in the body cavity changes depending on the change in the irradiation condition of the white light, although the image color is affected by individual differences, the imaging position, and the inflammation state, and the like, in the RG color space, at the inflammation site with the highest severity, it changes along the hemoglobin change axis AX1, and at the inflammation site with the lightest degree of inflammation, it changes along the mucosa change axis AX2. Further, it can be inferred that the image color of the inflammation site when the degree of inflammation is moderate also changes in the same tendency. That is, when the pixel corresponding point corresponding to the inflammation site changes depending on the irradiation condition of the illumination light, it is offset in the azimuth direction from the reference point O'. In other words, when the pixel corresponding point corresponding to the inflammation site changes depending on the irradiation condition of the illumination light, the offset angle Θ with respect to the mucosa change axis AX2 constantly moves, and the distance from the reference point O' changes. This means that the offset angle Θ is a parameter that is substantially not affected by the change in the image brightness.

[0118] The smaller the offset angle θ, the heavier the R component is relative to the G component, which indicates that the redness in the diseased site is relatively large. In addition, the larger the offset angle θ, the heavier the G component is relative to the R component, which indicates that the redness is relatively small. Therefore, the feature quantity calculation section 220b normalizes the angle θ so that the value is 255 when the offset angle θ is zero, and the value is zero when the offset angle θ is θ MAX MAX is equal to the angle formed by the hemoglobin change axis AX1 and the mucosa change axis AX2. That is, the feature quantity calculation section 220b quantifies the degree of redness based on the information of the color components of each of the concerned pixels, thereby calculating the living tissue redness (1st pixel evaluation value) included in the range of 0 to 255.

[0119] In addition, the concerned pixels are selected one by one for all the pixels of the image.

[0120] In addition, in the example shown in FIG. 8, the RG color space is used as the color space, but the RB color space can be used instead of the RG color space. Figure 5

[0121] The feature quantity calculation section 220b calculates the living tissue redness based on the offset angle θ to be used as the 1st pixel evaluation value, but depending on the situation, the living tissue whiteness (3rd pixel evaluation value) described later, which indicates the degree of ulceration characteristics of the living tissue, can also be calculated. For example, by performing gain adjustment that assigns a linear gain (gain) to each color component pixel value of each pixel of the living tissue image, substantially expanding the dynamic range around the color gamut unique to the lesion, performing tone enhancement processing for enhancing the effective resolution of color expression, for example, it is possible to display the ulceration site including the white fur of ulcerative colitis and the pus-like mucus as white, thereby distinguishing the inflamed site including edema or easy bleeding as red and the normal site as green by the color components. The whiteness of the living tissue can be calculated using the offset angle with respect to a reference axis different from the hemoglobin change axis AX1, which is represented on the color space with 2 color components (2 out of R, G, and B components) or 3 color components (R, G, and B components) as the coordinate axes, as shown in FIG. 9. In addition, the tone enhancement processing is performed by the pre-processing section 220a. Figure 5

[0122] ​​​The feature quantity calculation section 220b further generates a color map image for mosaicking the image of the living tissue in the display color that changes as the living tissue redness changes. In order to be able to display the color map image, a table in which the pixel evaluation value corresponds to a predetermined display color is stored in a storage area or the like not shown in the drawing. For example, in this table, every 5 values correspond to a different display color. Illustratively, when the pixel evaluation value is in the range of 0 to 5, it corresponds to blue, and every time the pixel evaluation value increases by 5 values, it corresponds to a different display color in the order of arrangement of the colors in the hue circle, and when the pixel evaluation value is in the range of 250 to 255, it corresponds to red. The greater the redness of the living tissue, the closer the display color is to the warm color system from the cold color system, for example, from blue to yellow and even red. The feature quantity calculation section 220b refers to the above table and determines the display color of the selected pixel of interest on the color map image according to the living tissue redness of the pixel of interest.

[0123] In this way, the feature quantity calculation section 220b generates a color map image for giving color according to the living tissue redness.

[0124] Further, the feature quantity calculation section 220b judges the probability of the blood vessel region of the living tissue in the image obtained by photographing the living tissue according to the shape representing the blood vessel, and extracts the blood vessel region as needed by the probability thus found.

[0125] As Figure 6 shown, in the image of the living tissue photographed, the vicinity of the lesion site X also includes an image of a striped blood vessel region Y that can be seen through the mucosa. In the color map image shown, the blood vessel region Y is displayed in the same color as the inflammation site. Figure 6 Figure 6 is a drawing schematically illustrating an example of a living tissue image and an example of a color map image obtained by a conventional method.

[0126] The feature quantity calculation section 220b finds the probability of the blood vessel region Y and extracts the blood vessel region Y according to the probability.

[0127] Figure 7 is a drawing illustrating an example of a method for extracting the blood vessel region Y in one embodiment.

[0128] ​The feature calculation unit 220b calculates a matching degree, representing the correlation between the shape of the examination target region AR (a portion of the biopsy image) and each straight line shape of multiple templates TP1 to TP4. Among the matching degrees corresponding to each of the multiple templates TP1 to TP4, the highest matching degree with the largest value is set as the probability of the vascular region Y in the examination target region AR. Templates TP1 to TP4 are composed of pixels and have multiple straight line shapes with different straight line extension directions. Each pixel in templates TP1 to TP4, combined with its respective straight line shape, has a pixel value. For example... Figure 7 As shown, by sequentially overlapping and moving the examination target region AR along the arrow direction starting from the end of the image, the correlation between the pixel evaluation value of the image within the examination target region AR and the value of each corresponding pixel in templates TP1 to TP4 is calculated. According to one embodiment, templates TP1 to TP4, as shapes representing blood vessels, have four straight line shapes extending in four different directions. When the examination target region AR contains a blood vessel region, since the pixel values ​​within the examination target region AR contain information about the characteristic shape of the blood vessel extending in a striped pattern, the blood vessel region Y can be extracted using an image with pixel evaluation values ​​set according to the offset angle θ. Templates TP1 to TP4 have... Figure 6 The values ​​of each pixel corresponding to the white and black areas are shown. Therefore, according to one embodiment, the matching degree is the correlation coefficient between the pixel values ​​of templates TP1 to TP4 and the pixel evaluation value corresponding to the inspection target area AR. Furthermore, according to one embodiment, the matching degree may also be the sum of the values ​​of each pixel of templates TP1 to TP4 as filter coefficients of a spatial filter, and the multiplication of each of these filter coefficients with the image evaluation value of the pixel corresponding to the inspection target area AR.

[0129] Among the matching degrees calculated for each of templates TP1 to TP4, the highest matching degree with the largest value is used as the value representing the probability of the vascular region and is assigned to the center pixel of the examination object region AR.

[0130] Figure 8 This is a diagram illustrating an example of filter coefficients when template TP1 is used as a spatial filter. (See diagram for example.) Figure 8 As shown, template TP1 has a shape that extends in a straight line in the vertical direction of the figure. Figure 7In this case, as an example, the template TP1 constitutes a 5 x 5 pixel spatial filter. In this case, 1 / 5 is assigned as a filter coefficient to the pixels of the portion extending in a straight line, and -1 / 20 is assigned as a filter coefficient to the other pixels. When the total value after multiplying each of the filter coefficients and the same image evaluation value of the pixel corresponding to the examination target region AR is calculated as the matching degree, the matching degree is zero in the case where the arbitrary one pixel evaluation value in the examination target region AR is the same value. On the other hand, when the examination target region AR contains a blood vessel image extending in a stripe shape in the vertical direction, the matching degree increases. It can be said that the greater the value of this matching degree, the closer the contained image is to the template TP1. Therefore, the matching degree is calculated for each of the templates TP1 to TP4, and the highest matching degree in which the calculated matching degrees have the greatest value is assigned as the blood vessel region Y probability to the center pixel of the examination target region AR. That is, the blood vessel region Y probability value is assigned to the center pixel in the examination target region AR.

[0131] Since such a matching degree is a result obtained by spatially filtering the pixel evaluation value of the living tissue redness by using the templates TP1 to TP4 respectively, the value of each pixel of the image having the pixel evaluation value after the spatial filtering process contains the matching degree information when matching with any one of the templates TP1 to TP4, and the image obtained by the above spatial filtering is a pixel value reflecting the blood vessel region Y. Therefore, the feature amount calculation section 220b judges whether the value of the blood vessel probability in each pixel is greater than a predetermined value, and when the value of the blood vessel probability in the pixel is greater than the predetermined value, judges that this pixel is located in the blood vessel region Y, thereby extracting the blood vessel region Y.

[0132] The feature amount calculation section 220b determines the living tissue redness in the pixel corresponding to the extracted blood vessel region Y as the blood vessel redness. In this case, the blood vessel redness of the region not corresponding to the blood vessel region Y is zero. Further, the feature amount calculation section 220b can obtain a value obtained by normalizing the blood vessel probability to the range of 0 to 1, or can obtain a result after correcting the living tissue redness as the blood vessel redness, and make the value higher as the value is higher and lower as the value is lower. In this way, the blood vessel redness is calculated for all pixels. For example, a result obtained by multiplying the value of the living tissue redness by the blood vessel probability value can be obtained as the blood vessel redness.

[0133] Thus, the feature quantity calculation section 220b calculates the living tissue redness as the first pixel evaluation value and the blood vessel redness as the second pixel evaluation value according to the features of one embodiment. Thus, the first pixel evaluation value in the present embodiment is an evaluation value for indicating the degree of the color component displayed by the inflammation site of the image, and the second pixel evaluation value is an evaluation value for indicating the degree of the color component contained in the portion in the image that forms the shape of the blood vessel.

[0134] The typical value calculation section 220c calculates the typical value of the living tissue redness of the photographed living tissue (first typical evaluation value) by aggregating the living tissue redness (first pixel evaluation value) of each pixel calculated by the feature quantity calculation section 220b, and further calculates the typical value of the blood vessel redness (second typical evaluation value) by aggregating the blood vessel redness (second pixel evaluation value) of each pixel calculated by the feature quantity calculation section 220b.

[0135] The processing of calculating the typical values of the living tissue redness and the blood vessel redness in the image, that is, the typical evaluation values, can be either the averaging processing of calculating the average values of the living tissue redness and the blood vessel redness of each pixel, or the processing of finding other known processing such as the median. The averaging processing includes the processing of finding the simple average value, and the processing of finding the weighted average value. Further, as the known processing, there can be the processing of dividing each of the pixel evaluation values, that is, the living tissue redness and the blood vessel redness, into at least two or more ranks of order, and calculating the typical value by substituting the total value P of the values obtained by multiplying the number of pixels belonging to each rank by a predetermined weighting coefficient into a predetermined formula. In this case, the predetermined formula is, for example, 1 / (1+e -P ). In this case, it is preferable that the weighting coefficient is a coefficient obtained by multiple logistic regression analysis so as to have a correlation with the subjective evaluation result of the doctor.

[0136] In one embodiment, it is preferable that the typical evaluation value in the image is the total number of bins (data bins) in which the frequency is one or more in the histogram of the pixel evaluation values in each pixel. Figure 9 is a diagram illustrating an example of the typical evaluation value used in the endoscope system of one embodiment. In the example shown in Figure 9 , the number of bins for the range of the pixel evaluation values in which the frequency is one or more is, specifically, bin number 1 is 16 and bin number 2 is 2. Thus, in the example shown in Figure 9 , the typical evaluation value is the total bin number 18 (= 16 + 2). In the case where such a total bin number is used as the typical evaluation value, the data bins in the histogram are set in advance and fixed.

[0137] In the past, in a case where the average value or the maximum value of the pixel evaluation values is used as the representative evaluation value, even if the pixel evaluation values of pixels having high pixel evaluation values do not change, when the pixels having low pixel evaluation values (for example, pixels having low redness in the living tissue redness) increase, the average value is also caused to decrease by the amount of the pixels having low pixel evaluation values. Further, even if the pixels having high pixel evaluation values do not change much, when a noise component is mixed in the maximum pixel evaluation value and increases, the maximum value also increases. Thus, the representative evaluation value fluctuates due to the amount of pixels having pixel evaluation values that are not of interest or due to a noise component, which is not preferable for the severity that is high in calculation accuracy. In this regard, the integrated interval number is not affected by the amount of pixels having pixel evaluation values that are not of interest, and is also not affected by a noise component, and thus can be used as a representative evaluation value that is robust and stable. Therefore, the severity that is high in accuracy can be stably calculated.

[0138] In a case where such an integrated interval number is used as a representative evaluation value, the representative evaluation value is preferably a value that represents the degree of a color component exhibited by an inflammatory portion of a living tissue, such as a representative value of the living tissue redness. The redness of an inflammatory portion of a living tissue is high, and the redness of a healthy portion other than the inflammatory portion is low, and thus, in this regard that a stable representative evaluation value is obtained, it is preferable to use the integrated interval number as the representative evaluation value.

[0139] The integration section 220d calculates the severity of the lesion by integrating the representative value of the living tissue redness (the first representative evaluation value) and the representative value of the vascular redness (the second representative evaluation value). That is, the severity of the lesion is calculated on the basis of the representative value of the living tissue redness (the first representative evaluation value) and the representative value of the vascular redness (the second representative evaluation value). At this time, the integration section 220d adjusts the degree of influence of the representative value of the living tissue redness (the first representative evaluation value) that represents the severity that changes with a change in the representative value of the living tissue redness (the first representative evaluation value) in accordance with the representative value of the vascular redness. Thus, since the degree of influence of the representative evaluation value on the severity is adjusted, it is possible to evaluate the progression degree of the lesion of the living tissue lesion portion with high accuracy compared to the past.

[0140] Figure 10 is a graph showing an example of the degree of influence of the representative value of the living tissue redness. Figure 10The curve (circle) shown indicates an evaluation value (value of severity) for indicating the degree of progression of a lesion in the reference image with respect to a typical value of redness of living tissue in the reference image. At this time, as the typical value of redness of blood vessels, three values of high (white circle of dotted line), medium (gray circle), and low (black circle) are exemplified. The three straight lines indicated by the black dotted line, the gray solid line, and the black solid line indicate regression straight lines calculated in a manner having the highest correlation with the evaluation value when the typical value of redness of blood vessels is high, medium, and low. As is apparent from the graph, the degree of influence of redness of living tissue on the value of severity (gradient of the regression straight line) changes with the typical value of redness of blood vessels. In Figure 10 In the example shown, the smaller the typical value of redness of blood vessels, the greater the degree of influence of the typical value of redness of living tissue on severity. Therefore, when calculating the severity of a lesion, the degree of influence of the typical value of redness of living tissue (first typical evaluation value) is adjusted in accordance with the typical value of redness of blood vessels (second typical evaluation value).

[0141] The adjustment of the degree of influence described above is at least performed in accordance with a second typical evaluation value different from the first typical evaluation value, but is not limited to the case where it is performed in accordance with the second typical evaluation value. For example, it can also be performed in accordance with information of another color component (second color component) of an image different from the red component of the inflamed portion of living tissue used when calculating the first typical evaluation value. The other color component (second color component) may, for example, be a feature quantity related to a color component exhibited by food or the like residue (remnant) introduced from the outside into the body cavity and present on living tissue. Food or the like residue (remnant) exhibits a strong brown color in the blue component compared to the color of blood or the like, and therefore, for example, the ratio of the B signal to the G signal in the R, G, B signals of an image can be used as a feature quantity related to the color component exhibited by the food or the like residue (remnant).

[0142] According to one embodiment, preferably, in the case where the severity is calculated in accordance with a polynomial having at least the first typical evaluation value as an explanatory variable, the degree of influence described above is a coefficient related to the first typical evaluation value of the polynomial. When the severity is calculated using a polynomial, the coefficient related to the first typical evaluation value is adjusted, and therefore, it is possible to easily change the degree of influence of the first typical evaluation value.

[0143] According to one embodiment, the above-described polynomial can also be a prediction formula (e.g., a regression equation) obtained by preprocessing. Specifically, using a reference image in which an evaluation value determined as an evaluation result of the extent of progression of the lesion is indicated, a first pixel evaluation value (e.g., a typical value of redness of living tissue) and a value of information indicating a second color component (e.g., a feature quantity related to a color component exhibited by a residue) or a second pixel evaluation value (e.g., a typical value of redness of blood vessels) are used as explanatory variables, an evaluation value in the reference image is used as a value of the severity, and this severity is used as a target variable, a regression equation can be obtained in such a manner that the severity calculated in correspondence with the evaluation value is most relevant, by performing a multiple regression analysis in advance with respect to the severity. Thus, a regression equation for optimizing the calculation of the severity can be obtained.

[0144] In Figure 1 In the example shown, specifically, using a plurality of reference images in which evaluation results determined as the extent of progression of the lesion are indicated, at least a typical value of redness of living tissue (a first typical evaluation value), a typical value of redness of blood vessels (a second typical evaluation value), and a typical value of whiteness of living tissue (a third typical evaluation value) are used as explanatory variables, a severity in the reference image is used as a target variable, and a regression equation is obtained by performing a multiple regression analysis in advance. This regression equation is used as an optimized prediction formula for calculating the severity using an image serving as an evaluation object that is captured by an electronic endoscope.

[0145] According to one embodiment, when a first typical evaluation value is set as X and a second typical evaluation value is set as Y, the integration unit 220d preferably performs the calculation in accordance with Severity = a1 · X + b1 · Y + g1 · X · Y + d1 by using coefficients a1, b1, g1, and a constant d1. The first typical evaluation value and the second typical evaluation value are not limited to any one of a typical value of redness of living tissue, a typical value of redness of blood vessels, and a typical value of whiteness of living tissue, and can also be typical evaluation values calculated in accordance with characteristics of other lesion sites.

[0146] Further, according to another embodiment, in a case where there is a second color component that does not affect the severity but affects the degree of influence of the first typical evaluation value, when the first typical evaluation value is set to X and a value (typical value) of information indicating the second color component is set to Z, the integrating section 220d preferably performs the calculation in accordance with Severity = a2■X + β2■X■Z + d2 by using coefficients a2, β2, and a constant d2. The first typical evaluation value is not limited to any one of the typical value of the redness of the living tissue, the typical value of the redness of the blood vessels, and the typical value of the whiteness of the living tissue, and can be a typical evaluation value calculated in accordance with the characteristics of another lesion site. The information of the second color component is preferably a characteristic amount related to a color component exhibited by a residue introduced from the outside into the body cavity and present on the living tissue, but is not limited to a characteristic amount related to a color component exhibited by food or the like residue present on the living tissue.

[0147] In a case where the calculation is performed in accordance with Severity = a1■X + β1■Y + γ1■X■Y + d1, since it is expressed as Severity = (a1 + γ1■Y)■X + β1■Y + d1, the degree of influence of the first typical evaluation value is adjusted in accordance with a1 + γ1■Y, that is, the second typical evaluation value. Likewise, in a case where the calculation is performed in accordance with Severity = a2■X + β2■X■Z + d2, since it is expressed as Severity = (a2 + β2■Z)■X + d2, the degree of influence of the first typical evaluation value is adjusted in accordance with a2 + β2■Z, that is, the information of the second color component.

[0148] Using such a prediction formula, it is possible to easily change the degree of influence.

[0149] According to one embodiment, in addition to the second typical evaluation value or the value (typical value) of information indicating the above-described second color component, the integrating section 220d preferably adjusts the degree of influence of the first typical evaluation value by at least using a third typical evaluation value obtained by aggregating a third pixel evaluation value indicating the degree of another third characteristic different from the first characteristic and the second characteristic. When the first typical evaluation value is set to X, the second typical evaluation value is set to Y, and the third typical evaluation value is set to Z, the calculation is performed in accordance with, for example, using coefficients a3, β3, γ3, σ3, θ3, ω3, and a constant d3. In this case, the degree of influence of the first typical evaluation value X is

[0150] ​Further, the integrating section 220d is able to adjust the degree of influence of the first typical evaluation value in accordance with a value (typical value) of information indicating the above-mentioned second color component, in addition to the second typical evaluation value. When the first typical evaluation value (for example, a typical value of the living tissue redness) is set to X, the second typical evaluation value (a typical value of the blood vessel redness) is set to Y, and a value of information indicating the second color component (for example, a color component exhibited by the residue) is set to Z, the calculation is performed by using coefficients a4, b4, s4, t4, and a constant d4, for example, in accordance with In this case, the degree of influence of the first typical evaluation value X is changed while the second typical evaluation value Y is able to individually influence the severity, and the value Z is not able to individually influence the severity, due to the influence of the second typical evaluation value Y and the value Z. Further, the degree of influence of the first typical evaluation value X is That is, by using such a prediction formula, it is possible to easily change the degree of influence.

[0151] According to one embodiment, the appearance feature exhibited by the lesion site used for calculating the severity is any one of the degree of a color component (red) exhibited by the living tissue inflammation site, the degree of a color component (red) exhibited by a blood vessel region indicating a blood vessel extending in a striped shape on the living tissue, and the degree of a color component (white) exhibited by the living tissue ulcer site, and is preferable in terms of calculating the severity with high accuracy.

[0152] The integrating section 220d uses the calculated severity together with the color map image created by the feature quantity calculating section 220b to generate a signal for picture display and transmits the signal to the monitor 300.

[0153] Figure 11 is a diagram showing an example of a flow of a method of calculating the severity performed by the endoscope system according to one embodiment.

[0154] First, the image processing unit 220 acquires an image of a current frame (step S10).

[0155] Next, the preprocessing section 220a performs preprocessing including the above-mentioned RGB conversion, color space conversion, setting of a reference axis, color correction, and tone enhancement processing as necessary, and further, the feature quantity calculating section 220b calculates a plurality of pixel evaluation values (first pixel evaluation value, second pixel evaluation value, third pixel evaluation value) indicating the degree of each of a plurality of features related to a color component or a shape exhibited by the lesion site, for example, living tissue redness and blood vessel redness, living tissue whiteness, and the like, for each pixel with respect to the image on which the preprocessing is performed (step S12).

[0156] The feature quantity calculation section 220b determines whether or not to calculate the pixel evaluation value for all pixels of the image of the current frame (step S14). When the calculation of the pixel evaluation value for all pixels is completed, the typical value calculation section 220c calculates typical values, i.e., the first to third typical evaluation values, which integrate the pixel evaluation values (step S16). The typical values are calculated for each type of the pixel evaluation value.

[0157] After that, the integration section 220d combines a plurality of typical values (typical evaluation values) to calculate one severity (step S18). That is, a value as the severity of the lesion, which is one value obtained by operating and integrating a plurality of typical values (typical evaluation values), is calculated. In the calculation of the severity, for example, as described above, a regression equation using a plurality of typical values (typical evaluation values) obtained in the preprocessing as explanatory variables is used. This regression equation is an equation with respect to the severity that most corresponds to the subjective evaluation result.

[0158] After that, the integration section 220d generates a signal for on-screen display of the color-mapped image generated from the pixel evaluation values calculated in step S12 and the severity calculated in step S18, and transmits it to the monitor 300. Thus, the monitor 300 displays the color-mapped image and the information of the severity (step S20).

[0159] In this way, the image processing unit 220 repeatedly performs the above-described processing during the period in which the captured images are sequentially transmitted from the electronic endoscope 100 (step S22).

[0160] Thus, the calculated severity is displayed on the monitor 300, so the user can know the progression degree of the lesion by watching the monitor 300.

[0161] Hereinafter, another embodiment of the present application will be described.

[0162] As described above, in the conventional endoscope system, a plurality of typical evaluation values are calculated by integrating the calculated pixel evaluation values of the respective pixels for each of a plurality of appearance features, and a value obtained by operating and integrating at least two of the calculated plurality of typical evaluation values is calculated as a value of the severity of the lesion. The above-described operation is performed in accordance with a formula that expresses the relationship between the severity and the plurality of typical evaluation values in advance.

[0163] However, in the case where the same formula is used even if the imaging position is different, in the entrance side portion and the deep side portion in the longitudinal direction of the tubular elongated organ, even if the progression of the lesion is the same degree, the calculated typical evaluation values are different from each other, and sometimes the value of the severity changes.

[0164] Therefore, in the processor for an endoscope of one embodiment, when a value of the severity of the lesion is calculated from a plurality of typical evaluation values, the calculation processing of the severity adjusted from the imaging position information corresponding to the evaluation target image is set and is calculated. That is, the value of the severity is adjusted from the information of the imaging position. Therefore, the progression degree of the lesion can be evaluated stably and with high accuracy regardless of the imaging position of the evaluation target image.

[0165] Figure 12 is a view that illustrates an example of processing performed by the processor for an endoscope and the endoscope system of one embodiment. In Figure 12 In the example illustrated in FIG. 8, as the plurality of typical evaluation values, at least a typical value of redness of living tissue indicating the degree of inflammation of the living tissue, a typical value of vascular redness indicating the degree of redness in the blood vessel region, and a typical value of whiteness of living tissue indicating the degree of ulceration are included. The calculation method of these typical evaluation values will be described later.

[0166] First, as a preprocessing, a plurality of reference images of which the evaluation results of the progression degree (severity) of the lesion are determined by electronic endoscopy are prepared, and the plurality of typical evaluation values described above are calculated in advance in each of the reference images. The severity-related evaluation can be a subjective evaluation result of a doctor (e.g., a MAYO endoscopic subscore) or a histological evaluation result. The calculation processing of the severity is optimized using the plurality of typical evaluation values so that the severity calculated by combining the plurality of typical evaluation values including the three typical evaluation values is most relevant to the evaluation result of the progression degree (severity) of the lesion determined in each of the reference images. At this time, the optimization of the calculation processing of the severity is performed for each imaging position using the imaging position information of the reference images.

[0167] The optimization of the severity calculation includes, for example, setting a coefficient value related to each of the typical evaluation values in the relationship formula according to the imaging position, calculating a value of the severity using a typical evaluation value (normalized evaluation value) normalized by two reference values related to the typical evaluation value corresponding to the imaging position as information of the typical evaluation value input to the relationship formula, or normalizing a calculation result of the severity calculated using the relationship formula by two reference values related to the severity set according to the imaging position and using the normalized value as a value of the severity. Before the severity of the evaluation target image is calculated in the processor for an endoscope, the overall processing content of the optimization of the severity performed by using the relationship formula obtained in this manner is set.

[0168] Next, a plurality of typical evaluation values including a typical value of the redness of the living tissue, a typical value of the redness of the blood vessels, and a typical value of the whiteness of the living tissue obtained from the images by taking the living tissue by the electronic endoscope are calculated, and the severity is calculated by inputting the calculated plurality of typical evaluation values into the relational expression. In this case, according to the contents of the calculation processing of the severity optimized, the coefficient values related to each of the typical evaluation values in the relational expression are set according to the imaging position, and the value of the severity is calculated using the calculation formula. Alternatively, as the information of the typical evaluation values input to the relational expression which does not change with the change of the imaging position, the value of the severity is calculated using the typical evaluation values normalized by two reference values related to the typical evaluation values corresponding to the imaging position (normalized evaluation values). Alternatively, after the severity is calculated by using the relational expression which does not change with the change of the imaging position, the value of the severity is the value after the value of this calculation result is normalized by two reference values set according to the imaging position.

[0169] Thus, when the value of the severity of the lesion is calculated according to the plurality of typical evaluation values in the evaluation target image, the calculation processing of the severity adjusted according to the imaging position information of the evaluation target image is set and calculated, and thus the progression degree of the lesion can be evaluated stably and with high accuracy regardless of the imaging position of the image.

[0170] Figure 13 is a diagram illustrating the configuration of the image processing unit 220 which performs a quantification processing for calculating the severity of the lesion of the living tissue by expressing the degree of the lesion characteristics by a numerical value.

[0171] The image processing unit 220 is a portion for calculating the severity of the lesion by quantifying the progression degree of the lesion in the living tissue according to the living tissue image obtained by the electronic endoscope 100. The image processing unit 220 includes a preprocessing section 220a, a feature quantity calculation section 220b, a typical value calculation section 220c, a consolidation section 220d, and an imaging position information processing section 220e.

[0172] Since the preprocessing section 220a, the feature quantity calculation section 220b, and the typical value calculation section 220c have been described with reference to Figure 3 The preprocessing section 220a, the feature quantity calculation section 220b, and the typical value calculation section 220c have been described, and thus repeated description thereof is omitted.

[0173] The consolidation section 220d calculates the value of the severity of the lesion by consolidating the typical value of the redness of the living tissue and the typical value of the redness of the blood vessels. At this time, the consolidation section 220d sets the calculation processing of the severity adjusted according to the imaging position information of the taken image.

[0174] In the imaging position information processing section 220e, the imaging position information corresponds to the captured image. For example, the imaging position information is transmitted from a position measuring system provided in the endoscope system 1 to the imaging position information processing section 220e.

[0175] The position measuring system exemplifies a system that acquires the position of the solid-state imaging element 108 at the distal end of the electronic endoscope inserted into an organ and the respective positions of the flexible tube connected thereto, for example, using a sensor, a system that acquires the insertion length of the electronic scope 100 inserted from the opening end of an organ, or a system that displays the captured image on the monitor 300 and the surgeon who views the image inputs an instruction manually to acquire a signal indicating that the distal end portion of the electronic scope 100 has passed through a specific portion as a characteristic portion within the inserted organ.

[0176] In the system that acquires the position of the solid-state imaging element 108 using a sensor, for example, a plurality of magnetic sensors are provided at positions near the solid-state imaging element 108 at the distal end portion of the electronic endoscope 100 and at predetermined intervals from the distal end portion to the flexible tube connected to the processor 200 side, a magnetic field having different intensities is applied according to the position from the outside of the human body in which the electronic endoscope 100 is inserted into an organ, and the magnetic sensors described above can acquire the position of the magnetic sensor provided at the distal end portion by measuring the intensity of the magnetic field, and further, the bending shape of the flexible tube within the organ can be acquired from the positions of the plurality of magnetic sensors. Thereby, the position of the distal end portion of the solid-state imaging element 108 can be acquired, and the shape of the electronic endoscope 100 within the organ and the insertion length of the electronic endoscope 100 inserted from the opening end of the organ can be acquired.

[0177] When it is a system for acquiring the insertion length of the electronic endoscope 100 inserted from the opening end of the organ, for example, the relevant movement distance information such as how much the living tissue has moved is acquired using an optical flow process between adjacent frame images in the captured moving image, and the movement distance information described above is calculated by accumulating every time the frame image changes, so that the insertion length information of the current electronic endoscope 100 can be acquired. Further, for example, the insertion length information of the current electronic endoscope 100 can be acquired by measuring the length of the connected flexible tube that extends from the distal end portion of the inserted electronic endoscope 100 toward the inside of the organ.

[0178] In the system for acquiring a specific portion of an organ passing a signal, the surgeon generates a specific portion passing signal by pressing a button located at hand while observing an image displayed on the monitor 300 at a time point when an identifiable specific portion inside the organ appears in the image and passes, and the imaging position information processing section 220e can acquire this specific portion passing signal. The specific portion position inside the organ, for example, when the organ is a large intestine, includes a starting position of the ascending colon, a starting position of the transverse colon after the ascending colon ends and the large intestine bends, a starting position of the descending colon after the transverse colon ends and the large intestine bends, a starting position of the sigmoid colon after the descending colon ends and the large intestine bends, a starting position of the rectum after the sigmoid colon ends, and a position of the rectum ending at the anus.

[0179] The integration section 220d sets the calculation processing content of the severity degree in accordance with the imaging position information corresponding to the image obtained in this way. According to one embodiment, in a case where the integration section 220d calculates the value of the severity degree in accordance with a relational expression establishing a correlation between the severity degree and the typical evaluation value of each of a plurality of features, in the relational expression, a value corresponding to the imaging position information is set as a coefficient related to the typical evaluation value. By performing the preprocessing shown in FIG. 8 on such a coefficient, the coefficient value of the relational expression is calculated for each imaging position and saved in the memory 204, so that the correlation between the numerical value of the evaluation result of the progression degree of the lesion in the reference image and the calculation result of the plurality of severity degrees is highest. Thereby, it is possible to suppress the degree of variation of the value of the severity degree with the change of the imaging position information. Figure 12

[0180] Accordingly, the integration section 220d reads out the coefficient value corresponding to the imaging position information from the memory 204 and assigns the coefficient value to the relational expression.

[0181] According to one embodiment, the memory 204 holds, as a value determined in accordance with the imaging position information of the reference image, two reference values related to the typical evaluation value for each imaging position, where the typical evaluation value is a value set in accordance with a plurality of reference images obtained by capturing a plurality of lesion sites having different progression degrees of lesions by the electronic endoscope (refer to FIG. 6). Figure 12 In this case, the integration section 220d calculates the severity degree in the evaluation target image captured by the electronic endoscope 100 by using a relational expression previously establishing a correlation between a normalized evaluation value and a severity degree, where the normalized evaluation value is a value obtained by the integration section 220d normalizing the typical evaluation value of the reference image by the two reference values corresponding to the imaging position information of the reference image captured by the electronic endoscope 100.

[0182] ​As two reference values, for example, a typical evaluation value exhibited by a healthy portion of a living tissue and a typical evaluation value exhibited by a lesion portion at a time of the most severe lesion of the lesion portion are exemplified. The two reference values are preferably an upper limit value and a lower limit value of the typical evaluation value of the reference image that can be taken at the imaging position. The reference values are acquired in association with the relationship between the evaluation value and the severity by optimizing the calculation processing of the severity of the preprocessing. Therefore, in the preprocessing, a relationship in which the normalized evaluation value, which is a value obtained by normalizing the typical evaluation value by the two reference values, is associated with the severity is extracted.

[0183] When the calculation of the severity is performed on the basis of the evaluation target image photographed by the electronic endoscope 100, the integration section 220d reads out the two reference values corresponding to the imaging position information of the evaluation target image from the storage 204 and calculates the value of the severity by inputting the normalized evaluation value, which is a value obtained by normalizing the typical evaluation value by the two reference values, into the above-described relationship in which the normalized evaluation value is associated with the severity. Thus, it is possible to suppress the case where the value of the severity varies with the imaging position. In addition, when the degree of variation of the value of the severity with the imaging position is still large, it is possible to configure such that, in addition to the use of the normalized evaluation value in the relationship, a coefficient value used in the relationship in which the normalized evaluation value is associated with the severity is set in accordance with the imaging position.

[0184] According to one embodiment, the storage 204 holds, as a value determined in accordance with the imaging position of the reference image, two reference values related to the severity that are set in accordance with a plurality of reference images (see Figure 12 ) obtained by photographing a plurality of lesion portions having different degrees of lesion progression by the electronic endoscope 100 for each imaging position of the electronic endoscope 100. The integration section 220d calculates the severity by using a relationship in which the typical evaluation value of the evaluation target image photographed by the electronic endoscope 100 is associated with the severity and calculates the value of the severity as a normalized value obtained by normalizing the value of the calculation result by the two reference values determined in correspondence with the imaging position information of the evaluation target image.

[0185] Here, as the two reference values, for example, a value of the severity shown by the healthy part of the living tissue before normalization and a value of the severity shown by the lesion part at the time of the most severe lesion of the lesion part before normalization are exemplified. The two reference values are preferably upper and lower limit values of the value (value before normalization) of the severity of the reference image that can be taken at the imaging position. The reference values are acquired in association with the relationship formula by optimizing the calculation process of the severity in the preprocessing. Therefore, in the preprocessing, the relationship formula that associates the two reference values, the typical evaluation value, and the severity is extracted.

[0186] Therefore, the integration unit 220d inputs the typical evaluation value of the evaluation target image into the relationship formula that associates the severity and the typical evaluation value of each of the plurality of features, and calculates the value of the severity by normalizing the value of the calculation result by the two reference values read out from the memory 204 according to the imaging position. Thereby, it is possible to suppress the degree of change in the value of the severity with the change in the imaging position. In addition, when the degree of change in the value of the severity with the change in the imaging position is still large, in addition to using the normalized value as the value of the severity, it is also possible to set the coefficient used in the relationship formula to a value corresponding to the imaging position information, and furthermore, it is also possible to be configured to use a relationship formula that associates the normalized evaluation value obtained by normalizing the typical evaluation value and the severity.

[0187] According to one embodiment, the above-described relationship formula is preferably a regression equation of a polynomial extracted by performing a multiple regression analysis, so that the value of the severity calculated from the typical evaluation value of each of the plurality of features corresponds to the evaluation result of the progression degree of the lesion of the reference image for which the evaluation result related to the severity is prepared and determined in advance. Such a regression equation formula is calculated in the process of optimizing the calculation process of the severity in the preprocessing. By using the regression equation formula, the calculation result of the severity can correspond to the evaluation result of the severity with a high degree of correlation.

[0188] Further, according to one embodiment, the imaging position information is preferably position information of one region when a region in the longitudinal direction of the tubular organ is divided into a plurality of regions. In the tubular organ, there is a portion in which the living tissue surface form differs between the entrance portion and the longitudinal side portion of the organ, and thus, even in the same healthy portion, the value of the severity degree differs depending on the imaging position information, and the degree of value variation increases. Therefore, in the case where the tubular organ is the examination target, it is particularly preferable to set the calculation processing content of the severity degree in accordance with the imaging position information. When the organ is, for example, the large intestine including the rectum, sigmoid colon, descending colon, transverse colon, and ascending colon, it can be divided into two intervals of the rectum to sigmoid colon interval of the large intestine and the interval from the inside of the sigmoid colon to the ascending colon, or it can be divided into five intervals of the rectum, sigmoid colon, descending colon, transverse colon, and ascending colon.

[0189] Figure 14 FIG. 6 is a diagram showing an example of a flow of a method of calculating the severity degree performed by the endoscope system according to one embodiment.

[0190] First, imaging is performed in order to obtain an evaluation target image by the electronic endoscope 100, and at this time, the imaging position information corresponding to the evaluation target image is acquired by the position information acquisition section in the endoscope processor (step S110). On the other hand, after the captured image is processed by the driver signal processing circuit 112, the image processing unit 220 acquires the image of the current frame by the system controller 202 (step S112).

[0191] Next, the preprocessing section 220a performs preprocessing including the above-described RGB conversion, color space conversion, setting of the reference axis, color correction, and tone enhancement processing as necessary, and further, the feature quantity calculation section 220b calculates, for each pixel, a plurality of pixel evaluation values for representing the degree of each of a plurality of features related to the color component or shape displayed by the lesion portion, such as the living tissue redness and blood vessel redness, living tissue whiteness, and the like, with respect to the image on which the preprocessing has been performed (step S114).

[0192] The feature quantity calculation section 220b repeats the processing until the pixel evaluation values are calculated with respect to all the pixels of the image of the current frame. Thereafter, the typical value calculation section 220c calculates a typical value, that is, a typical evaluation value, which integrates the pixel evaluation values (step S116). The typical value is calculated for each type of the pixel evaluation values. In the above-described example, the typical evaluation value is a typical value of the living tissue redness, blood vessel redness, or living tissue whiteness, or the like.

[0193] The integration section 220d sets the content of the calculation processing of the severity degree corresponding to the imaging position information acquired in step S110, and calculates the severity degree (step S118).Figure 15A to Figure 15C is a diagram showing an example of a flow of a calculation process of the severity.

[0194] Figure 15A An example of a flow of the calculation process 1 is shown. In the calculation process 1, the integration section 220d sets the coefficient value related to each typical evaluation value in the relational expression corresponding to the imaging position information (step S30). This coefficient value is a value calculated by the preprocessing shown in FIG. 8 and stored and held in the memory 204 so as to be most corresponding to the evaluation result of the severity. Therefore, when the imaging position information is acquired, the coefficient value corresponding to the imaging position information is read out from the memory 204. Figure 12

[0195] Further, the integration section 220d provides the set coefficient value to the coefficient in the regression equation and calculates the value of the severity by inputting the typical evaluation value to this regression equation (step S32). Since the coefficient value is a value corresponding to the imaging position information, it is possible to suppress the case where the value of the severity changes with the change of the imaging position. Therefore, it is possible to evaluate the progression degree of the lesion using the value of the severity stably and with high accuracy regardless of the imaging position of the image.

[0196] Figure 15B An example of a flow of the calculation process 2 is shown. In the calculation process 2, the integration section 220d determines two reference values (1st reference value, 2nd reference value) from the imaging position information of the evaluation target image and calculates the normalized evaluation value obtained by normalizing the typical evaluation value by using these two reference values (step S40). For example, in the case where the 1st reference value is XI, the 2nd reference value is X2 (> XI), and the typical evaluation value is P, it is (P - XI) / (X2 - XI). P does not have to be a value between XI and X2, but when P is a value between XI and X2, (P - XI) / (X2 - XI) is a value between 0 and 1.

[0197] ​Next, the integration section 220d inputs the normalized evaluation value into a regression equation that expresses the normalized evaluation value as an explanatory variable, and calculates a value of the severity (step S42). The regression equation is a relational expression that is optimized for the severity calculation by preprocessing and using the normalized evaluation values of the reference images, and by this optimization, two reference values (1st reference value, 2nd reference value) are calculated in accordance with each imaging position information, and stored in the memory 204. The two reference values are values corresponding to the imaging position information, and the normalized evaluation value is a value obtained after normalization in such a manner that the value does not change with changes in the imaging position, and thus it is possible to suppress a case where the value of the severity changes with changes in the imaging position. Therefore, regardless of the imaging position of the evaluation target image, it is possible to evaluate the progression degree of the lesion using the value of the severity stably and with high accuracy.

[0198] Figure 15C An example of the flow of the calculation processing 3 is shown. In the calculation processing 3, the integration section 220d sets two reference values (3rd reference value, 4th reference value) related to the severity in accordance with the imaging position information of the evaluation target image (step S50).

[0199] Next, the integration section 220d inputs the typical evaluation value calculated in step S116 into the regression equation, and calculates the severity (step S52).

[0200] The integration section 220d normalizes the calculation result of the severity by the two reference values, and calculates a normalized value of the severity as the value of the severity (step S54). For example, when the 3rd reference value is X3, the 4th reference value is X4, and the calculation result of the severity is Q, it is (Q-X3) / (X4-X3). Q does not have to be a value between X3 and X4, but when Q is a value between X3 and X4, (Q-X3) / (X4-X3) is a value between 0 and 1. The two reference values are values corresponding to the imaging position information, and the normalized value of the severity is a value obtained after normalization in such a manner that the value does not change with changes in the imaging position, and thus it is possible to suppress a case where the value of the severity changes with changes in the imaging position. Therefore, regardless of the imaging position of the image, it is possible to evaluate the progression degree of the lesion using the value of the severity stably and with high accuracy.

[0201] Thus, after the value of the severity is obtained, the process returns to step S120 of Figure 14 The image processing unit 220 performs control so that the above-described color map image made in the feature amount calculation section 220b from the pixel evaluation value and the value of the severity obtained in step S118 are displayed on the display screen of the monitor 300.

[0202] Thus, the image processing unit 220 determines whether or not to continue the photographing for obtaining the evaluation target image, and when the photographing is to be continued, returns to step S110 to repeat the above processing. When the photographing is not to be continued, the above processing ends.

[0203] Thus, since the value of the calculated severity is displayed on the monitor 300, the user can know the progression degree of the lesion by watching the monitor 300.

[0204] In the foregoing, the endoscope system according to the present application has been described in detail, but the endoscope system according to the present application is not limited to the above-described embodiments, and various improvements and changes can be made within the scope of the gist of the present application.

[0205] The present application relates to patent applications of Japanese Patent Application No. 2020-160703 filed with the Japan Patent Office on September 25, 2020, and Japanese Patent Application No. 2020-161900 filed with the Japan Patent Office on September 28, 2020, and the entire contents of these applications are incorporated by reference into the present specification.

Claims

1. A processor for an endoscope, characterized by: An image processing unit configured to calculate a severity of a lesion from an image of a lesion site of a living tissue obtained by an electronic endoscope configured to capture the living tissue in a body cavity, and using at least information of the image, The image processing unit includes: a feature amount calculation unit configured to calculate, for each pixel, a first pixel evaluation value related to a first color component or a shape of the lesion site and indicating a degree of a first feature, from the image, wherein the first pixel evaluation value is able to distinguish a first feature of an appearance of the lesion site from a feature of a healthy site of the living tissue according to the first color component or the shape of the lesion site, and is a pixel evaluation value corresponding to the first feature; a typical value calculation unit configured to calculate a first typical evaluation value of the first feature of the living tissue by aggregating the first pixel evaluation values of the pixels in the image; an integration unit configured to adjust an influence degree of the first typical evaluation value indicating that the severity changes with a change in the first typical evaluation value, at least according to second color component information of the image different from the first color component, or at least according to a second typical evaluation value obtained by aggregating second pixel evaluation values indicating a degree of a second feature different from the first feature, or at least according to the first typical evaluation value, to calculate the severity of the lesion, the severity is calculated from a polynomial having at least the first typical evaluation value as an explanatory variable, and the influence degree is a coefficient related to the first typical evaluation value in the polynomial, the polynomial is a regression equation related to the severity, obtained by performing a multiple regression analysis in advance using a reference image in which an evaluation value indicating an evaluation result related to a degree of progression of the lesion is determined, and using the first pixel evaluation value and a value of information indicating the second color component or the second pixel evaluation value as explanatory variables, the evaluation value in the reference image as a value of the severity, and a value of the severity as a target variable.

2. The processor for an endoscope according to claim 1, wherein When the first typical evaluation value is set as X and the second typical evaluation value is set as Y, the integration section calculates the severity by using coefficients a1, b1, g1 and a constant d1, according to the severity = a1X + b1Y + g1X + d1. 1· X + b1 1· Y + g1 1· X · Y + d1.

3. The endoscope processor according to claim 1, wherein When the first typical evaluation value is set as X and a value for information representing the second color component is set as Z, the integration section calculates the severity = a 2· X + β 2· X · Z + d2 by using the coefficients a2, b2 and the constant d2.

4. The processor for an endoscope according to claim 1, wherein The influence degree of the first typical evaluation value can be adjusted at least by using a third typical evaluation value obtained by aggregating third pixel evaluation values indicating a degree of a third feature different from the first and second features, in addition to the second typical evaluation value or the value of information indicating the second color component.

5. The endoscope processor according to claim 1, wherein, Each of the first and second features is any one of a degree of a color component exhibited by an inflammation site of the living tissue, a degree of a color component exhibited by a blood vessel region indicating a blood vessel extending in a striped pattern on the living tissue, and a degree of a color component exhibited by an ulcer site of the living tissue.

6. The processor for an endoscope according to claim 1, wherein The first typical evaluation value is a value indicating a degree of a color component exhibited by an inflammation portion of the living tissue.

7. The endoscope processor according to claim 6, wherein, The first typical evaluation value is a value indicating a degree of a color component exhibited by an inflammation portion of the living tissue.

8. The processor for an endoscope according to any one of claims 1 to 7, wherein, The second color component information is a feature quantity related to a color component exhibited by a residue introduced from the outside into the body cavity and present on the living tissue.

9. An endoscope system comprising: an electronic endoscope configured to capture an image of a living tissue in a body cavity; and an endoscope processor according to any one of claims 1 to 8 connected to the electronic endoscope.

10. A processor for an endoscope, characterized by: The image processing unit is configured to calculate, based on an image of a lesion portion of a living tissue obtained by an electronic endoscope and using at least information of the image, a value indicating a degree of progression of the lesion of the living tissue, The image processing unit includes: a feature quantity calculation unit configured to calculate, based on the image, a pixel evaluation value that can distinguish, based on a color component exhibited by the lesion portion or a shape of the lesion portion, a plurality of appearance features exhibited by the lesion portion from features of a healthy portion of the living tissue, and that is an evaluation value corresponding to each of the appearance features, and that is related to the color component or the shape of the lesion portion and is used to indicate a degree of each of the plurality of appearance features on a per-pixel basis; a typical value calculation unit configured to calculate, for one of the images, a typical evaluation value of each of the plurality of features of the living tissue by aggregating the pixel evaluation values of the pixels in the image; an image capturing position information processing unit that associates image capturing position information in the body cavity at the time of capturing the image with the image; an integration unit configured to calculate, for an evaluation target image captured by the electronic endoscope, a value indicating a degree of severity of the lesion based on the typical evaluation value of each of the plurality of features, and to set, when calculating the value indicating the degree of severity, a calculation process of the degree of severity adjusted based on the information of the image capturing position corresponding to the evaluation target image, the integration unit calculates the value indicating the degree of severity based on a relational expression that establishes a correlation between the degree of severity and the typical evaluation value of each of the plurality of features, and in the relational expression, sets a value corresponding to the information of the image capturing position as a coefficient related to the typical evaluation value, the relational expression is a regression equation of a polynomial extracted by performing multiple regression analysis, such that the value indicating the degree of severity calculated based on the typical evaluation value of each of the plurality of features corresponds to an evaluation result of a reference image for which an evaluation result indicating the degree of progression of the lesion is prepared in advance.

11. The processor for an endoscope according to claim 10, wherein, The storage section holds, for each imaging position of the electronic endoscope, two reference values related to the typical evaluation value as values determined from the information of the imaging position of the reference image, wherein the typical evaluation value is a value set from a plurality of reference images obtained by imaging a plurality of lesion sites having different lesion progression degrees with the electronic endoscope, The integrating section calculates the value of the severity by using a relational expression in which a correlation between a normalized evaluation value and the severity is established in advance, wherein the normalized evaluation value is a value obtained by normalizing the typical evaluation value of the reference image by the two reference values corresponding to the information of the imaging position of the reference image imaged by the electronic endoscope; and the value of the severity is calculated by inputting the normalized evaluation value into the relational expression, wherein the normalized evaluation value is a value obtained by normalizing the typical evaluation value of the evaluation target image by the two reference values determined from the information of the imaging position of the evaluation target image.

12. The processor for endoscopy according to claim 10, wherein, The storage section holds, for each imaging position of the electronic endoscope, two reference values related to the severity as values determined from the information of the imaging position of the reference image, wherein the severity is a value set from a plurality of reference images obtained by imaging a plurality of lesion sites having different lesion progression degrees with the electronic endoscope, The integrating section calculates the severity by using a relational expression in which a correlation between the typical evaluation value of the evaluation target image and the severity is established in advance, and calculates a normalized value obtained by normalizing the value of the calculation result by the two reference values determined corresponding to the information of the imaging position of the evaluation target image as the value of the severity.

13. The processor for an endoscope according to any one of claims 10 to 12, wherein, The information of the imaging position is position information of one region when a region of a tubular organ in a longitudinal direction is divided into a plurality of regions.

14. An endoscope system comprising: an electronic endoscope configured to image a living body in a body cavity; and an endoscope processor according to any one of claims 10 to 13 connected to the electronic endoscope.

Images