Computer program, information processing method and endoscope

Abstract

To provide a computer program and the like which automatically adjust a light volume to be applied to a region of interest included in an image captured with an endoscope and suspected of being a lesion identified by a learning model.SOLUTION: The computer program causes a computer to execute processes of: acquiring an image captured with an endoscope; inputting the image into a learning model having learned to output a recognition result of a region of interest included in the image, when the image captured with the endoscope is inputted; outputting the recognition result; and adjusting a light volume which an LED provided at the distal end of the endoscope applies to the region of interest, based on the recognition result.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present technology relates to a computer program, an information processing method, and an endoscope. [Background technology]

[0002] Conventionally, devices have been proposed that perform automatic light adjustment during lesion investigation using an endoscope, etc. For example, the device described in Patent Document 1 changes the luminance of a predetermined area in a medical captured image of an observation target by automatic light adjustment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-42412 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the device described in Patent Document 1 has a problem in that when a region of interest suspected of being a lesion is identified using a learning model, it is unable to improve the accuracy of the determination by adjusting the amount of light irradiated onto the region of interest.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide a computer program, etc. that automatically adjusts the amount of light irradiated to a region of interest that is suspected of being a lesion identified by a learning model. [Means for solving the problem]

[0006] A computer program according to one embodiment of the present disclosure causes a computer to execute a process of acquiring an image captured by an endoscope, inputting the image into a learning model trained to output a recognition result of a region of interest contained in the image when the image captured by the endoscope is input, outputting the recognition result, and adjusting the amount of light irradiated to the region of interest by an LED provided at the tip of the endoscope based on the recognition result.

[0007] In one embodiment of the computer program of the present disclosure, the learning model outputs a probability that the region of interest included in the image is a lesion, and adjusts the light amount if the probability is less than a predetermined value.

[0008] A computer program according to an embodiment of the present disclosure outputs an image including the region of interest on a screen separate from the screen on which the image is displayed, if the certainty that the region of interest is a lesion is less than a predetermined value.

[0009] A computer program according to one embodiment of the present disclosure, when the certainty that the region of interest is a lesion is equal to or greater than a predetermined value, switches the acquired image on the screen on which the image is displayed to an enlarged image of the region of interest, or outputs an image including the region of interest on a screen separate from the screen on which the image is displayed.

[0010] In one embodiment of the present disclosure, the computer program includes a plurality of LEDs for adjusting the amount of light emitted by the LEDs corresponding to the position of the region of interest.

[0011] A computer program according to an embodiment of the present disclosure accepts a change in a lower limit or an upper limit of the amount of light that the LED irradiates on the region of interest.

[0012] A computer program according to one embodiment of the present disclosure obtains the whiteness of the image, and if the whiteness is equal to or greater than a reference value, reduces the amount of light, and if the whiteness is less than the reference value, increases the amount of light.

[0013] A computer program according to an embodiment of the present disclosure changes the amount of light to the amount of light before the adjustment when the region of interest is no longer included in an image acquired after the amount of light is adjusted.

[0014] An information processing method according to one embodiment of the present disclosure includes acquiring an image captured by an endoscope, inputting the image into a learning model trained to output a recognition result of a region of interest contained in the image when the image captured by the endoscope is input, outputting the recognition result, acquiring the recognition result of the region of interest contained in the image from the learning model, and adjusting the amount of light irradiated to the region of interest by an LED provided at the tip of the endoscope based on the recognition result.

[0015] An endoscope according to one embodiment of the present disclosure includes an LED provided at a tip, an acquisition unit that acquires an image captured by the endoscope, an output unit that inputs the acquired image into a learning model that is trained to output a recognition result of a region of interest contained in an image captured by the endoscope when the image is input, and outputs the recognition result, and a light intensity adjustment unit that adjusts the amount of light that the LED irradiates to the region of interest based on the recognition result. Effect of the Invention

[0016] According to an embodiment of the present disclosure, the amount of light irradiated to a region of interest with a low degree of lesion certainty is automatically adjusted, so that it is possible to improve the accuracy of lesion determination. [Brief description of the drawings]

[0017] [Figure 1] 1 is an external view of an endoscope according to a first embodiment. [Diagram 2] FIG. 2 is a plan view of the tip of the insertion tube. [Diagram 3] FIG. 1 is a block diagram of an endoscope apparatus. [Figure 4] FIG. 1 is an explanatory diagram of a learning model (region of interest learning model). [Diagram 5] 1 is an explanatory diagram showing an example of an image (controller acquired image) acquired by the control unit from a learning model. FIG. [Figure 6] FIG. 11 is an explanatory diagram showing an example of an LED correspondence table showing the correspondence between screen areas and white LEDs. [Figure 7] FIG. 2 is an explanatory diagram showing an example of an image (monitor image) displayed on a monitor. [Figure 8] 11 is an explanatory diagram showing an example of an image (a monitor image after light amount adjustment) displayed on the monitor after the light amount is increased. FIG. [Figure 9] 10 is a flowchart showing an example of a processing procedure performed by a control unit of the processor device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] (Embodiment 1) The present invention will be described in detail below with reference to the drawings showing the embodiments. Fig. 1 is an external view of an endoscope 1 according to the first embodiment. The endoscope 1 includes an insertion tube 2, an operation section 3, a universal tube 4, and a connector section 5. The insertion tube 2 is a section to be inserted into a body cavity, and includes a long flexible section 20 and a tip section 22 connected to one end of the flexible section 20 via a curved section 21. The other end of the flexible section 20 is connected to the operation section 3 via a cylindrical connecting section 23. The universal tube 4 has one end connected to the operation section 3 and extends in a different direction from the insertion tube 2, and the connector section 5 is connected to the other end of the universal tube 4.

[0019] The operation section 3 is provided to be held by a user (operator) of the endoscope 1, such as a doctor, to perform various operations, and includes a bending operation knob 30, a plurality of operation buttons 31, etc. The bending operation knob 30 is connected to the bending section 21 by a wire (not shown) passing through the connecting section 23 and the inside of the flexible section 20. The bending section 21 is bent in two directions perpendicular to each other in the axial cross section by operating the bending operation knob 30, and the orientation of the tip section 22 inserted into the body cavity changes.

[0020] FIG. 2 is a plan view of the tip of the insertion tube. The tip 22 includes an imaging unit 6, an objective lens 25, and an illumination unit 7. The imaging unit 6 is incorporated inside the tip 22, facing the inside of the objective lens 25. That is, the imaging unit 6 captures an image of an object to be observed (subject), which is an internal part such as a body cavity, through the objective lens 25. The objective lens 25 is fitted into the inner frame of a hole provided in the tip 22 of the insertion tube 2, and functions as an observation window. In FIG. 2, the positions of the imaging unit 6 and the objective lens 25 are indicated by two-dot chain lines. The illumination unit 7 is incorporated facing the inside of a light distribution lens (not shown).

[0021] The imaging unit 6 includes an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) and an optical system for forming an image on the imaging surface of the imaging element, and images the inside of the body cavity through the objective lens 25. The objective lens 25 is, for example, a wide-angle lens, and the imaging unit 6 is configured to be able to capture images at a viewing angle of 180° or more by setting the optical system including the objective lens 25. The imaging unit 6 outputs imaging data (image signal) of the captured observation object (subject) to a receiving circuit 61 (see FIG. 3). The imaging data (image signal) output by the imaging unit 6 is subjected to pre-processing such as AD conversion or white balance correction in the receiving circuit 61 and the gain circuit 62 (see FIG. 3), and is output to the signal processing circuit 12 (see FIG. 3) of the processor device 10.

[0022] The illumination unit 7 includes an annular substrate 70 surrounding the imaging unit 6, and a plurality of white LEDs 71-78 mounted on one surface of the substrate 70 facing the light distribution lens 26. The white LEDs 71-78 are arranged at approximately equal intervals on one surface of the annular substrate 70. The white LEDs 71-78 are formed, for example, by covering the light emitting surface of a blue LED chip that emits blue light with a yellow phosphor. The white LED 71 may be another light emitting element such as an LD or optical fiber. The number of white LEDs included in the endoscope is not limited to eight.

[0023] 3 is a block diagram of an endoscope device. The endoscope 1 is connected to a processor device 10 via a connector unit 5. The processor device 10 includes a control unit 11, a signal processing circuit 12, an input / output I / F 13, a storage unit 16, and the like. The control unit 11, the signal processing circuit 12, the input / output I / F 13, and the storage unit 16 are communicatively connected via an internal bus. The control unit 11 has an arithmetic processing device equipped with a timekeeping function, such as one or more central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), and the like, and performs various information processing, control processing, and the like related to the processor device 10 by reading and executing programs stored in the storage unit 16.

[0024] The endoscope 1 includes an imaging driver 60 that drives the imaging unit 6 and an illumination driver 79 that drives the illumination unit 7. The imaging driver 60 drives the imaging unit 6 by a rolling shutter system in accordance with a control command given from the control unit 11 of the processor device 10. The imaging driver 60 outputs an image signal to a gain circuit 62 on a frame-by-frame basis via a receiving circuit 61. The gain circuit 62 performs predetermined pre-processing such as white balance processing on the image signal, and outputs the pre-processed image signal to the signal processing circuit 12 of the processor device 10 as imaging data.

[0025] The illumination driving unit 79 drives the illumination unit 7 in accordance with a control command given from the control unit 11, causing the white LEDs 71-78 to emit light. The imaging operation of the imaging unit 6 is executed in synchronization with the driving of the illumination unit 7, and image outputs obtained under illumination by the white LEDs 71-78 are continuously input to the signal processing circuit 12. The illumination unit 7 can be switched between a driven state and a non-driven state by operating the operation button 31 provided on the operation unit 3.

[0026] The signal processing circuit 12 performs image processing such as gamma correction and interpolation processing on the input imaging data (image signal), and outputs the processed imaging data to the control unit 11.

[0027] The input / output I / F 13 is compliant with a communication standard such as USB or DSUB, and is a communication interface for serial communication with an external device connected to the input / output I / F 13. A monitor 14 and an input unit 15 such as a mouse and a keyboard are connected to the input / output I / F 13. The control unit 11 performs information processing based on an execution command or an event input from the input unit 15. The control unit 11 also performs further image processing on an image output by a learning model 161 (described later), and outputs the image to an external monitor 14. The monitor 14 is a display device such as a liquid crystal display or an organic EL display, and displays an image captured by the imaging unit 6 based on an image signal output from the processor device 10. A user of the endoscope 1 can observe a desired location in a body cavity by viewing the display on the monitor 14.

[0028] The endoscope 1 further includes a light amount adjustment section 8. The light amount adjustment section 8 may be mounted on the substrate 70 in the same manner as the white LEDs 71-78. The light amount adjustment unit 8 is connected to the control unit 11 of the processor device 10 via an internal bus so as to be able to communicate with each other. Based on information transmitted from the control unit 11 of the processor device 10, the light amount adjustment unit 8 determines which of the white LEDs 71 to 78 should have its light amount adjusted, and increases or decreases the amount of light emitted by the white LED. The amount of light emitted by the white LEDs 71 to 78 is adjusted, for example, by PWM (Pulse Width Modulation) control, and the duty ratio is lowered when the amount of light is decreased, and the duty ratio is increased when the amount of light is increased. The lower limit and upper limit of the amount of light emitted by the white LEDs 71 to 78 are determined by setting an allowable range of the duty ratio. The allowable range of the duty ratio is, for example, 0.2 to 0.8, but can be changed by receiving an input from the input unit 17. The light amount of the white LED when the light amount is not adjusted is the normal light amount, and the duty ratio at this time is, for example, 0.5.

[0029] The storage unit 16 includes a volatile storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, etc., and a non-volatile storage area such as an EEPROM or a hard disk. Programs and data to be referenced during processing are stored in advance in the storage unit 16. The programs stored in the storage unit 16 may be programs read from a recording medium 16a that is readable by the control unit 11 and stored therein. Alternatively, the programs may be programs downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the storage unit 16.

[0030] The memory unit 16 stores entity files (instance files of a neural network (NN)) constituting a learning model 161 (region of interest learning model) described below. These entity files may be configured as part of a program. Note that an information processing device connected to the processor device 10 may be provided separately, the learning model 161 may be stored in the information processing device, and calculations based on the learning model 161 may be performed by the information processing device. Furthermore, the learning model 161 may be stored in a server provided in a hospital where the endoscope 1 is used or in a cloud server provided outside the hospital, and calculations based on the learning model 161 may be performed by the server or the cloud server. The storage unit 16 further stores various preset values ​​(preset data) that are determined in advance when generating and outputting (displaying) an image. The preset data may include, for example, a normal light amount, an upper limit or lower limit of the light amount irradiated by the white LEDs 71-78, a fixed value and a predetermined value for classifying a lesion probability indicating the probability (probability) that the region of interest is a lesion, or a reference value for the whiteness of the screen. The details of the preset data will be described later. The various preset data can be changed by receiving an input at the input unit 17. The storage unit 16 also stores an LED correspondence table 162 .

[0031] FIG. 4 is an explanatory diagram of a learning model 161 (region of interest learning model). The learning model 161 is a model for performing object detection, such as RCNN (Regions with Convolutional Neural Network), Fast RCNN, Faster RCNN, SSD (Single Shot Multibook Detector), YOLO (You Only Look Once), or a neural network (NN) having a segmentation network function. When the learning model 161 is configured with a neural network including a CNN (Convolutional Neural Network) that extracts image features, such as RCNN, the input layer included in the learning model 161 has a plurality of neurons that accept input of pixel values ​​of an endoscopic image, and transfers the input pixel values ​​to an intermediate layer. The intermediate layer has a plurality of neurons that extract image features of an endoscopic image, and transfers the extracted image features to an output layer. The output layer has one or a plurality of neurons that output information about the region of interest, including the position of the region of interest, and outputs the position of the region of interest and the lesion probability based on the image features output from the intermediate layer. Alternatively, the learning model 161 may input the image feature value output from the intermediate layer to a support vector machine (SVM) to recognize the lesion. The position of the region of interest is defined by image coordinate values. The neural network (learning model 161) trained using training data is expected to be used as a program module that is a part of artificial intelligence software. The learning model 161 is used in the control unit 11 (CPU, etc.) as described above, and is executed by the control unit 11 having arithmetic processing capabilities in this way to configure a neural network system. That is, the control unit 11 performs a calculation to extract the feature value of the endoscopic image input to the input layer according to a command from the learning model 161 stored in the storage unit 16, and outputs the position of the region of interest and the lesion probability from the output layer together with the image. As shown in the image of the lesion detection result in Fig. 4, control unit 11 surrounds the region of interest with a bounding box of dashed lines based on the output of learning model 161, and displays the lesion probability of the region of interest above the bounding box. In Fig. 4, the lesion probability is 0.6.

[0032] Learning model 161 includes the position of the region of interest and the lesion accuracy in the image as information about the region of interest, and outputs the image to control unit 11. If the lesion accuracy of the region of interest is less than a certain value, for example, 0.2, learning model 161 does not output information about the region of interest to control unit 11. The certain value can be changed by receiving an input at input unit 17.

[0033] FIG. 5 is an explanatory diagram showing an example of an image (controller acquired image) acquired by the control unit from the learning model. In the image acquired by the control unit 11 from the learning model 161, the region of interest is surrounded by a bounding box of dashed lines, and the lesion accuracy of the region of interest is displayed at the top of the bounding box. When the lesion accuracy is less than a predetermined value, the control unit 11 adjusts the light intensity of the white LED corresponding to the position of the region of interest. The predetermined value is, for example, 0.7, but can be changed by receiving an input at the input unit 17. A method of identifying the white LED for which the light intensity is adjusted will be described later.

[0034] FIG. 6 is an explanatory diagram showing an example of an LED correspondence table showing the correspondence between screen regions and white LEDs. As shown in FIG. 5, the image acquired by the control unit 11 is divided into eight sections (1 to 8) at 45 degrees each radially from the center of the screen. As shown in FIG. 6, in the LED correspondence table 162, the screen regions 1 to 8 correspond to the white LEDs 71 to 78, respectively, and the control unit 11 adjusts the amount of light emitted by the white LEDs corresponding to the screen region including the region of interest output by the learning model 631. The region information for identifying the screen region is stored in a memory such as a RAM included in the storage unit 16, for example, and is defined by the range of the image coordinate system in the image output by the learning model 161. For example, when the region of interest is included in the screen region 4 as shown in Fig. 5, the control unit 11 adjusts the amount of light emitted by the white LED 74. Also, for example, after the control unit 11 acquires the image shown in Fig. 5, when the endoscope 1 rotates 180 degrees around the axis along which the insertion tube 2 extends in the body cavity and the region of interest is included in the screen region 8, the control unit 11 ends the adjustment of the amount of light emitted by the white LED 74 and adjusts the amount of light emitted by the white LED 78. Note that, when one region of interest is included across multiple screen regions, the amount of light emitted by all the LEDs corresponding to the multiple screen regions may be adjusted, or the amount of light emitted by the white LED corresponding to the screen region including the largest portion of the region of interest may be adjusted.

[0035] FIG. 7 is an explanatory diagram showing an example of an image (monitor image) displayed on a monitor. When the image acquired from the learning model 161 contains information about the region of interest (the position of the region of interest and the lesion certainty), the control unit 11 separately displays an image including the region of interest together with the lesion certainty on a sub-screen on the monitor 14 as shown in FIG. 7. At this time, an image in which the region of interest is enlarged may be displayed on the sub-screen, and in that case, as shown in FIG. 7, the enlargement magnification may be displayed in the sub-screen. In the sub-screen of the monitor image shown in FIG. 7, the enlargement magnification is 1.4 times. Also, a monitor separate from the monitor 14 may be provided, and the image including the region of interest may be displayed on the monitor. When the image acquired by the control unit 11 from the learning model 161 contains multiple regions of interest, for example, the image containing each region of interest is displayed on multiple sub-screens, and the color of the dashed line surrounding the region of interest and the color of the border line of the sub-screens are made to correspond to each other by the same color.

[0036] 8 is an explanatory diagram showing an example of an image (monitor image after light amount adjustment) displayed on the monitor after the light amount is increased. When the lesion probability of the region of interest included in the image acquired from the learning model 161 is less than a predetermined value, the control unit 11 acquires the whiteness of the image, and determines whether to decrease or increase the light amount of the white LED corresponding to the position of the region of interest based on the whiteness. The whiteness is, for example, a value expressed as a percentage of the ratio of the white part of the endoscope screen, and becomes high when the amount of light emitted by the white LED is large due to halation or the like. For example, when the entire image is dark and the whiteness of the image is lower than the reference value as shown in Fig. 7, the control unit 11 increases the light intensity of the white LED corresponding to the position of the region of interest, and displays a light-intensity-adjusted monitor image with high whiteness as shown in Fig. 8 on the monitor 14. The reference value is, for example, 50, but can be changed by receiving an input at the input unit 17. Note that for the light-intensity-adjusted monitor image, a light-intensity-adjusted mark 14a may be displayed as shown in Fig. 8. Even if the lesion probability of the region of interest is higher than a predetermined value, control unit 11 separately displays the region of interest together with the lesion probability in a sub-screen on monitor 14. In the monitor image after light amount adjustment shown in Fig. 8, the lesion probability of the region of interest is 0.95. When the whiteness of the image acquired by the control unit 11 from the learning model 161 is equal to or greater than the reference value, the control unit 11 reduces the amount of light from the white LED corresponding to the position of the region of interest. When the amount of light from the white LED is reduced, the method of displaying the image on the monitor 14 is the same as when the amount of light from the white LED is increased.

[0037] When the endoscope 1 moves inside the body cavity and the region of interest is no longer included in the image captured by the endoscope 1, the control unit 11 gradually changes the light amount of the white LED that has been adjusted for light amount to the normal light amount.

[0038] FIG. 9 is a flowchart showing an example of a processing procedure performed by the control unit of the processor device. The control unit 11 of the processor device 10 starts the process of the flowchart based on, for example, the input contents from the input unit 17 connected to the processor device 10. The amount of light emitted by the white LEDs 71 to 78 at the start of the process is the normal amount of light.

[0039] The control unit 11 of the processor device 10 acquires an endoscopic image output from the endoscope 1 (S1). The control unit 11 may acquire the endoscopic image from the endoscope 1 in synchronization with the start of imaging inside a body cavity by the endoscope 1. The endoscopic image may be a still image or a video image.

[0040] The control unit 11 of the processor device 10 inputs an endoscopic image to the learning model 161 (S2). When the input endoscopic image contains a region of interest and the lesion accuracy of the region of interest is a certain value or more, the learning model 161 outputs the position of the region of interest and the lesion accuracy as information related to the region of interest. When the input endoscopic image does not contain a region of interest or the lesion accuracy of the region of interest is less than a certain value, the learning model 161 does not output information related to the region of interest to the control unit 11.

[0041] The control unit 11 of the processor device 10 determines whether or not information about the region of interest has been acquired from the learning model 161 (S3). If information about the region of interest has not been acquired from the learning model 161 (S3: NO), the control unit 11 changes the light intensity of the white LED to the normal light intensity (S31). Note that if the normal light intensity was set before the process of S31 was executed, the normal light intensity is maintained. The control unit 11 performs a loop process to execute the process of S1 again.

[0042] When information about the region of interest is acquired from the learning model 161 (S3: YES), the control unit 11 of the processor device 10 determines whether the lesion accuracy included in the information about the region of interest is equal to or greater than a predetermined value (S4). When the lesion accuracy is equal to or greater than the predetermined value (S4: YES), the control unit 11 of the processor device 10 displays the region of interest on a sub-screen (S5), and ends the process.

[0043] If the lesion probability is less than a predetermined value (S4: NO), the control unit 11 of the processor device 10 displays the region of interest on a sub-screen (S41). The control unit 11 identifies a white LED corresponding to the position of the region of interest based on the position of the region of interest included in the information about the region of interest and the LED correspondence table (S42). The white LED identified at this time is a white LED that adjusts the light amount.

[0044] The control unit 11 of the processor device 10 judges whether the whiteness of the acquired endoscope screen is equal to or greater than a reference value (S43). If the whiteness of the endoscope screen is equal to or greater than the reference value (S43: YES), the control unit 11 of the processor device 10 reduces the amount of light emitted by the white LED identified in S42 (S431), and if it is less than the reference value (S43: NO), increases the amount of light emitted by the white LED identified in S42 (S432). For example, when the control unit 11 reduces the amount of light emitted by the white LED, it gradually lowers the duty ratio output to the white LED from 0.5 to 0.2, and when it increases the amount of light, it gradually raises the duty ratio output to the white LED from 0.5 to 0.8.

[0045] The control unit 11 of the processor device 10 judges whether the amount of light emitted by the white LED is the upper limit or the lower limit based on the duty ratio output to the white LED (S44). If the amount of light is the upper limit or the lower limit (S44: YES), the control unit 11 changes the amount of light emitted by the white LED to a normal amount of light (S45) and ends the process. If the amount of light is not the upper limit or the lower limit (S44: NO), the control unit 11 performs a loop process to execute the process of S1 again.

[0046] According to this embodiment, the amount of light irradiated to a region of interest with a low lesion certainty is automatically adjusted, so that the accuracy of lesion determination can be improved.

[0047] (Modification) In the above embodiment, the position of the region of interest is defined by image coordinate values, but the position of the region of interest may be defined by pixel numbers, in which case the screen region is defined by a range of pixel numbers. Furthermore, when the data output by the learning model 161 is in the form of an image array, the position of the region of interest may be defined by an array number, in which case the screen region is defined by the range of the array number.

[0048] In the above embodiment, when the lesion certainty of the region of interest included in the acquired endoscopic image is equal to or greater than a predetermined value, the region of interest is displayed on a sub-screen, but instead of displaying the region on a sub-screen, the region of interest having a lesion certainty of equal to or greater than a predetermined value may be enlarged and displayed. Also, the region of interest may be enlarged and displayed on a sub-screen.

[0049] In the above-described embodiment, a determination was made as to whether the whiteness of the screen was equal to or greater than a reference value, and a decision was made as to whether to decrease or increase the amount of light emitted by the white LED. However, a photometric circuit may be provided within the endoscope, and based on the amount of reflected light within the body cavity measured by the photometric circuit, a decision may be made as to whether to decrease or increase the amount of light emitted by the white LED.

[0050] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and the scope equivalent to the claims. [Explanation of symbols]

[0051] 1 Endoscope 2 Insertion tube 20 Soft part 21 Curved section 22 Tip 23 Connecting part 25 Objective Lens 26 Light distribution lens 3 Control section 31 Operation buttons 4 Universal Tube 5 Connector part 6. Imaging unit 60 Imaging drive unit 61 Receiving circuit 62 Gain circuit 7. Lighting Section 70 Substrate 71~78 White LED 79 Lighting drive unit 8 Light intensity adjustment section 10. Processor Unit 11 Control section 12 Signal Processing Circuit 13 Input / Output Interface 14 Monitor 15 Input section 16 Memory section 161 Learning Model 16a Recording media

Claims

1. An image captured by an endoscope is obtained. inputting an image captured by the endoscope into a learning model that has been trained to output a recognition result of a region of interest included in the image when the image is input, and outputting the recognition result; Based on the recognition result, the amount of light irradiated onto the region of interest by an LED provided at the tip of the endoscope is adjusted. A computer program for causing a computer to execute a process, The learning model outputs a likelihood that the region of interest included in the image is a lesion; and The amount of light is adjusted when the probability is less than a predetermined value. Computer program.

2. An image captured by an endoscope is obtained. inputting an image captured by the endoscope into a learning model that has been trained to output a recognition result of a region of interest included in the image when the image is input, and outputting a recognition result including a probability that the region of interest is a lesion; adjusting the amount of light irradiated onto a region of interest by an LED provided at a tip of the endoscope based on the recognition result; If the likelihood that the region of interest is a lesion is less than a predetermined value, an image including the region of interest is output on a screen separate from the screen on which the image is displayed. A computer program that causes a computer to carry out processing.

3. An image captured by an endoscope is obtained. inputting an image captured by the endoscope into a learning model that has been trained to output a recognition result of a region of interest included in the image when the image is input, and outputting a recognition result including a probability that the region of interest is a lesion; adjusting the amount of light irradiated onto a region of interest by an LED provided at a tip of the endoscope based on the recognition result; When the likelihood that the region of interest is a lesion is equal to or greater than a predetermined value, the screen on which the acquired image is displayed switches from the image to an enlarged image of the region of interest, or outputs an image including the region of interest on a screen separate from the screen on which the image is displayed. A computer program that causes a computer to carry out processing.

4. The learning model outputs a likelihood that the region of interest included in the image is a lesion; and The amount of light is adjusted when the probability is less than a predetermined value.

4. A computer program according to claim 2 or 3.

5. If the likelihood that the region of interest is a lesion is less than a predetermined value, an image including the region of interest is output on a screen separate from the screen on which the image is displayed.

4. A computer program according to claim 1 or 3.

6. When the likelihood that the region of interest is a lesion is equal to or greater than a predetermined value, the screen on which the acquired image is displayed is switched from the image to an enlarged image of the region of interest, or an image including the region of interest is output on a screen separate from the screen on which the image is displayed.

3. A computer program according to claim 1 or 2.

7. the endoscope includes a plurality of LEDs; Adjust the amount of light emitted by the LED corresponding to the position of the region of interest A computer program according to any one of claims 1 to 6.

8. Accepting a change to a lower or upper limit of the amount of light that the LED irradiates on the region of interest. A computer program according to any one of claims 1 to 7.

9. Obtaining the whiteness of the image; If the whiteness is equal to or greater than a reference value, the amount of light is reduced; If the whiteness is less than the reference value, the amount of light is increased. A computer program according to any one of claims 1 to 8.

10. If the region of interest is no longer included in the image acquired after the light amount is adjusted, the light amount is changed to the light amount before the adjustment. A computer program according to any one of claims 1 to 9.

11. The control unit acquires an image captured by the endoscope, When an image captured by an endoscope is input by the control unit, the image is input to a learning model that has been trained to output a recognition result of a region of interest included in the image, and the recognition result is output; A control unit obtains a recognition result of a region of interest included in the image from the learning model; A control unit adjusts the amount of light that an LED provided at the tip of the endoscope irradiates onto the region of interest based on the recognition result.

1. An information processing method, comprising: The learning model outputs a likelihood that the region of interest included in the image is a lesion; and A control unit adjusts the amount of light when the accuracy is less than a predetermined value. Information processing methods.

12. The control unit acquires an image captured by the endoscope, When an image captured by an endoscope is input by the control unit, the image is input to a learning model that has been trained to output a recognition result of a region of interest included in the image, and the recognition result is output; A control unit obtains a recognition result of a region of interest included in the image from the learning model; A control unit adjusts the amount of light that an LED provided at the tip of the endoscope irradiates onto the region of interest based on the recognition result.

1. An information processing method, comprising: When the probability that the region of interest is a lesion is less than a predetermined value, the control unit outputs an image including the region of interest on a screen separate from the screen on which the image is displayed. Information processing methods.

13. The control unit acquires an image captured by the endoscope, When an image captured by an endoscope is input by the control unit, the image is input to a learning model that has been trained to output a recognition result of a region of interest included in the image, and the recognition result is output; A control unit obtains a recognition result of a region of interest included in the image from the learning model; A control unit adjusts the amount of light that an LED provided at the tip of the endoscope irradiates onto the region of interest based on the recognition result.

1. An information processing method, comprising: When the probability that the region of interest is a lesion is equal to or greater than a predetermined value, the control unit switches the acquired image on a screen on which the image is displayed to an enlarged image of the region of interest, or outputs an image including the region of interest on a screen separate from the screen on which the image is displayed. Information processing methods.

14. An LED provided at the tip; an acquisition unit for acquiring an image captured by an endoscope; an output unit that, when an image captured by an endoscope is input, inputs the acquired image into a learning model that is trained to output a recognition result of a region of interest included in the image, and outputs the recognition result; a light amount adjusting unit that adjusts the amount of light that the LED irradiates onto the region of interest based on the recognition result; An endoscope comprising: The learning model outputs a likelihood that the region of interest included in the image is a lesion; and The light amount adjusting unit adjusts the light amount when the probability is less than a predetermined value. Endoscope.

15. An LED provided at the tip; an acquisition unit for acquiring an image captured by an endoscope; an output unit that, when an image captured by an endoscope is input, inputs the acquired image into a learning model that is trained to output a recognition result of a region of interest included in the image, and outputs the recognition result; a light amount adjusting unit that adjusts the amount of light that the LED irradiates onto the region of interest based on the recognition result; An endoscope comprising: When the probability that the region of interest is a lesion is less than a predetermined value, the output unit outputs an image including the region of interest on a screen separate from a screen on which the image is displayed. Endoscope.

16. An LED provided at the tip; an acquisition unit for acquiring an image captured by an endoscope; an output unit that, when an image captured by an endoscope is input, inputs the acquired image into a learning model that is trained to output a recognition result of a region of interest included in the image, and outputs the recognition result; a light amount adjusting unit that adjusts the amount of light that the LED irradiates onto the region of interest based on the recognition result; An endoscope comprising: When the likelihood that the region of interest is a lesion is equal to or greater than a predetermined value, the output unit switches the acquired image on a screen on which the image is displayed to an enlarged image of the region of interest, or outputs an image including the region of interest on a screen separate from the screen on which the image is displayed. Endoscope.

Images