Endoscopic image diagnosis assistance processor, endoscopic image diagnosis assistance processor operation method, and endoscopic image diagnosis assistance processor program

Abstract

[Problem] To provide an endoscopic image diagnosis assistance processor 2 for stably and efficiently identifying an observation organ. [Solution] An endoscopic image diagnosis assistance processor 2 comprises: a site inference unit 13 that uses a first model to infer a degree of certainty P that an endoscopic image of a prescribed frame rate X is an image of a site; an inference interval setting unit 17 that sets the prescribed frame rate X at which the site inference unit 13 performs the inference from a plurality of endoscopic images inputted at a first frame rate; a site identification unit 14 that uses the degree of certainty P to identify an observation site; and an organ identification unit 15 that identifies an observation organ corresponding to the observation site. The inference interval setting unit 17 increases the prescribed frame rate X to a third frame rate if the observation site cannot be identified continuously for a number of times equal to or greater than a first determination value TK1, and returns the prescribed frame rate to the second frame rate once the observation site is identified.

Description

Endoscopic Image Diagnosis Support Processor, Operating Method of Endoscopic Image Diagnosis Support Processor, and Program for Endoscopic Image Diagnosis Support Processor

[0001] Embodiments of the present invention relate to an endoscopic image diagnosis support processor capable of efficiently identifying an organ being observed, an operating method of the endoscopic image diagnosis support processor capable of efficiently identifying an organ being observed, and a program for the endoscopic image diagnosis support processor capable of efficiently identifying an organ being observed.

[0002] For an image captured using an endoscope, computer-aided detection (CADe: Computer-Aided Detection) indicating the position of a lesion candidate and computer-aided diagnosis (CADx: Computer-Aided Diagnosis) indicating discrimination information of the lesion candidate are known (hereinafter, CADe and CADx are collectively referred to as "CAD"). In CAD, for example, a machine learning model that has performed deep learning or the like using a plurality of images as teacher data is used.

[0003] International Publication No. WO2022 / 181748 discloses a technique for generating a plurality of models respectively adapted to a plurality of organs by learning using images for each of the plurality of organs. For example, create a plurality of types of models such as a pharynx model learned using only pharynx images and an esophagus model learned using only esophagus images. Then, by using a model adapted to the organ being observed with the endoscope from among the plurality of types of models, the accuracy as CAD can be increased.

[0004] Japanese Patent Application Laid-Open No. 2007-151809 discloses a receiving device that detects that the organ being photographed by a capsule endoscope has changed and receives data from the capsule endoscope.

[0005] However, it is not easy to identify the organ being observed with an endoscope when the imaging conditions are not good, and in some cases, there is a risk of identifying the wrong organ. Also, even when the imaging conditions are good, there are cases where the organ being observed cannot be correctly identified 100%.

[0006] International Publication WO2022 / 181748, Japanese Patent Publication No. 2007-151809

[0007] Embodiments of the present invention aim to provide an endoscopic image diagnostic support processor for efficiently identifying an organ being observed with an endoscope, a method for operating the endoscopic image diagnostic support processor for efficiently identifying an organ being observed with an endoscope, and a program for the endoscopic image diagnostic support processor that enables a computer to efficiently identify an organ being observed with an endoscope.

[0008] The endoscopic image diagnostic support processor according to an embodiment of the present invention comprises: a site inference unit that uses a first model trained on multiple images of multiple sites as training data to infer the confidence level of each of multiple endoscopic images at a predetermined frame rate being an image of each of the multiple sites; an inference interval setting unit that sets the predetermined frame rate at which the site inference unit performs inference from the multiple endoscopic images input at a first frame rate; a site identification unit that identifies an observation site based on predetermined conditions using the confidence level; and an organ identification unit that identifies an observation organ which is an organ corresponding to the observation site. The inference interval setting unit raises the predetermined frame rate from a second frame rate, which is the standard frame rate, to a third frame rate if the site identification unit fails to identify the observation site for a number of consecutive times equal to or greater than a first determination value, and returns the predetermined frame rate to the second frame rate if the site identification unit is able to identify the observation site.

[0009] The operation method of the endoscopic image diagnostic support processor according to an embodiment of the present invention involves using a first model that has learned multiple images of multiple body parts as training data to infer the confidence level of each of the multiple endoscopic images at a predetermined frame rate that each of the multiple body parts is an image of the respective body part, setting the predetermined frame rate for inference from the multiple endoscopic images input at a first frame rate, identifying the observation site based on predetermined conditions using the confidence level, identifying the observation organ that corresponds to the observation site, and if the observation site cannot be identified for a number of consecutive times equal to or greater than a first judgment value, increasing the predetermined frame rate from the second frame rate, which is the standard frame rate, to the third frame rate, and returning the predetermined frame rate to the second frame rate if the observation site can be identified.

[0010] The program for the endoscopic image diagnostic support processor according to an embodiment of the present invention uses a first model that has learned multiple images of multiple body parts as training data to infer the confidence level of each of the multiple endoscopic images at a predetermined frame rate that each of the multiple body parts is an image of the respective body part, sets the predetermined frame rate for inference from the multiple endoscopic images input at a first frame rate, identifies the observation site based on predetermined conditions using the confidence level, identifies the observation organ that corresponds to the observation site, raises the predetermined frame rate from a second frame rate (standard frame rate) to a third frame rate if the observation site cannot be identified for a number of consecutive times equal to or greater than a first judgment value, and returns the predetermined frame rate to the second frame rate if the observation site can be identified.

[0011] According to embodiments of the present invention, it is possible to provide an endoscopic image diagnostic support processor that efficiently identifies an organ being observed with an endoscope, a method for operating the endoscopic image diagnostic support processor that efficiently identifies an organ being observed with an endoscope, and a program for the endoscopic image diagnostic support processor that allows a computer to efficiently identify an organ being observed with an endoscope.

[0012] Figure 1 is a configuration diagram of the endoscopic image diagnostic support processor of the embodiment. Figure 2 is a flowchart of the operation method of the endoscopic image diagnostic support processor of the first embodiment. Figure 3 is an example of the screen of the display device of the embodiment. Figure 4 is a diagram for explaining the operation method of the first embodiment. Figure 5 is a diagram for explaining the operation method of the first embodiment. Figure 6 is a flowchart of the operation method of the endoscopic image diagnostic support processor of modification 1 of the first embodiment. Figure 7 is a flowchart of the operation method of the endoscopic image diagnostic support processor of modification 2 of the first embodiment. Figure 8 is a flowchart of the operation method of the endoscopic image diagnostic support processor of the second embodiment. Figure 9 is a diagram for explaining the operation method of the second embodiment.

[0013] Embodiments of the present invention will be described below with reference to the drawings. The drawings based on the embodiments are schematic. The illustration and reference numerals of some components have been omitted.

[0014] <First Embodiment> As shown in Figure 1, the endoscopic image diagnostic support processor 2 (hereinafter referred to as "processor 2") of this embodiment, together with the endoscope 9, display device 30, notification unit 31, and storage device 32, constitutes the endoscopic system 1.

[0015] The camera unit 8 of the endoscope 9 captures images of the inside of the subject's body. The endoscope 9 outputs image data at a predetermined frame rate. The user can capture still images, for example, by operating the operation buttons 7 on the control panel (not shown) of the endoscope 9. The endoscope 9 may be either a flexible or rigid endoscope.

[0016] The processor 2 includes an observed organ determination device 10 (hereinafter referred to as "determination device 10") and a lesion detection device 20.

[0017] The determination device 10 includes a CPU 11, an image input unit 12, a site inference unit 13, a site identification unit 14, an organ identification unit 15, a memory 16, and an inference interval setting unit 17. The processor 2 is also connected to the server 40.

[0018] The CPU 11 is a system controller that controls the entire processor 2. The image input unit 12 processes images taken by the endoscope 9 at a first frame rate. Alternatively, the image data from the endoscope 9 may be processed by an image processing processor (not shown) or a server 40 before being input to the image input unit 12.

[0019] The site inference unit 13 infers the confidence level for each of the multiple sites in the endoscopic image, whether it is an inappropriate image, or whether it is an undeterminable image. For example, the site inference unit 13 is a first AI processing unit that uses a first model, which has been trained using multiple images of multiple sites as training data, to infer the confidence level for each of the multiple sites in the input endoscopic image using the first model.

[0020] The site identification unit 14 identifies an observation site, which is a part of the endoscopic image, from among multiple sites based on predetermined conditions using a confidence level P.

[0021] The organ identification unit 15 identifies the organ to be observed corresponding to the observation site. The memory 16 stores various data, such as the correspondence between the observation site and the observed organ.

[0022] The inference interval setting unit 17 sets a predetermined frame rate X at which the region inference unit 13 performs inference processing. The frame rate X is less than or equal to the first frame rate.

[0023] The lesion detection device 20 is a second AI processing unit that infers lesions in images using multiple second models (organ CADs) that have been trained using multiple lesion images for multiple organs as training data.

[0024] The lesion detection device 20 includes a nasal cavity CAD (second model 1) 21, an oral cavity CAD (second model 2) 22, a pharyngeal CAD (second model 3) 23, an esophageal CAD (second model 4) 24, a gastric CAD (second model 5) 25, a duodenal CAD (second model 6) 26, and an ON / OFF control unit 29 for controlling the activation / deactivation of each model. Each model may also have its own ON / OFF control unit.

[0025] Furthermore, at least one of the multiple configurations of the processor 2 may consist of internal circuits of semiconductor elements processed by software, or dedicated hardware circuits, or it may include both internal circuits of semiconductor elements and dedicated hardware circuits. At least one of the functional parts of the processor 2 may also be a component of the server 40. For example, the lesion detection device 20 may be an operating part of the server 40.

[0026] For example, the body part inference unit 13 may include an FPGA (Field Programmable Gate Array) with hardware circuits and a GPU (Graphics Processing Unit) with software circuits. Also, the body part identification unit 14, organ identification unit 15, etc., may be part of a CPU 11 that processes data using software.

[0027] The display device 30, which displays the endoscopic image, is a monitor such as a liquid crystal display. The notification unit 31 informs the user of information from the processor 2 using sound, light, etc. The notification unit 31 may be part of the display device 30.

[0028] The storage device 32 is a non-temporary storage device (e.g., a magnetic disk, optical disk, or HDD) that stores the program of the processor 2. The storage device 32 may also be configured as a server 40 connected to the processor 2 via an internet connection or the like.

[0029] As described later, if the part identification unit 14 fails to identify the observed part for a number of consecutive times equal to or greater than the first determination value, the inference interval setting unit 17 raises a predetermined frame rate from the standard frame rate (second frame rate) to the third frame rate. If the part identification unit 14 is able to infer the observed part, the predetermined frame rate is returned to the second frame rate.

[0030] In processor 2, the determination device 10 can efficiently and stably identify the organ being observed in the endoscopic image. The second CAD model of the lesion detection device 20 is created for each of the multiple organs. As a result, processor 2 has good model learning efficiency and high inference reliability.

[0031] <Processor Operation Method> The operation method of processor 2 will be explained according to the flowchart in Figure 2.

[0032] <Step S10> Image Input The endoscopic system 1 of this embodiment performs an examination of the upper gastrointestinal tract. The insertion part of the endoscope 9 is inserted through the oral cavity or nasal cavity of the subject and enters the pharynx. For example, multiple endoscopic images from the endoscope 9 or the image processing processor are input continuously to the image input unit 12 of the processor 2 at a first frame rate (e.g., 120 fps: 120 frames / second). Processing is not performed if no images are input.

[0033] The image input unit 12 performs image processing on images directly input from the endoscope 9 (for example, RAW images) for display on the display device 30. If an image processed by an image processing processor is input, no image processing is required.

[0034] <Step S30> For the endoscopic image input at the first inference frame rate, the inference interval setting unit 17 sets a predetermined frame rate X at which the site inference unit 13 performs inference processing. The frame rate X is variable, and at startup, the frame rate X is set to a second frame rate (for example, 20 fps), which is the standard frame rate.

[0035] The site inference unit 13 infers a confidence level PN (N=1 to n) from the endoscopic image input at a first frame rate, determining that the endoscopic image at a predetermined frame rate X represents each of n different sites. The confidence level P is in the range of (0 to 1). A confidence level P of 0 means a probability of 0%, and a confidence level P of 1 means a probability of 100%.

[0036] The machine learning model in the AI ​​processing unit is a deep neural network (DNN) that performs deep learning with multiple hidden layers. The model can also be a convolutional neural network (CNN), R-CNN (Regions with CNN features) which utilizes CNN, or FCN (Fully Convolutional Networks), etc.

[0037] The first model is trained using images of n different parts of the upper digestive tract as training data. As shown in Table 1 below, in this embodiment, n = 15.

[0038]

[0039] Each of the 15 body parts corresponds to one of the m types of organs. As shown in Table 2 below, in this embodiment, m = 6.

[0040]

[0041] The number of types of body parts n is, for example, between 2 and 100. The number of types of organs m is (m < n), for example, between 2 and 20. The lesion detection device 20 has m types of organ CADs corresponding to m types of organs.

[0042] The site inference unit 13 also infers the confidence level PA of whether the endoscopic image is an inappropriate image. An inappropriate image is a so-called out-of-focus image, a blurred image, etc. For example, a model trained on inappropriate / appropriate images as training data outputs a confidence level PA, and if the confidence level PA is above a predetermined value, the endoscopic image is inferred to be an inappropriate image.

[0043] Furthermore, the site inference unit 13 estimates a confidence level PB for "undeterminable" if the endoscopic image is not any of the n types of sites. For example, a model trained on undeterminable / determinable images as training data outputs a confidence level PB, and if the confidence level PB is greater than or equal to a predetermined value, the endoscopic image is inferred to be an undeterminable image. The site inference unit 13 may also infer "undeterminable" if the confidence levels P for all n types of sites output by the first model are less than or equal to a predetermined value.

[0044] Note that the site inference unit 13 may be configured to infer (n + 2) confidence levels for each of the endoscopic image being an image of n types of sites, an inappropriate image, and an undeterminable image, using one model.

[0045] <Step S40> If the confidence level PX of site X is the maximum confidence level P (YES), the process from Step S50 is performed.

[0046] <Step S50> The site identification unit 14 identifies the endoscopic image as an image of site X. In other words, site X is identified as the observation site being observed.

[0047] <Step S60> The organ identification unit 15 identifies the organ Y corresponding to site X as the observation organ being observed.

[0048] <Step S70> Is the corresponding second model activated? It is determined, for example, by the CPU 11 whether the second model Y (organ CAD) corresponding to organ Y is activated. For example, if site X is site 9 (midbody), the corresponding organ is (organ 5: stomach). The second model Y corresponding to (organ 5: stomach) is the second model 5 (stomach CAD). If the activated second model is the second model 5 (YES), the process proceeds to Step S90. That is, the activated second model 5 is continuously used.

[0049] <Step S80> Activate the corresponding second model (ON) The ON / OFF control unit 29 of the lesion detection device 20 activates the CAD (second model Y) corresponding to the observation organ Y. Also, the ON / OFF control unit 29 stops (turns off) the CAD other than (second model Y).

[0050] <Step S90> The second frame rate inference interval setting unit 17 sets the frame rate X at which the site inference unit 13 performs inference processing to the second frame rate which is the standard frame rate. Of course, if the frame rate X is already the second frame rate, the frame rate change process is not performed.

[0051] <Step S100> K=0 The count number K is initialized (K=0). The count number K is the number of times the part identification unit 14 fails to identify the observation part consecutively. The count number K is also initialized at startup. The count number initialization process (step S100) can be performed at any time after step 50 and before step 110.

[0052] <Step S110> Lesion detection processing: The activated CAD of the lesion detection device 20 infers the lesion from the endoscopic image. For example, the activated gastric CAD 30 performs inference using the second model 5. The inference result is output as a confidence level PP (PP = 0 to 1) inferred by the CAD corresponding to the observed organ Y.

[0053] For example, the confidence level PP1 for normal tissue is 0.66, and the confidence level PP2 for diseased tissue is 0.34 (PP1 + PP2 = 1). The lesion detection device 20 may output either confidence level PP1 or confidence level PP2.

[0054] Furthermore, the "organ CAD" may include a "boundary CAD" that performs lesion detection processing on images containing multiple organs. For example, if the confidence level of site 4 (esophagus) and the confidence level of site 5 (esophagogastric junction) are both above a predetermined confidence level P, the boundary organ including the esophagus and stomach may be identified, and the corresponding boundary CAD may infer lesions in the endoscopic image.

[0055] Furthermore, when processor 2 starts up, i.e., at the start of the examination, there are no organs identified as being observed, and therefore no CAD is running. For this reason, lesion detection processing is not performed.

[0056] <Step S120> The display device 30 displays the observation area identified by the area identification unit 14, and / or the observation organ identified by the organ identification unit 15, as numerical values, illustrations, colors, etc. The display device 30 may also serve as a notification unit that notifies the observed organ.

[0057] Figure 3 shows an example of the screen of the display device 30. The screen in Figure 3 displays the endoscopic image 30A, the status of the judgment device 10 (AUTO mode) 30B, and the name of the detected observation site (antrum) as "Detected Site" 30C. Furthermore, the fact that the CAD currently in operation is (stomach) is graphically displayed using a figure 30D representing a stomach.

[0058] Although not shown in the diagram, if the confidence level PP2 for identifying lesion tissue is above a predetermined threshold, the lesion candidate may be superimposed on the endoscopic image 35A, or the confidence level PP2 may be displayed numerically or in color. For example, lesion candidates with a confidence level PP2 of 0.7 or higher may be superimposed on the endoscopic image 35A in red. Conversely, if the confidence level PP2 for identifying lesion tissue is below a predetermined threshold, no display is made by the lesion detection device 20. Then, the process from step S10 is repeated.

[0059] <Step S130> Confidence levels PA and PB are at their maximum. If the confidence level PX for site X is not at its maximum in step S40 (NO), then the confidence level PA, which indicates that the endoscopic image is an inappropriate image, or the confidence level PB, which indicates that the endoscopic image is an undeterminable image that is not one of the n types of sites, becomes the highest confidence level.

[0060] <Step S140> K = K + 1. 1 is added to the count K.

[0061] <Step S150> K ≥ TK1 If the count K is not greater than or equal to the first judgment value TK1 (NO), the lesion detection process in step S100 is performed using the second model that is running. Note that the first judgment value TK1 may be set according to the site or organ.

[0062] <Step S160> Third frame rate In step S150, if the count K is greater than or equal to the first determination value TK1 (YES), the inference interval setting unit 17 increases the frame rate X from the standard frame rate (second frame rate) to the third frame rate. The third frame rate is greater than the second frame rate and less than or equal to the first frame rate. Then, after the lesion detection process (step S110) and the display process (step S120) are performed, in step S20, the image extraction process for inference of the third frame rate is performed.

[0063] In this case, the third frame rate in the inference image extraction process is higher than the standard frame rate (second frame rate). As a result, the processor 2 has an improved chance of the site identification unit 14 being able to identify the observed site, and can stably identify the observed organ being observed with the endoscope. Also, since the standard frame rate is lower than the first frame rate, the load on the processor 2 is smaller, and the processor 2 can efficiently identify the observed organ.

[0064] Next, the operation method of processor 2 will be explained using Figures 4 and 5. The first judgment value TK1 is set to "3". The first frame rate is set to 120 fps, the second frame rate (standard frame rate) is set to 20 fps, and the third frame rate is set to 120 fps.

[0065] Figure 4 shows an example in which the site identification unit 14 is able to identify the observed site. The images input to the processor 2 per second are frames 1 to 120. Since the frame rate X is the second frame rate (20 fps), after the image in frame 1 is identified as a nasal cavity image, the site inference unit 13 does not perform inference processing on the images in frames 2 to 6. After the image in frame 7 is identified as a nasal cavity image, the site inference unit 13 does not perform inference processing on the images in frames 8 to 12. After the image in frame 13 is identified as a pharyngeal image, the site inference unit 13 does not perform inference processing on the images in frames 14 to 18.

[0066] Figure 5 shows an example including cases where the part identification unit 14 cannot identify the observation area.

[0067] Since the frame rate X at startup is the second frame rate (20 fps), after the image in frame 1 is identified as a nasal cavity image, the site inference unit 13 does not perform inference processing on the images in frames 2-6. When the image in frame 7 is identified as an inappropriate image, the count K becomes "1". However, since the count K is less than the first judgment value TK1, the site inference unit 13 does not perform inference processing on the images in frames 8-12. When the image in frame 13 is identified as an undeterminable image, the count K becomes "2". However, since the count K is less than the first judgment value TK1, the site inference unit 13 does not perform inference processing on the images in frames 14-18.

[0068] When the image in frame 19 is identified as an inappropriate image, the count K becomes "3". Since the count K is greater than or equal to the first judgment value TK1, the inference interval setting unit 17 increases the frame rate X from the second frame rate (20 fps) to the third frame rate (120 fps). For this reason, the region inference unit 13 performs inference processing on the image in frame 20. When the image in frame 20 is identified as an inappropriate image, the count K becomes "4". The inference interval setting unit 17 maintains the frame rate X at the third frame rate.

[0069] The region inference unit 13 performs inference processing on the image of frame 21. When the image of frame 21 is identified as an image of the pharynx, the processing from step S40 is performed as shown in Figure 2. Then, in step S90, the inference interval setting unit 17 returns the frame rate X to the second frame rate. In step S100, the count number K is initialized. The region inference unit 13 does not perform inference processing on the images of frames 22-26, but performs inference processing on the image of frame 27.

[0070] As described above, the operation method of the endoscopic image diagnostic support processor of this embodiment involves using a first model that has learned multiple images of multiple body parts as training data to infer the confidence level of each of the multiple endoscopic images at a predetermined frame rate that is an image of the respective body part of the multiple body parts, setting the predetermined frame rate for inference from the multiple endoscopic images input at a first frame rate, identifying the observation site based on predetermined conditions using the confidence level, identifying the observation organ that corresponds to the observation site, and if the observation site cannot be identified for a number of consecutive times equal to or greater than a first judgment value, increasing the predetermined frame rate from the second frame rate, which is the standard frame rate, to the third frame rate, and returning the predetermined frame rate to the second frame rate if the observation site can be identified.

[0071] The program for the endoscopic image diagnostic support processor according to an embodiment of the present invention uses a first model that has learned multiple images of multiple body parts as training data to infer the confidence level of each of the multiple endoscopic images at a predetermined frame rate that is an image of the respective body part of the multiple body parts, sets the predetermined frame rate from the multiple endoscopic images input at a first frame rate, identifies the observation site based on predetermined conditions using the confidence level, identifies the observation organ that corresponds to the observation site, and if the observation site cannot be identified for a number of consecutive times equal to or greater than a first judgment value, the predetermined frame rate is increased from a second frame rate, which is the standard frame rate, to a third frame rate, and if the observation site can be identified, the predetermined frame rate is returned to the second frame rate.

[0072] The non-temporary storage medium, the storage device 32, may store only a part of the program. Furthermore, the program may be distributed or provided via a communication network. Users can execute all or part of the processing, and thus perform the processing of the processor described above, by installing the program from the storage medium to their computer, or by downloading the program via the communication network and installing it to their computer.

[0073] <Modifications of the First Embodiment> The endoscopic image diagnostic support processors of the modified embodiments and embodiments described below are similar to the endoscopic image diagnostic support processor 2 of the first embodiment and have the same effects as the endoscopic image diagnostic support processor 2. For this reason, components with the same functions as the endoscopic image diagnostic support processor 2 are denoted by the same reference numerals as the endoscopic image diagnostic support processor 2, and their descriptions are omitted.

[0074] <Modification 1 of the First Embodiment> In the determination device 10A of the processor 2A of this modification 2, as shown in the flowchart of Figure 6, the site identification unit 14 identifies site X as the observation site when the confidence level PX of the site with the highest confidence level P is equal to or greater than the second determination value TP (step S45: YES). If the confidence level PX is less than the second determination value TP (step S45: NO), the observation site / observation organ is not identified, and in step S80, lesion detection processing is performed using the activated second model. The second determination value TP may be set according to the organ.

[0075] Processor 2A can identify the observed organ more efficiently than processor 2.

[0076] Furthermore, the site identification unit 14 may identify a site as an observation site if the site with the highest confidence level P is the same site for a predetermined number of consecutive times or more. Furthermore, the organ identification unit 15 may, after provisionally determining an organ corresponding to the observation site inferred by the site inference unit 13 as an observation organ, finalize it as an observation organ if an organ different from the organ corresponding to the currently running CAD is provisionally observed for a certain number of consecutive times or more.

[0077] In other words, the organ identification unit 15 may make a final decision to change the organ after it has provisionally determined an organ different from the one in the previous endoscopic image, and after the same organ has been provisionally determined as the organ to be observed a certain number of times in a row.

[0078] <Modification 2 of the First Embodiment> In the determination device 10B of the processor 2B of this modification 2, as shown in the flowchart of Figure 7, the part identification unit 14 identifies part X as the observation part when the confidence level PX of the part with the highest confidence level P is equal to or greater than the second determination value TP (step S45: YES). If the confidence level PX is less than the second determination value TP (step S45: NO), it is determined to be an undeterminable image, and in step S140, 1 is added to the count K.

[0079] Processor 2B can identify the observed organ more efficiently than processor 2.

[0080] <Second Embodiment> As shown in the flowchart of Figure 8, in the determination device 10C of the processor 2C of this embodiment, if the part identification unit 14 cannot identify the observation part for a number of times greater than the first determination value TK1 and greater than or equal to the third determination value TK3 (step S142: YES), the inference interval setting unit 17 changes a predetermined frame rate X to a fourth frame rate that is higher than the third frame rate (step S144). The fourth frame rate is greater than the third frame rate and less than or equal to the first frame rate.

[0081] In the case of (step S142: NO), in step S150 as well, if the count K is greater than or equal to the first determination value TK1, the inference interval setting unit 17 changes the predetermined frame rate X to the third frame rate.

[0082] Next, the operation method of processor 2C will be explained using Figure 9. The first judgment value TK1 is set to "2", and the third judgment value TK3 is set to "4". The first frame rate is set to 120 fps, the second frame rate (standard frame rate) is set to 20 fps, the third frame rate is set to 60 fps, and the fourth frame rate is set to 120 fps.

[0083] Since the frame rate X at startup is the second frame rate (20 fps), after the image in frame 1 is identified as a nasal cavity image, the site inference unit 13 does not perform inference processing on the images in frames 2-6. When the image in frame 7 is identified as an inappropriate image, the count K becomes "1". However, since the count K is less than the first judgment value TK1, the site inference unit 13 does not perform inference processing on the images in frames 8-12. When the image in frame 13 is identified as an undeterminable image, the count K becomes "2". Since the count K is greater than or equal to the first judgment value TK1 and less than or equal to the third judgment value TK3, the inference interval setting unit 17 increases the frame rate X from the second frame rate (20 fps) to the third frame rate (60 fps). Therefore, the site inference unit 13 performs inference processing on the image in frame 15.

[0084] When the image in frame 15 is identified as an undeterminable image, the count K becomes "3". Since the count K is less than the third determination value TK3, the inference interval setting unit 17 maintains the frame rate X at the third frame rate (60 fps). Therefore, the region inference unit 13 performs inference processing on the image in frame 17.

[0085] When the image in frame 17 is identified as an undeterminable image, the count K becomes "4". Since the count K is greater than or equal to the third determination value TK3, the inference interval setting unit 17 increases the frame rate X from the third frame rate (60 fps) to the fourth frame rate (120 fps). Therefore, the region inference unit 13 performs inference processing on the image in frame 18.

[0086] When the image in frame 18 is identified as an unclassifiable image, the count K becomes "5". The inference interval setting unit 17 maintains the frame rate X at the fourth frame rate.

[0087] The region inference unit 13 performs inference processing on the image of frame 19. When the image of frame 19 is identified as an image of the pharynx, the processing from step S40 is performed as shown in Figure 2. Then, in step S90, the inference interval setting unit 17 returns the frame rate X to the second frame rate. In step S100, the count K is initialized. Since the region inference unit 13 does not perform inference processing on the images of frames 20-24, the inference processing of the image of frame 25 is performed.

[0088] The third judgment value TK3 may be set according to the organ. Processor 2C can identify the observed organ more efficiently than processor 2.

[0089] The range of values ​​described above is not limited to that range and can be increased or decreased as appropriate. Furthermore, the present invention is not limited to the embodiments described above, and various changes and modifications can be made as long as they do not alter the essence of the invention.

[0090] 1... Endoscopy system 2, 2A-2C... Endoscopy image diagnostic support processor (processor) 9... Endoscope 10, 10A-10C... Observed organ determination device (determination device) 11... CPU 12... Image input unit 13... Site inference unit 14... Site identification unit 15... Organ identification unit 16... Memory 17... Inference interval setting unit 20... Lesion detection device 30... Display device 31... Notification unit 32... Storage device 40... Server

Claims

1. Endoscopic image diagnostic support processor comprising: a site inference unit that uses a first model trained on multiple images of multiple body parts as training data to infer the confidence level of each of multiple endoscopic images at a predetermined frame rate being an image of each of the multiple body parts; an inference interval setting unit that sets the predetermined frame rate at which the site inference unit performs inference from the multiple endoscopic images input at a first frame rate; a site identification unit that identifies an observation site based on predetermined conditions using the confidence level; and an organ identification unit that identifies an observation organ which is an organ corresponding to the observation site, wherein the inference interval setting unit raises the predetermined frame rate from a second frame rate, which is the standard frame rate, to a third frame rate if the site identification unit fails to identify the observation site for a number of consecutive times equal to or greater than a first judgment value, and returns the predetermined frame rate to the second frame rate if the site identification unit is able to identify the observation site.

2. The endoscope image diagnostic support processor according to claim 1, wherein the site identification unit identifies the site with the highest confidence level as the observation site.

3. The endoscope image diagnostic support processor according to claim 2, wherein the site identification unit identifies a site where the confidence level is equal to or greater than the second determination value as the observation site.

4. The endoscopic image diagnostic support processor according to claim 1, wherein the first determination value is set according to the site or organ.

5. The endoscopic image diagnostic support processor according to claim 3, wherein the second determination value is set according to the site.

6. The endoscopic image diagnostic support processor according to claim 1, further comprising a lesion detection device that infers lesions in an image using a plurality of second models that have been trained using a plurality of lesion images for a plurality of organs as training data, wherein the lesion detection device uses a second model from the plurality of second models that corresponds to the observed organ identified by the organ identification unit.

7. The endoscopic image diagnostic support processor according to claim 1, further comprising a notification unit that notifies the observed organ identified by the organ identification unit.

8. The endoscopic image diagnostic support processor according to claim 1, wherein if the site identification unit is unable to identify the observation site for a number of consecutive times equal to or greater than a third determination value exceeding the first determination value, the inference interval setting unit changes the predetermined frame rate to a fourth frame rate higher than the third frame rate.

9. The endoscopic image diagnostic support processor according to claim 8, wherein the third determination value is set according to the site or organ.

10. An operation method for an endoscopic image diagnostic support processor, comprising: using a first model trained on multiple images of multiple body parts as training data, inferring the confidence level of each of multiple endoscopic images at a predetermined frame rate being an image of each of the multiple body parts; setting the predetermined frame rate for inference from the multiple endoscopic images input at a first frame rate; identifying an observation site based on predetermined conditions using the confidence level; identifying an observation organ that corresponds to the observation site; raising the predetermined frame rate from a second frame rate, which is the standard frame rate, to a third frame rate if the observation site cannot be identified for a number of consecutive times equal to or greater than a first judgment value; and returning the predetermined frame rate to the second frame rate if the observation site can be identified.

11. A program for an endoscopic image diagnostic support processor that causes a computer to perform the following processes: using a first model trained on multiple images of multiple body parts as training data, inferring the confidence level of each of multiple endoscopic images at a predetermined frame rate being an image of each of the multiple body parts; setting the predetermined frame rate for inference from the multiple endoscopic images input at a first frame rate; identifying an observation site based on predetermined conditions using the confidence level; identifying an observation organ that corresponds to the observation site; raising the predetermined frame rate from a second frame rate (standard frame rate) to a third frame rate if the observation site cannot be identified for a number of consecutive times equal to or greater than a first judgment value; and returning the predetermined frame rate to the second frame rate if the observation site can be identified.

Images