Inspection support device, inspection support method, and inspection support program

Abstract

To provide an inspection support device capable of preventing oversight of a lesion while permitting switching of display according to inspection circumstances.SOLUTION: An inspection support device is used by being connected to an endoscope system having a camera unit inserted into the body of a subject, and includes: an acquisition part for sequentially acquiring images captured with a camera unit; a diagnosis part for executing diagnosis of the body on the basis of the captured image; a display part for displaying a diagnosis result output by the diagnosis part; a reception part for receiving the selection on an on-mode for displaying the diagnosis result on the display part and an off-mode for not displaying the diagnosis result; and a control part for displaying, in the display part, the diagnosis result in real time when the selection of the on-mode is received by the reception part, and storing the diagnosis result in a storage part without displaying the diagnosis result in the display part at least while an inspection of the body with the camera unit is continued when the selection of the off-mode is received.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an inspection support device, an inspection support method, and an inspection support program. [Background technology]

[0002] There is known an endoscope system that uses an endoscope to capture, for example, the inside of a subject's stomach and displays the image on a monitor. Recently, examination support devices that analyze images captured by the endoscope system and notify doctors of the results are becoming more widespread (see, for example, Patent Document 1). In addition, a usage mode in which an examination support device is connected to an endoscope system and the examination support device analyzes in-body images captured by the endoscope system in real time is also approaching the practical stage. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-42156 A Summary of the Invention [Problem to be solved by the invention]

[0004] If an examination is performed while displaying the diagnosis results calculated by the examination support device on the display unit, the doctor can enjoy the advantage of being able to concentrate on observing a specific area according to the displayed diagnosis results. On the other hand, there are cases where the doctor wants to turn off the display of the diagnosis results, which causes a slight delay before the diagnosis results are displayed, for reasons such as wanting to shorten the examination time required to insert the endoscope into the subject's body. However, if the diagnosis function is completely turned off, the risk of overlooking a lesion increases.

[0005] The present invention has been made to solve such problems, and provides an examination support device and the like that prevents lesions from being overlooked while allowing the display to be switched according to the circumstances of the examination. [Means for solving the problem]

[0006] An examination support device in a first aspect of the present invention is an examination support device that is used by being connected to an endoscopic system having a camera unit that is inserted into the body of a subject, and includes an acquisition unit that sequentially acquires images captured by the camera unit, a diagnosis unit that performs a diagnosis of the inside of the body based on the captured images, a display unit that displays a diagnosis result output by the diagnosis unit, a reception unit that receives from a user a selection of an on mode in which the diagnosis result is displayed on the display unit and an off mode in which the diagnosis result is not displayed, and a control unit that, when the reception unit receives the selection of the on mode, causes the diagnosis result to be displayed on the display unit in real time, and, when the reception unit receives the selection of the off mode, causes the diagnosis result to be stored in a memory unit without being displayed on the display unit at least while examination of the inside of the body by the camera unit is continuing. Equipped with.

[0007] An examination support method in a second aspect of the present invention is a method of examination support using an examination support device that is connected to an endoscopic system having a camera unit that is inserted into the body of a subject, and includes a reception step of receiving from a user a selection between an on mode in which diagnostic results are displayed on a display unit and an off mode in which diagnostic results are not displayed, an acquisition step of sequentially acquiring images captured by the camera unit, a diagnosis step of performing an internal diagnosis based on the captured images, a display control step of displaying the diagnostic results on the display unit in real time if the selection of the on mode is accepted in the reception step, and a memory control step of storing the diagnostic results in a memory unit without displaying them on the display unit, at least while the internal examination by the camera unit is continuing, if the selection of the off mode is accepted in the reception step.

[0008] The examination assistance program in a third aspect of the present invention is an examination assistance program that controls an examination assistance device that is connected to an endoscopic system having a camera unit that is inserted into the body of a subject, and causes a computer to execute the following steps: a reception step of receiving from a user a selection between an on mode in which a diagnostic result is displayed on a display unit and an off mode in which a diagnostic result is not displayed; an acquisition step of sequentially acquiring images captured by the camera unit; a diagnosis step of performing an internal diagnosis based on the captured images; a display control step of displaying the diagnostic result on the display unit in real time when the selection of the on mode is received in the reception step; and a memory control step of storing the diagnostic result in a memory unit without displaying it on the display unit at least while the internal examination by the camera unit is continuing when the selection of the off mode is received in the reception step. Effect of the Invention

[0009] According to the present invention, it is possible to provide an examination support device and the like that prevents lesions from being overlooked while allowing display switching according to the circumstances of the examination. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing an endoscopic examination using an endoscope system and an examination support device according to an embodiment of the present invention. FIG. [Diagram 2] FIG. 2 is a hardware configuration diagram of the inspection support device. [Diagram 3] 5A to 5C are diagrams illustrating examples of display screens in each mode. [Figure 4] FIG. 13 is a diagram for explaining the process up to displaying the diagnosis result in the on mode. [Diagram 5] FIG. 13 is a diagram for explaining the process up to storing a diagnosis result in the background mode of the off mode. [Figure 6] FIG. 13 is a diagram showing an example of a display screen that displays diagnostic results stored during testing when the testing is completed. [Figure 7] FIG. 13 is a diagram showing another example of a display screen that displays diagnostic results stored during testing when the testing is completed. [Figure 8]FIG. 13 is a diagram showing an example of a mode setting screen. [Figure 9] FIG. 13 is a diagram showing an example of a setting screen in an on mode. [Figure 10] FIG. 13 is a diagram showing an example of a setting screen in an off mode. [Figure 11] FIG. 11 is a flow chart for explaining the processing procedure of the calculation processing unit, mainly for explaining the processing procedure when the ON mode is set. [Figure 12] FIG. 11 is a flow chart for explaining the processing procedure of the calculation processing unit, mainly for explaining the processing procedure when the off mode is set. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present invention will be described below through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems.

[0012] FIG. 1 is a diagram showing an endoscopic examination using an endoscope system 200 and an examination support device 100 according to this embodiment, and in particular, a diagram showing a state in which the examination support device 100 functions in an on mode described later. Both the endoscope system 200 and the examination support device 100 are installed in an examination space. The endoscope system 200 includes a camera unit 210, which is inserted from the oral cavity of a lying subject to an internal organ such as the stomach, as shown in the figure, and transmits an image signal of an image captured inside the organ to the system body. The insertion of the camera unit 210 into the organ and the image capturing operation are performed by a doctor.

[0013] The endoscope system 200 includes a system monitor 220 configured by, for example, a liquid crystal panel, and processes an image signal sent from the camera unit 210 to display a visible captured image 221 on the system monitor 220. The endoscope system 200 also displays examination information 222 including subject information, camera information of the camera unit 210, and the like on the system monitor 220.

[0014] The examination support device 100 is connected to the endoscope system 200 by a connection cable 250. The endoscope system 200 also transmits a display signal to be transmitted to the system monitor 220 to the examination support device 100 via the connection cable 250. That is, the display signal in this embodiment is an example of an image signal provided by the endoscope system 200 to an external device. The examination support device 100 includes a display monitor 120 constituted by, for example, a liquid crystal panel as a display unit, and extracts an image signal corresponding to a captured image 221 from the display signal transmitted from the endoscope system 200, and sequentially displays the extracted image signal on the display monitor 120 as a visible captured image 121.

[0015] When the examination support device 100 is set to the on mode, the doctor proceeds with the examination while visually checking the captured image 121 displayed in real time in response to the operation of the camera unit 210. The doctor operates the camera unit 210 so that the target area that the doctor wants the examination support device 100 to diagnose is displayed as the captured image 121.

[0016] In the on mode, when the examination support device 100 detects, for example, a diagnosis trigger by a doctor, it generates image data from the captured image 121 at that time to be input to an analytical neural network, which will be described later. Then, the diagnosis result generated from the output of the analytical neural network is converted into computer graphics and displayed on the display monitor 120 as a CG index 122. In the example shown in the figure, the diagnosis result indicates that the probability of being estimated as cancer is 75%. The region determined as such is indicated by a diagnosis region frame 121a superimposed on the captured image 121. The diagnosis result is displayed for a certain period of time, and then switched to a real-time captured image 121 captured by the camera unit 210.

[0017] 2 is a hardware configuration diagram of the test support device 100. The test support device 100 is mainly composed of an arithmetic processing unit 110, a display monitor 120, an input / output interface 130, an input device 140, and a storage unit 150. The arithmetic processing unit 110 is a processor (CPU: Central Processing Unit) that controls the test support device 100 and executes programs. The processor may be configured to cooperate with an arithmetic processing chip such as an ASIC (Application Specific Integrated Circuit) or a GPU (Graphics Processing Unit). The arithmetic processing unit 110 reads out the test support program stored in the storage unit 150 and executes various processes related to test support.

[0018] As described above, the display monitor 120 is, for example, a monitor equipped with a liquid crystal panel, and visually displays the captured image 121, the CG index 122, etc. The input / output interface 130 is a connection interface for transmitting and receiving information to and from an external device, including a connector for connecting a connection cable 250. The input / output interface 130 includes, for example, a LAN unit, and retrieves an inspection assistance program and update data for an analytical neural network 151 (described later) from an external device, and transfers them to the arithmetic processing unit 110.

[0019] The input device 140 is, for example, a keyboard, a mouse, or a touch panel overlaid on the display monitor 120, and a doctor or assistant can operate these to change the settings of the examination support device 100 or input information necessary for the examination.

[0020] The storage unit 150 is a non-volatile storage medium, and is configured, for example, by an HDD (Hard Disk Drive). The storage unit 150 may store various parameter values, functions, display element data, lookup tables, and the like used for control and calculation, in addition to programs for executing control and processing of the examination support device 100. The storage unit 150 particularly stores an analytical neural network 151. The analytical neural network 151 is a trained model that, when image data captured by the camera unit 210 is input, calculates an area in the image where a lesion is estimated to exist and an estimated probability that the area is estimated to be a lesion. The storage unit 150 may be configured with multiple hardware components, and, for example, a storage medium for storing a program and a storage medium for storing the analytical neural network 151 may be configured with separate hardware components.

[0021] The arithmetic processing unit 110 also plays a role as a functional arithmetic unit that executes various calculations according to the processing instructed by the examination assistance program. The arithmetic processing unit 110 can function as an acquisition unit 111, a diagnosis unit 112, a control unit 113, a reception unit 114, and a detection unit 115. The acquisition unit 111 sequentially acquires the display signals sent from the endoscope system 200 and develops each of them into a signal reproduction image. Furthermore, the acquisition unit 111 determines an image area representing a captured image captured by the camera unit 210 by searching for a change area where the difference amount between the preceding and succeeding signal reproduction images is equal to or greater than a reference amount. Once the acquisition unit 111 determines the image area, it sequentially generates a captured image 121 as a frame image from the image area of ​​the display signal that is sequentially acquired thereafter.

[0022] The diagnosis unit 112 inputs the captured image 121 as a diagnostic image to the analytical neural network 151 read from the storage unit 150, and causes the neural network 151 to output areas in the image that are suspected of being lesions together with their estimated probabilities. The diagnosis unit 112 generates a diagnosis result by associating this output with the input image, etc.

[0023] The control unit 113 generates a display signal for the display screen to control the display on the display monitor 120, controls the reading and writing of data from and to the storage unit 150, and controls the transmission and reception of various control signals to and from external devices via the input / output interface 130. In particular, when the on mode is executed, the diagnosis results generated by the diagnosis unit 112 are displayed in real time on the display monitor 120. When the background mode of the off mode is executed, the diagnosis results generated by the diagnosis unit 112 are stored in the storage unit 150 without being displayed on the display monitor 120 at least while the examination of the subject's body is continuing.

[0024] The reception unit 114 receives a selection between the on mode and the off mode from the doctor, who is the user, via the input device 140. The detection unit 115 detects that the examination in which the camera unit 210 is inserted into the body of the subject has ended.

[0025] In the conventional examination support device, the images captured by the camera unit inserted into the body of the subject are sequentially captured, and when a diagnostic trigger by a doctor is detected, a diagnosis is made based on the captured images at that time, and the results are immediately displayed. That is, the doctor generally visually checks the diagnostic results in real time in accordance with the operation of the camera unit. On the other hand, depending on the disease state of the subject and the nature of the examination, there is a demand to shorten the examination time for inserting the camera unit into the body of the subject as much as possible. Specifically, the subject may be elderly, have a chronic illness, or take medication. In such cases, the time for stopping the camera unit to give a diagnostic trigger or the time for calculation processing to calculate the estimated probability of a lesion may not be acceptable. In such cases, the examination support device was turned off.

[0026] However, if the examination support device is turned off, it cannot enjoy its original function of assisting the doctor in diagnosis, and the risk of overlooking a lesion increases. Also, in small and medium-sized clinics, biopsies are often not possible, and the demand to actively utilize the examination support device cannot be met. Or, a doctor may make a simple mistake by turning it off during the previous examination and forgetting to turn it on during the next examination.

[0027] Therefore, the test support device 100 in this embodiment employs two off modes in addition to the conventional on mode. The two off modes are a background mode and a completely off mode, and when a doctor selects the off mode, the initial setting is to switch to the background mode.

[0028] The on mode is a mode in which the diagnostic result is displayed in real time. The background mode among the off modes is a mode in which the diagnostic processing by the diagnostic unit 112 is executed, but the diagnostic result is not displayed while the internal examination continues, and is stored in the storage unit 150. The completely off mode among the off modes is a mode in which the diagnostic processing by the diagnostic unit 112 is not executed. In this embodiment, displaying in real time means allowing a delay of the processing time required for image processing and diagnostic processing with respect to real time, and it is sufficient to perform processing continuously and display immediately. For example, a case in which a progress bar is displayed during diagnostic processing to indicate the progress of processing, and the diagnostic result is displayed immediately after the processing is completed, is also expressed as real-time display.

[0029] Fig. 3 shows examples of display screens in each mode. Specifically, all of these are examples of display screens during an examination in which the camera unit 210 is inserted into the body of a subject. Fig. 3(A) is an example of a display screen when the on mode is being executed. When the on mode is being executed, there are mainly two display states: a standby display state in which captured images captured by the camera unit 210 are displayed in real time, and a result display state in which a diagnosis result obtained from a specific captured image is displayed.

[0030] The left diagram in FIG. 3(A) shows an example of a standby display state. In the standby display state, a captured image 121 that is updated sequentially is arranged on the left, and a standby indicator 122a that indicates a diagnosis standby state is arranged on the right. The captured image 121 that is updated sequentially is a so-called live display that displays an image captured by the camera unit 210 with almost no delay. The standby indicator 122a is a computer graphic including the text "ON" indicating the on mode and "AI Ready" indicating the diagnosis standby state, and is one aspect of the CG indicator 122. Such a display aspect in the on mode is called an on mode live display.

[0031] The right diagram of FIG. 3(A) shows an example of a result display state. In the result display state, the captured image 121 used as the diagnostic image is arranged on the left, and a result index 122b indicating the diagnosis result is arranged on the right. The result index 122b is a computer graphic including a numerical value of an estimated probability that it is estimated to be a lesion ("85%" in the illustrated example) and a text of the type of lesion ("Cancer" in the illustrated example), and is one aspect of the CG index 122. In addition, a diagnostic region frame 121a is superimposed on the captured image 121 so that the region estimated to be a lesion can be seen. The result display state continues for a certain period of time (for example, 2 seconds), and then returns to the display of the diagnosis standby state.

[0032] FIG. 3B is an example of a display screen when the background mode of the off mode is being executed. In the background mode, the captured image 121 that is updated sequentially is arranged on the left side, and the first off indicator 122c that indicates that the mode is both off mode and background mode is arranged on the right side. The captured image 121 that is updated sequentially is a so-called live display that displays the captured image captured by the camera unit 210 with almost no delay, similar to the on mode live display. The first off indicator 122c is a computer graphic including the text "OFF" indicating that the mode is off mode and "Background Processing" indicating that diagnostic processing is being performed in the background, and is one aspect of the CG indicator 122. In the background mode, the captured image 121 that is updated sequentially is displayed, and the entire display area is shaded in gray so that the mode can be recognized at a glance as being both off mode and background mode. Such a display aspect in the off mode is called off mode live display.

[0033] 3C is an example of a display screen when the completely off mode is being executed among the off modes. In the completely off mode, a second off indicator 122d indicating the completely off mode is positioned on the right side. The second off indicator 122d is a computer graphic including only the text "OFF" indicating the off mode, and is one aspect of the CG indicator 122. In the completely off mode, the captured image 121, which is updated sequentially, is not displayed, and the entire display area is shaded in gray, so that the off mode and the completely off mode can be recognized at a glance.

[0034] 4 is a diagram for explaining the process up to displaying the diagnosis result in the on mode. A signal playback image 225 acquired by the acquisition unit 111 based on the display signal sent from the endoscope system 200 and displayed is the same as the display image displayed on the system monitor 220 of the endoscope system 200. The signal playback image 225 includes a captured image 221 and examination information 222. The examination information 222 is, for example, text information.

[0035] When it is determined as described above which area of ​​the image area of ​​the signal reproduction image 225 is the image area representing the captured image, the acquisition unit 111 cuts out the determined image area and sets it as the captured image 121. The captured image 121 can be said to be an image obtained by reproducing the captured image captured by the camera unit 210 by the inspection support device 100.

[0036] When the extracted captured image 121 is not an image to be diagnosed, the acquiring unit 111 passes the captured image 121 to the control unit 113. In order to realize the display screen in the standby display state described with reference to FIG. 3(A), the control unit 113 sequentially updates the captured image 121 and displays it on the display monitor 120. When the extracted captured image 121 is an image to be diagnosed, the acquiring unit 111 passes the captured image 121 to the control unit 113 and also to the diagnosis unit 112.

[0037] The diagnostic unit 112 adjusts the captured image 121 according to the specifications of the analytical neural network 151 and generates a diagnostic image. In particular, in the on mode, in order to reduce the amount of calculation of the analytical neural network 151, the diagnostic unit 112 generates a diagnostic image in which the pixels of the captured image 121 are thinned out to reduce the image size. Then, the diagnostic unit 112 inputs the image data of the generated diagnostic image to the analytical neural network 151 to calculate the estimated probability that a lesion exists in the diagnostic image. The analytical neural network 151 outputs a region suspected of a lesion existing in the image as a diagnostic result together with the estimated probability. In the example of FIG. 4, the upper left region showing an estimated probability of 60% and the region near the center showing an estimated probability of 85% are output. The diagnostic unit 112 passes the diagnostic result to the control unit 113.

[0038] The control unit 113 displays the captured image data of the captured image 121 received from the acquisition unit 111 and the diagnosis result received from the diagnosis unit 112 on the display monitor 120 by expanding and arranging them according to a preset display criterion. Specifically, the display is performed in the form of the result display state described with reference to FIG. 3(A). The display criterion may be, for example, a criterion that, when multiple lesions are detected in the diagnosis result, the one with the highest estimated probability is selected, or a criterion that the lesion detected in the area closer to the center of the diagnostic image is displayed preferentially. In addition, a display criterion that does not display the diagnosis result can be set when the generated diagnostic image is unclear and the reliability of the diagnosis result is doubtful, or when there is no diagnosis result with an estimated probability exceeding a preset threshold. The display criterion may be set appropriately according to the observation site, the purpose of the examination, and the like. In addition, when multiple lesions are detected, the result indicator 122b and the diagnostic region frame 121a may be displayed by switching them sequentially according to each lesion.

[0039] 5 is a diagram for explaining the process up to storing the diagnosis result in the background mode of the off mode. As in the on mode, the acquisition unit 111 acquires the display signal sent from the endoscope system 200 and extracts the captured image 121 from the played image 225 that is developed. The control unit 113 displays the captured image 121, which is updated sequentially, on the display monitor 120 in order to realize the display screen in the background mode described with reference to FIG. 3(B).

[0040] When the captured image 121 is an image to be diagnosed, the acquisition unit 111 transfers the captured image 121 to the diagnosis unit 112. The diagnosis unit 112 adjusts the captured image 121 to generate a diagnostic image according to the specifications of the analysis neural network 151. In the background mode, since it is not necessary to immediately display the diagnosis result, the image size may be adjusted to be larger than the image size in the on mode. That is, the thinning rate of the captured image 121 may be suppressed to generate a high-definition diagnostic image closer to the captured image 121. If the processing capacity of the calculation processing unit 110 allows, the captured image 121 may be used as a diagnostic image as it is without thinning. By using a high-definition diagnostic image, improvement in diagnostic accuracy can be expected.

[0041] The diagnostic unit 112 inputs the image data of the generated diagnostic image to the analytical neural network 151, which calculates the estimated probability that a lesion exists in the diagnostic image. The analytical neural network 151 outputs the areas in the image that are suspected of being a lesion, together with the estimated probability, as a diagnostic result. The diagnostic unit 112 passes the diagnostic result to the control unit 113.

[0042] The control unit 113 stores the captured image data of the captured image 121 received from the acquisition unit 111 and the diagnosis result received from the diagnosis unit 112 in the storage unit 150 according to a preset storage criterion. The storage criterion will be described later in detail. In the on mode, the captured image 121 from which the diagnostic image was generated was displayed as a still image for a certain period of time together with the diagnosis result, but in the background mode, the diagnosis result is not displayed, so the captured image 121 from which the diagnostic image was generated is not displayed as a still image.

[0043] In the background mode, when the detection unit 115 detects that the examination inside the body by the camera unit 210 has ended, the control unit 113 reads out the diagnosis result stored in the storage unit 150 and displays it on the display monitor 120. The detection unit 115 detects the timing when the camera unit 210 is removed from the oral cavity, for example, by capturing a sudden change in the brightness of the captured image 121. Alternatively, the end of the examination may be detected by reading the examination information 222 included in the signal reproduction image 225.

[0044] Fig. 6 shows an example of a display screen that displays the diagnostic results stored during the test at the end of the test. The upper diagram in Fig. 6 shows an example of a display screen that displays the stored diagnostic results as thumbnails, and the lower diagram shows an example of the display screen when a specific thumbnail image is selected.

[0045] The upper diagram of FIG. 6 is an example of display of a diagnosis result when a storage criterion is set to store when a lesion is detected. That is, the control unit 113 stores the diagnosis result received from the diagnosis unit 112 in the storage unit 150 if the diagnosis result includes lesion detection, and discards it if the diagnosis result does not include lesion detection. Then, when the detection unit 115 detects the end of the examination, the control unit 113 reads out the stored diagnosis result, classifies it into "malignant lesion" and "benign or other lesion", and displays it as thumbnails. Specifically, the image number 162 is superimposed on the thumbnail image 161 cut out so as to include a lesion area from the captured image 121 that is the source of the diagnosis and is stored in association with the diagnosis result. Then, if the diagnosis result is a malignant lesion, it is displayed under the title 123 of "malignant lesion", and if it is a benign or other lesion, it is displayed under the title 123 of "benign or other lesion".

[0046] When a doctor wishes to check a specific thumbnail image 161 in detail, the doctor selects the thumbnail image 161 using the input device 140. When the control unit 113 accepts the selection of the specific thumbnail image 161, the control unit 113 generates a high-definition image 160 corresponding to the thumbnail image 161 from the original captured image 121, and displays the high-definition image 160 on the display monitor 120 together with a diagnosis result 163, larger than the thumbnail image 161.

[0047] 7 is a diagram showing another example of a display screen that displays the diagnosis results stored during the examination at the end of the examination. Specifically, this is a display example when the storage criterion is set to store all the diagnosis results diagnosed by the diagnosis unit 112. Therefore, the display screen includes a classification of "no abnormal findings" in addition to "malignant lesions", "benign or other lesions" as shown in the figure. For diagnosis results in which no lesions were detected, the control unit 113 superimposes an image number 162 on a thumbnail image 161 of the captured image 121 that is the basis of the diagnosis, and displays them under the title 123 of "no abnormal findings".

[0048] A scroll indicator 127 is disposed at the bottom right of the display screen, and in the illustrated example, indicates that thumbnail images 161 of "No Abnormal Findings" continue downward. A doctor can scroll the display screen by selecting the scroll indicator 127 to check other thumbnail images 161. Note that, similar to the example of FIG. 6, when a doctor selects a specific thumbnail image 161, the control unit 113 displays a high-resolution image 160 corresponding to the thumbnail image 161 on the display monitor 120 together with a diagnosis result 163.

[0049] Although the examples of FIG. 6 and FIG. 7 are examples of displaying all of the diagnostic results stored in the storage unit 150, some of the diagnostic results stored in the storage unit 150 may be selected and displayed. That is, the storage criteria for storing diagnostic results in the storage unit 150 may be different from the selection criteria for displaying on the display monitor 120 at the end of the examination. For example, even if the storage criteria is set to store all of the diagnostic results diagnosed by the diagnosing unit 112, the selection criteria for displaying on the display monitor 120 may be set to limit the results to those with an estimated probability of being a lesion equal to or greater than a threshold. Even if the diagnostic results are not displayed on the display monitor 120 at the end of the examination, or even if the settings are such that none of the diagnostic results are displayed on the display monitor 120 at the end of the examination, the doctor can read out and check the diagnostic results stored in the storage unit 150 at any time after the examination.

[0050] Next, the mode setting operation performed by the doctor before starting the examination will be described with reference to several setting screens. The reception unit 114 receives the doctor's selection regarding the mode setting via the input device 140. FIG. 8 is a diagram showing an example of the mode setting screen. The mode setting screen has a title 123 of "Mode Setting", selection items 125 of "On Mode" and "Off Mode", and radio buttons 124 for exclusively selecting and designating each of the selection items 125. When the reception unit 114 receives the selection of "On Mode", the examination support device 100 operates in the on mode, and when the reception unit 114 receives the selection of "Off Mode", the examination support device 100 operates in the background mode of the off mode unless the selection of "completely off" described later is received.

[0051] 9 is a display screen to which the display transitions when "On Mode" is selected on the mode setting screen, and is a diagram showing an example of the setting screen in the on mode. In the setting screen in the on mode, the doctor can set the timing at which the diagnosis unit 112 executes the diagnosis process in the on mode.

[0052] In the illustrated example, under the title 123 of "On Mode Setting", "Trigger Designation" and "Periodic Observation" are each displayed selectably with a check box 126. "Trigger Designation" is a setting in which, for example, when a doctor continues to press a freeze button provided in the endoscope system, causing the captured image 121 to become a still image, and when the acquisition unit 111 detects such a still image period, the diagnosis unit 112 starts a diagnosis process using this as a trigger. In other words, this is a setting in which the diagnosis process starts when it is determined that a doctor has requested a diagnosis. Therefore, when "Trigger Designation" is selected, it is preferable that the diagnosis result is displayed uniformly regardless of its content.

[0053] "Periodic observation" is a process of continuously executing diagnostic processing according to a set period. For the setting of "periodic observation", input boxes 128 for "period" and "display criteria" are provided, which allow the user to set the period (seconds) for executing diagnostic processing and the threshold value (%) of the estimated probability for whether or not to display the diagnostic result. In the example shown in the figure, the diagnostic processing is executed every 3 seconds, and the diagnostic result is set to be displayed if there is a lesion site with an estimated probability of 40% or more in the diagnostic result. If the doctor selects both the "trigger designation" and "periodic observation" checkboxes 126, the doctor can execute diagnostic processing either by his own trigger or by the set period.

[0054] 10 is a display screen to which the display transitions when "Off Mode" is selected on the mode setting screen, and is a diagram showing an example of the setting screen in off mode. When "Off Mode" is selected on the mode setting screen, the display transitions first to the setting screen shown in the upper left. On the setting screen shown in the upper left, there are arranged a title 123 for "Off Mode Settings", a radio button 124 for selecting the complete off mode, a selection item 125 for the cycle and storage criteria when operating in background mode, and an input box 128.

[0055] When the "Completely Off" radio button 124 is selected, the display screen shown in the upper right diagram is displayed. In the completely off mode, the area related to the setting related to the background mode is displayed with a shaded area. In the completely off mode, the diagnosis process by the diagnosis unit 112 is not performed, and the diagnosis result cannot be reviewed after the examination is completed, so that the mode must be selected more consciously. Therefore, in the initial setting, when the off mode is selected, the background mode is set first, and the completely off mode is set to be selected at a layer deeper than the layer where the background mode is set. By adopting such a user interface, it is possible to prevent a doctor from inadvertently selecting the completely off mode.

[0056] Return to the display screen in the upper left diagram. In the background mode, the doctor can set the period for executing the diagnostic process by the diagnostic unit 112. For example, when "2" seconds is entered in the check box 126 as shown in the figure, the diagnostic unit 112 generates a diagnostic image every two seconds and executes the diagnostic process. In this way, the period in the background mode can be made different from the period when periodic observation is performed in the on mode. That is, the ratio at which the diagnostic unit 112 executes the diagnostic process for the captured image 121 acquired by the acquisition unit 111 can be made different. Since the real-time display of the diagnostic result in the on mode has the aspect of interrupting the progress of the examination, it can be said that it is convenient to suppress the frequency of the diagnostic process. On the other hand, in the background mode of the off mode, the progress of the examination is not interrupted, so the diagnostic process may be executed more frequently. In this way, if the ratio at which the diagnostic process is executed can be appropriately changed according to the circumstances of the examination, the convenience of the examination support device 100 is improved.

[0057] The doctor can also set a storage criterion. The storage criterion is a criterion set for at least one of the captured image and the diagnosis result, and the control unit 113 stores the diagnosis result in the storage unit 150 when the set storage criterion is satisfied. As the storage criterion, first, either all of the diagnosis results obtained by executing the diagnosis process are stored, or only diagnosis results showing an estimated probability equal to or greater than a threshold are stored. The value of the estimated probability threshold can be specified by a check box 128. If only diagnosis results with estimated probabilities equal to or greater than a threshold are stored, the work can be reduced at the stage when the doctor checks the diagnosis result after the examination. On the other hand, the estimated probability threshold set as the storage criterion can be set to a lower value than the estimated probability threshold set as the display criterion in the on mode. In the background mode, the examination results are not displayed in real time, so even if the estimated probability threshold is set to a low value, there is no risk of impeding the progress of the examination. In this way, even if the setting items are related to each other, the display criterion and the storage criterion may be different according to their respective properties.

[0058] When the scroll indicator 127 is selected on the display screen shown in the upper left, the display screen shown in the middle left is displayed, and the doctor can set the storage criteria for the captured image. In the example shown in the figure, "resolution designation" and "observation site designation" can be set. "Resolution designation" designates that the diagnostic process is executed only when the captured image 121 acquired by the acquisition unit 111 is a high-resolution image. When the "Yes" radio button 124 is selected, such a restriction is adopted, and when the "No" radio button 124 is selected, the diagnostic process is executed without restriction. The resolution is evaluated, for example, by the high-frequency components contained in the captured image 121, and it is determined to be "high-resolution" when the high-frequency components occupy a proportion equal to or greater than a preset threshold. If the captured image 121, which is the original image from which the diagnostic image is generated, is limited to a high-resolution image, the reliability of the estimated probability output by the analytical neural network 151 is also improved.

[0059] The storage criteria for the captured image 121 may be determined by determining whether the captured image 121 satisfies the storage criteria when the diagnostic result is stored in the storage unit 150, or may be determined by determining whether the captured image 121 satisfies the storage criteria when the acquisition unit 111 acquires the captured image 121. If the determination is made at the time the diagnostic result is stored in the storage unit 150, evaluation calculations such as resolution for the captured image can be omitted if it is determined not to store the captured image based on other storage criteria (e.g., a threshold value of an estimated probability). If the determination is made at the time the acquisition unit 111 acquires the captured image 121, diagnosis processing by the diagnosis unit 112 can be omitted.

[0060] "Specify Observation Site" specifies that diagnostic processing is to be performed only when the captured image 121 acquired by the acquisition unit 111 is an image of the specified observation site. In other words, the diagnostic site is limited to a specific organ. When the "Yes" radio button 124 is selected, such a restriction is adopted, and when the "No" radio button 124 is selected, diagnostic processing is performed without restriction. When "Yes" is selected, a window 129 for specifying the observation site appears, as shown in the middle right diagram.

[0061] The window 129 includes a title 123 of "Specify Observation Site", selection items 125 indicating the observation site, radio buttons 124 for exclusively selecting each selection item, and a scroll indicator 127. As the selection items 125, "Automatic", "Esophagus", "Stomach", and the like are prepared. "Automatic", for example, checks whether the prepared analytical neural network 151 is a learning model targeting a specific observation site, and if it is a learning model targeting a specific observation site, sets the observation site as the specified site. Alternatively, when an observation site is specified in the endoscope system 200, if the examination information 222 appearing in the signal reproduction image 225 contains text information related to the observation site, the text information is read and the observation site is set as the specified site. When a specific observation site such as "esophagus" or "stomach" is specified, the designation is followed. When an observation site is specified in the window 129, the display screen returns to the display screen shown in the middle left diagram, and the text of the specified observation site appears as "Specified site: stomach".

[0062] When the observation site is specified in this manner, the diagnosis unit 112 judges whether the captured image 121 delivered from the acquisition unit 111 is an image of the set observation site. Then, the control unit 113 stores the diagnosis result in the storage unit 150 only when the diagnosis unit 112 judges that the set observation site is an image. Note that the diagnosis unit 112 may judge whether the captured image 121 delivered from the acquisition unit 111 is an image of the set observation site prior to the diagnosis process, and may not perform the diagnosis process if it judges that the image is not the set observation site. When judging whether the captured image 121 delivered from the acquisition unit 111 is an image of the set observation site, the diagnosis unit 112 may use a site determination neural network prepared separately from the analysis neural network 151. For example, when the "stomach" is specified as the observation site, the diagnosis unit 112 judges the captured image 121 delivered from the acquisition unit 111 using a site determination neural network that outputs the probability that the input image is the stomach. Then, only when the probability of output from the part determination neural network is equal to or greater than a threshold value, the captured image is input to analysis neural network 151 and diagnostic processing is performed.

[0063] When the scroll indicator 127 is selected on the display screen in the middle left diagram, the display screen in the lower left diagram is displayed. The display screen in the lower left diagram is related to the list display at the end of the examination and the setting of the storage destination of the diagnosis result. "List display" is used to specify whether or not to display the diagnosis result in a list using thumbnail display at the end of the examination, as described with reference to Figs. 6 and 7. If the "Yes" radio button 124 is selected, the list display is executed, and if the "No" radio button 124 is selected, the list display is not executed. "Storage destination designation" is used to specify which storage medium the diagnosis result is to be stored in. In the above description, the diagnosis result is stored in the storage unit 150, which is an internal memory, but the storage destination does not have to be the storage unit 150. An external medium can also be specified. The external medium may be, for example, a cloud server connected to a network. If the check box 126 for the external medium is selected, a setting screen for setting an IP address may be displayed.

[0064] Next, an example of the processing procedure of the main processing executed by the arithmetic processing unit 110 will be described with reference to a flow diagram. Fig. 11 is a flow diagram for explaining the processing procedure of the arithmetic processing unit 110, and is a flow diagram for explaining the processing procedure mainly when the on mode is set. The flow starts when the test support device 100 is started and the test support program is executed.

[0065] In step S101, the reception unit 114 receives various settings from the doctor via the input device 140. The doctor performs settings appropriate for the purpose of the examination through the user interface described with reference to Figures 8 to 10. When the reception of the settings is completed, the process proceeds to step S102.

[0066] In step S102, control unit 113 checks whether the mode is set to the on mode or the off mode. If the mode is set to the on mode, the process proceeds to step S103, and if the mode is set to the off mode, the process proceeds to step S121 in Fig. 12. Here, the case where the mode is set to the on mode will be described first.

[0067] In step S103, the acquisition unit 111 acquires a display signal from the endoscope system 200 to generate a signal playback image 225, and further in step S104, extracts the captured image 121 from the signal playback image 225 and transfers it to the control unit 113. In step S105, the control unit 113 executes on-mode live display in which the captured image 121 is successively updated and displayed on the display monitor 120, as in the example shown in the left diagram of Fig. 3(A).

[0068] In step S106, the acquisition unit 111 judges whether the generated captured image 121 is a diagnostic target image. For example, when a trigger specification is set, the acquisition unit 111 judges whether the generated captured image 121 has been in a still image state for a certain period of time. If it is judged to be a diagnostic target image, the acquisition unit 111 passes the captured image 121 to the diagnosis unit 112 and proceeds to step S107, and if it is judged not to be a diagnostic target image, the acquisition unit 111 proceeds to step S110.

[0069] In step S107, the diagnosis unit 112 generates a diagnostic image from the captured image 121 delivered from the acquisition unit 111, and executes a diagnostic process. Specifically, the diagnosis unit 112 calculates a region suspected of a lesion and its estimated probability using the analytical neural network 151. Then, the control unit 113 delivers these diagnostic results to the control unit 113. In step S108, the control unit 113 determines whether the delivered diagnostic results match the display criteria set in step S101 or predefined.

[0070] For example, when the on mode is executed in periodic observation and the display criterion is specified as 40% or more, if the estimated probability included in the diagnosis result is 40% or more, it is determined that the display criterion is met, and if it is less than 40%, it is determined that the display criterion is not met. If it is determined that the display criterion is not met, the process proceeds to step S110 without displaying the diagnosis result. If it is determined that the display criterion is met, the process proceeds to step S109, where the control unit 113 displays the diagnosis result as shown in the right diagram of FIG. 3(A). Then, if the result display state continues for a certain period of time, the process proceeds to step S110.

[0071] In step S110, the detection unit 115 detects whether or not the internal examination by the camera unit 210 has ended. If the end has not been detected, the process returns to step S103 and the examination in the on mode is continued. If the end is detected, a specified end process is executed to end the series of processes.

[0072] FIG. 12 is a flow diagram for explaining the processing procedure of the arithmetic processing unit 110, mainly for explaining the processing procedure when the off mode is set, and is a flow following step S102 in FIG.

[0073] In step S121, the control unit 113 checks whether the mode is set to the background mode or the complete off mode. If the mode is set to the background mode, the process proceeds to step S122, and if the mode is set to the complete off mode, the process proceeds to step S131.

[0074] In step S122, the acquisition unit 111 acquires a display signal from the endoscope system 200 to generate a signal playback image 225, and further in step S123, extracts the captured image 121 from the signal playback image 225 and transfers it to the control unit 113. In step S124, the control unit 113 executes off-mode live display in which the captured image 121 is successively updated and displayed on the display monitor 120 while being shaded, as in the example of Fig. 3(B).

[0075] In step S106, the acquisition unit 111 judges whether the generated captured image 121 is a diagnostic target image. For example, if the cycle is set to 2 seconds, the acquisition unit 111 judges whether the generated captured image 121 matches that timing. If an observation site is specified, the acquisition unit 111 transfers the captured image 121 to the diagnosis unit 112, and entrusts it with the judgment of whether the site shown therein is the specified observation site. If it is determined that the captured image 121 is a diagnostic target image, the acquisition unit 111 transfers the captured image 121 to the diagnosis unit 112 and proceeds to step S126, and if it is determined that the captured image is not a diagnostic target image, the acquisition unit 111 proceeds to step S129.

[0076] In step S126, the diagnosis unit 112 generates a diagnostic image from the captured image 121 delivered from the acquisition unit 111, and executes a diagnostic process. Specifically, the diagnosis unit 112 calculates a region suspected of a lesion and its estimated probability using the analytical neural network 151. Then, the control unit 113 delivers these diagnostic results to the control unit 113. In step S127, the control unit 113 determines whether the delivered diagnostic results match the storage criteria set in step S101 or predefined.

[0077] For example, when the resolution designation is set to "Yes", if it is determined that the captured image 121 that is the original image of the diagnostic image is not high resolution, it is determined that it does not meet the storage criteria. If it is determined that it does not meet the storage criteria, the process proceeds to step S129 without storing the diagnostic result. If it is determined that it meets the storage criteria, the process proceeds to step S128, where the control unit 113 stores the diagnostic result in a storage unit such as the storage unit 150. When the storage process is completed, the process proceeds to step S129.

[0078] In step S129, the detection unit 115 detects whether or not the internal body inspection by the camera unit 210 has ended. If the end has not been detected, the process returns to step S122 and the inspection in the background mode continues. If the end is detected, the process proceeds to step S130.

[0079] When the process proceeds to step S130, the control unit 113 reads out the diagnosis result stored in the storage unit, and displays it on the display monitor 120 as in the example of Fig. 6 or Fig. 7. After that, when the reception unit 114 receives an instruction to end the display, a specified end process is executed to end the series of processes.

[0080] When the process proceeds from step S121 to step S131, the control unit 113 displays the completely off mode as shown in Fig. 3(C). While the completely off mode is displayed, the process proceeds periodically to step S131, and the detection unit 115 detects whether or not the internal examination by the camera unit 210 has ended. If the end has not been detected, the process returns to step S131, and the completely off mode display continues. If the end is detected, a specified end process is executed, and the series of processes ends.

[0081] Although the present embodiment has been described above, the user interface described here is an example, and is not limited to this as long as it is possible to accept various setting items and provide information. For example, voice input that recognizes the voice of the doctor who is the user, or augmented reality (AR) technology that superimposes information provided on wearable glasses may be adopted. In addition, the setting items described above are an example, and may be appropriately selected depending on the specifications of the examination device, etc.

[0082] As a modification of this embodiment, the following embodiment may also be adopted. That is, this is an embodiment in which the background mode is executed in parallel when the on mode in the above embodiment is executed. For example, for a diagnostic process executed when a diagnostic trigger by a doctor is detected, a diagnostic image in which many pixels of the captured image 121 are thinned out and the image size is reduced is used, in order to shorten the delay time until the diagnostic result is displayed as much as possible. On the other hand, for a periodic diagnostic process in which the diagnostic result is not displayed during the examination so that it can be confirmed after the examination is completed, a diagnostic image in which the pixels of the captured image 121 are not thinned out much and the image size is reduced is used, in order to prioritize ensuring diagnostic accuracy.

[0083] In this way, by executing in parallel the calculation process (first diagnostic process) for the diagnostic results to be displayed on the display monitor 120 in real time and the calculation process (second diagnostic process) for the diagnostic results to be stored in the storage unit during the examination without being displayed on the display unit, which requires a larger amount of calculation, the convenience of the examination support device can be improved. The configuration of such an examination support device can be summarized as follows.

[0084] [Alternative Configuration] An examination support device that is used by connecting to an endoscope system having a camera unit that is inserted into the body of a subject, an acquisition unit that sequentially acquires images captured by the camera unit; A diagnosis unit that performs a diagnosis inside the body based on the captured image; a display unit that displays a diagnosis result output by the diagnosis unit; Equipped with the diagnostic unit executes a first diagnostic process for displaying the diagnostic result on the display unit in real time during the examination of the inside of the body, and a second diagnostic process for presenting the diagnostic result after the examination of the inside of the body; The second diagnostic processing involves a larger amount of calculation than the first diagnostic processing.

[0085] Such an examination support device can meet both the demand to check the diagnosis result as soon as possible even during the examination and the demand to obtain a highly accurate diagnosis result. To obtain a highly accurate diagnosis result, the amount of calculation increases, and the processing time increases accordingly. However, as long as at least the captured image 121 to be diagnosed can be acquired, the diagnosis process can be completed since there is no restriction on real-time display. The process to obtain a highly accurate diagnosis result is not limited to the amount of thinning of the diagnostic image, and may be, for example, switching the neural network for analysis to be used. Specifically, a first neural network for a first diagnosis process that prioritizes calculation speed over accuracy and a second neural network for a second diagnosis process that prioritizes accuracy over calculation speed may be prepared.

[0086] In addition, even in such an embodiment according to the modified example, the above-mentioned storage criteria and display criteria can be set. In the above-mentioned embodiment, the settings are accepted on the premise of the "on mode" and "off mode", but it is possible to reconfigure so that the settings are accepted as settings for the "first diagnostic process" and the "second diagnostic process". In this case, for example, it is possible to accept the designation of the resolution and the observation part for the second diagnostic process.

[0087] In the above-described embodiment, it is assumed that the endoscope system 200 and the examination support device 100 are connected via the connection cable 250, but they may be connected wirelessly instead of wired. In addition, the embodiment in which the endoscope system 200 outputs a display signal to the outside and the examination support device 100 uses this display signal has been described, but as long as the image signal provided to the external device by the endoscope system 200 includes an image signal of an image captured by the camera unit 210, the format of the output signal does not matter. In addition, in the above-described embodiment, it is assumed that the camera unit 210 included in the endoscope system 200 is a flexible endoscope, but even if the camera unit 210 is a rigid endoscope, there is no difference in the configuration and processing procedure of the examination support device 100. Note that the diagnosis by the diagnosis unit in this embodiment is merely to assist the doctor's diagnosis, and the final decision is made by the doctor. [Explanation of symbols]

[0088] 100... examination support device, 110... calculation processing unit, 111... acquisition unit, 112... diagnosis unit, 113... control unit, 114... reception unit, 115... detection unit, 120... display monitor, 121... captured image, 121a... diagnosis area frame, 122... CG indicator, 122a... standby indicator, 122b... result indicator, 122c... first off indicator, 122d... second off indicator, 123... title, 124... radio button, 125... selection item, 126... check box, 127... scroll indicator, 128...input box, 129...window, 130...input / output interface, 140...input device, 150...storage unit, 151...analytical neural network, 160...high-resolution image, 161...thumbnail image, 162...image number, 163...diagnosis result, 200...endoscopic system, 210...camera unit, 220...system monitor, 221...captured image, 222...examination information, 225...signal replay image, 250...connection cable

Claims

1. An examination support device that is used by connecting to an endoscope system having a camera unit that is inserted into the body of a subject, an acquisition unit that sequentially acquires images captured by the camera unit; A diagnosis unit that performs a diagnosis inside the body based on the captured image; a display unit that displays a diagnosis result output by the diagnosis unit; a reception unit that receives, from a user, a selection between an on mode in which the diagnostic result is displayed on the display unit and an off mode in which the diagnostic result is not displayed; a control unit that, when the reception unit receives the selection of the on mode, causes the display unit to display the diagnosis result in real time, and, when the reception unit receives the selection of the off mode, continues the diagnosis by the diagnosis unit at least while the examination of the inside of the body by the camera unit continues, and stores the diagnosis result in a storage unit without displaying it on the display unit. Equipped with The diagnosis unit changes the proportion of the diagnosis performed on the captured image acquired by the acquisition unit depending on whether the acceptance unit accepts the selection of the on mode or the off mode.

2. An examination support device that is used by connecting to an endoscope system having a camera unit that is inserted into the body of a subject, an acquisition unit that sequentially acquires images captured by the camera unit; A diagnosis unit that performs a diagnosis inside the body based on the captured image; a display unit that displays a diagnosis result output by the diagnosis unit; a reception unit that receives, from a user, a selection between an on mode in which the diagnostic result is displayed on the display unit and an off mode in which the diagnostic result is not displayed; a control unit that, when the reception unit receives the selection of the on mode, causes the display unit to display the diagnosis result in real time, and, when the reception unit receives the selection of the off mode, continues the diagnosis by the diagnosis unit at least while the examination of the inside of the body by the camera unit continues, and stores the diagnosis result in a storage unit without displaying it on the display unit. Equipped with The diagnosis unit determines whether the captured image acquired by the acquisition unit is an image of a predetermined diagnosis site, When the control unit receives the selection of the off mode, and determines that the diagnosis unit has imaged the diagnosis area, the control unit stores the diagnosis result in the memory unit.

3. The examination support device according to claim 2 , wherein the diagnosis portion is set based on a learning model used by the diagnosis unit.

4. The examination support device according to claim 2 , wherein the diagnostic region is set based on character information included in an image area other than the captured image acquired by the acquisition unit from the endoscope system.

5. An examination support device that is used by connecting to an endoscope system having a camera unit that is inserted into the body of a subject, an acquisition unit that sequentially acquires images captured by the camera unit; A diagnosis unit that performs a diagnosis inside the body based on the captured image; a display unit that displays a diagnosis result output by the diagnosis unit; a reception unit that receives, from a user, a selection between an on mode in which the diagnostic result is displayed on the display unit and an off mode in which the diagnostic result is not displayed; a control unit that, when the reception unit receives the selection of the on mode, causes the display unit to display the diagnosis result in real time, and, when the reception unit receives the selection of the off mode, continues the diagnosis by the diagnosis unit at least while the examination of the inside of the body by the camera unit is continuing, and stores the diagnosis result in a storage unit without displaying it on the display unit; a detection unit that detects that the examination of the inside of the body by the camera unit has been completed; Equipped with When the control unit accepts the selection of the off mode, after the detection unit detects the end of the test, the control unit reads out the diagnostic results stored in the memory unit and displays them on the display unit.

6. The examination support device according to claim 5 , wherein the control unit controls the display unit to display only the diagnosis results in which the estimated probability of a lesion is equal to or greater than a threshold value.

7. An examination support device that is used by connecting to an endoscope system having a camera unit that is inserted into the body of a subject, an acquisition unit that sequentially acquires images captured by the camera unit; A diagnosis unit that performs a diagnosis inside the body based on the captured image; a display unit that displays a diagnosis result output by the diagnosis unit; a reception unit that receives, from a user, a selection between an on mode in which the diagnostic result is displayed on the display unit and an off mode in which the diagnostic result is not displayed; a control unit that, when the reception unit receives the selection of the on mode, causes the display unit to display the diagnosis result in real time, and, when the reception unit receives the selection of the off mode, continues the diagnosis by the diagnosis unit at least while the examination of the inside of the body by the camera unit continues, and stores the diagnosis result in a storage unit without displaying it on the display unit. Equipped with The reception unit receives the off mode by classifying the off mode into a background mode and a completely off mode, When the control unit receives a selection of the background mode from the reception unit, the control unit stores the diagnosis result in a memory unit without displaying it on the display unit, and when the control unit receives a selection of the completely off mode, the control unit does not cause the diagnosis unit to perform the diagnosis.

8. The test support device according to claim 7 , wherein the reception unit receives a selection of the completely off mode at a layer deeper than a background mode.

9. 9. The test support device according to claim 7, wherein the control unit displays on the display unit an indication that the mode is the off mode so that the background mode and the completely off mode can be visually distinguished from each other.

10. An examination support device that is used by connecting to an endoscope system having a camera unit that is inserted into the body of a subject, an acquisition unit that sequentially acquires images captured by the camera unit; A diagnosis unit that performs a diagnosis inside the body based on the captured image; a display unit that displays a diagnosis result output by the diagnosis unit; a reception unit that receives, from a user, a selection between an on mode in which the diagnostic result is displayed on the display unit and an off mode in which the diagnostic result is not displayed; a control unit that, when the reception unit receives the selection of the on mode, causes the captured image and the diagnosis result to be displayed on the display unit in real time, and, when the reception unit receives the selection of the off mode, continues the diagnosis by the diagnosis unit at least while the examination of the inside of the body by the camera unit is continuing, and causes the captured image to be displayed on the display unit in real time in a manner that is recognizable as being different from the on mode, while storing the diagnosis result in a storage unit without displaying it on the display unit. An inspection support device comprising:

11. The inspection support device according to claim 10, wherein when the control unit accepts the selection of the off mode, the control unit stores the diagnostic result in the memory unit if a preset storage criterion is satisfied for at least one of the captured image and the diagnostic result.

12. The examination support device according to claim 11 , wherein the storage criteria include an estimated probability that the diagnosis result is estimated to include a lesion being equal to or greater than a threshold.

13. The inspection support device according to claim 11 or 12, wherein the storage criteria include that the resolution of the captured image is equal to or greater than a threshold value.

14. The test support device according to claim 11 , wherein the reception unit receives a setting of the storage criterion from the user.

15. The inspection support device according to any one of claims 11 to 14, wherein when the reception unit receives the selection of the on mode, the control unit causes the display unit to display the diagnostic result together with the captured image used by the diagnosis unit for the diagnosis for a predetermined period of time if a display criterion set in advance for at least one of the captured image and the diagnostic result is satisfied.

16. The inspection support device according to claim 15 , wherein the storage criteria and the display criteria are different from each other.

17. 1. An examination support method using an examination support device that is connected to an endoscope system having a camera unit that is inserted into a body of a subject, a receiving step in which a receiving unit of the test support device receives from a user a selection between an on mode in which a diagnostic result is displayed on a display unit and an off mode in which a diagnostic result is not displayed; an acquisition step in which an acquisition unit of the inspection support device sequentially acquires images captured by the camera unit; a diagnosis step in which a diagnosis unit of the examination support device performs a diagnosis of the inside of the body based on the captured image; a display control step of causing a control unit of the test support device to display the diagnosis result in real time on the display unit when the control unit of the test support device accepts the selection of the on mode in the accepting step; a storage control step of continuing the diagnosis in the diagnosis step at least while the examination of the inside of the body by the camera unit is continuing, when the selection of the off mode is accepted in the accepting step, and causing the control unit to store the diagnosis result in a storage unit without displaying it on the display unit; having The diagnosis step is an examination support method in which the proportion of the diagnosis performed on the captured image acquired in the acquisition step is different depending on whether the selection of the on mode is accepted in the acceptance step or the selection of the off mode is accepted.

18. 1. An examination support method using an examination support device that is connected to an endoscope system having a camera unit that is inserted into a body of a subject, a receiving step in which a receiving unit of the test support device receives from a user a selection between an on mode in which a diagnostic result is displayed on a display unit and an off mode in which a diagnostic result is not displayed; an acquisition step in which an acquisition unit of the inspection support device sequentially acquires images captured by the camera unit; a diagnosis step in which a diagnosis unit of the examination support device performs a diagnosis of the inside of the body based on the captured image; a display control step of causing a control unit of the test support device to display the diagnosis result in real time on the display unit when the control unit of the test support device accepts the selection of the on mode in the accepting step; a storage control step of continuing the diagnosis in the diagnosis step at least while the examination of the inside of the body by the camera unit is continuing, when the selection of the off mode is accepted in the accepting step, and causing the control unit to store the diagnosis result in a storage unit without displaying it on the display unit; having The diagnosis step includes determining whether the captured image acquired in the acquisition step is an image of a predetermined diagnostic region, The storage control step is an examination support method in which, when the selection of the off mode is accepted, if it is determined that the diagnostic area has been imaged in the diagnosis step, the diagnosis result is stored in the memory unit.

19. 1. An examination support method using an examination support device that is connected to an endoscope system having a camera unit that is inserted into a body of a subject, a receiving step in which a receiving unit of the test support device receives from a user a selection between an on mode in which a diagnostic result is displayed on a display unit and an off mode in which a diagnostic result is not displayed; an acquisition step in which an acquisition unit of the inspection support device sequentially acquires images captured by the camera unit; a diagnosis step in which a diagnosis unit of the examination support device performs a diagnosis of the inside of the body based on the captured image; a display control step of causing a control unit of the test support device to display the diagnosis result in real time on the display unit when the control unit of the test support device accepts the selection of the on mode in the accepting step; a storage control step of continuing the diagnosis in the diagnosis step at least while the examination of the inside of the body by the camera unit is continuing, when the selection of the off mode is accepted in the accepting step, and causing the control unit to store the diagnosis result in a storage unit without displaying it on the display unit; a detection step in which a detection unit of the examination support device detects that the examination inside the body by the camera unit has been completed; having The storage control step is an examination support method in which, when the selection of the off mode is accepted, after the detection step detects the end of the examination, the diagnostic results stored in the storage unit are read out and displayed on the display unit.

20. 1. An examination support method using an examination support device that is connected to an endoscope system having a camera unit that is inserted into a body of a subject, a receiving step in which a receiving unit of the test support device receives from a user a selection between an on mode in which a diagnostic result is displayed on a display unit and an off mode in which a diagnostic result is not displayed; an acquisition step in which an acquisition unit of the inspection support device sequentially acquires images captured by the camera unit; a diagnosis step in which a diagnosis unit of the examination support device performs a diagnosis of the inside of the body based on the captured image; a display control step of causing a control unit of the test support device to display the diagnosis result in real time on the display unit when the control unit of the test support device accepts the selection of the on mode in the accepting step; a storage control step of continuing the diagnosis in the diagnosis step at least while the examination of the inside of the body by the camera unit is continuing, when the selection of the off mode is accepted in the accepting step, and causing the control unit to store the diagnosis result in a storage unit without displaying it on the display unit; having The receiving step receives the off mode by classifying the off mode into a background mode and a completely off mode, The storage control step, when the selection of the background mode is accepted in the accepting step, stores the diagnosis result in a storage unit without displaying the diagnosis result on the display unit, The examination support method, in which the diagnosis step does not perform the diagnosis when the selection of the completely off mode is accepted in the acceptance step.

21. 1. An examination support method using an examination support device that is connected to an endoscope system having a camera unit that is inserted into a body of a subject, a receiving step in which a receiving unit of the test support device receives from a user a selection between an on mode in which a diagnostic result is displayed on a display unit and an off mode in which a diagnostic result is not displayed; an acquisition step in which an acquisition unit of the inspection support device sequentially acquires images captured by the camera unit; a diagnosis step in which a diagnosis unit of the examination support device performs a diagnosis of the inside of the body based on the captured image; a display control step of causing a control unit of the test support device to display the diagnosis result in real time on the display unit when the control unit of the test support device accepts the selection of the on mode in the accepting step; a storage control step of, when the selection of the off mode is accepted in the accepting step, continuing the diagnosis in the diagnosing step at least while the examination of the inside of the body by the camera unit is continuing, and causing the control unit to display the captured image on the display unit in real time in a manner that can be recognized as being different from the on mode, while storing the diagnosis result in a storage unit without displaying it on the display unit; The inspection support method includes the steps of:

22. An examination assistance program for controlling an examination assistance device that is used in connection with an endoscope system having a camera unit that is inserted into a subject's body, a receiving step of receiving from a user a selection between an on mode in which the diagnostic result is displayed on a display unit and an off mode in which the diagnostic result is not displayed; an acquisition step of sequentially acquiring images captured by the camera unit; a diagnosis step of diagnosing the inside of the body based on the captured image; a display control step of displaying the diagnosis result on the display unit in real time when the selection of the on mode is accepted in the accepting step; a storage control step of continuing the diagnosis by the diagnosis step at least while the examination of the inside of the body by the camera unit is continuing, when the selection of the off mode is accepted in the accepting step, and storing the diagnosis result in a storage unit without displaying it on the display unit; on the computer, The diagnosis step is an inspection support program that differs the proportion of the diagnosis performed on the captured image acquired in the acquisition step depending on whether the selection of the on mode is accepted in the acceptance step or the selection of the off mode is accepted.

23. An examination assistance program for controlling an examination assistance device that is used in connection with an endoscope system having a camera unit that is inserted into a subject's body, a receiving step of receiving from a user a selection between an on mode in which the diagnostic result is displayed on a display unit and an off mode in which the diagnostic result is not displayed; an acquisition step of sequentially acquiring images captured by the camera unit; a diagnosis step of diagnosing the inside of the body based on the captured image; a display control step of displaying the diagnosis result on the display unit in real time when the selection of the on mode is accepted in the accepting step; a storage control step of continuing the diagnosis by the diagnosis step at least while the examination of the inside of the body by the camera unit is continuing, when the selection of the off mode is accepted in the accepting step, and storing the diagnosis result in a storage unit without displaying it on the display unit; on the computer, The diagnosis step includes determining whether the captured image acquired in the acquisition step is an image of a predetermined diagnostic region, The memory control step is an examination support program that, when the selection of the off mode is accepted, stores the diagnosis result in the memory unit if it is determined that the diagnosis part has been imaged in the diagnosis step.

24. An examination assistance program for controlling an examination assistance device that is used in connection with an endoscope system having a camera unit that is inserted into a subject's body, a receiving step of receiving from a user a selection between an on mode in which the diagnostic result is displayed on a display unit and an off mode in which the diagnostic result is not displayed; an acquisition step of sequentially acquiring images captured by the camera unit; a diagnosis step of diagnosing the inside of the body based on the captured image; a display control step of displaying the diagnosis result on the display unit in real time when the selection of the on mode is accepted in the accepting step; a storage control step of, when the selection of the off mode is accepted in the accepting step, continuing the diagnosis in the diagnosing step at least while the examination of the inside of the body by the camera unit is continuing, and storing the diagnosis result in a storage unit without displaying it on the display unit; a detection step of detecting that the examination of the inside of the body by the camera unit has been completed; on the computer, The memory control step is an examination support program that, when the selection of the off mode is accepted, reads out the diagnostic results stored in the memory unit and displays them on the display unit after detecting the end of the examination in the detection step.

25. An examination assistance program for controlling an examination assistance device that is used in connection with an endoscope system having a camera unit that is inserted into a subject's body, a receiving step of receiving from a user a selection between an on mode in which the diagnostic result is displayed on a display unit and an off mode in which the diagnostic result is not displayed; an acquisition step of sequentially acquiring images captured by the camera unit; a diagnosis step of diagnosing the inside of the body based on the captured image; a display control step of displaying the diagnosis result on the display unit in real time when the selection of the on mode is accepted in the accepting step; a storage control step of continuing the diagnosis by the diagnosis step at least while the examination of the inside of the body by the camera unit is continuing, when the selection of the off mode is accepted in the accepting step, and storing the diagnosis result in a storage unit without displaying it on the display unit; on the computer, The receiving step receives the off mode by classifying the off mode into a background mode and a completely off mode, The storage control step, when the selection of the background mode is accepted in the accepting step, stores the diagnosis result in a storage unit without displaying the diagnosis result on the display unit, The diagnostic step is an inspection support program that does not perform the diagnosis when the selection of the completely off mode is accepted in the acceptance step.

26. An examination assistance program for controlling an examination assistance device that is used in connection with an endoscope system having a camera unit that is inserted into a subject's body, a receiving step of receiving from a user a selection between an on mode in which the diagnostic result is displayed on a display unit and an off mode in which the diagnostic result is not displayed; an acquisition step of sequentially acquiring images captured by the camera unit; a diagnosis step of diagnosing the inside of the body based on the captured image; a display control step of displaying the diagnosis result on the display unit in real time when the selection of the on mode is accepted in the accepting step; a storage control step of, when the selection of the off mode is accepted in the accepting step, continuing the diagnosis in the diagnosing step at least while the examination of the inside of the body by the camera unit is continuing, and displaying the captured images on the display unit in real time in a manner that is recognizable as being different from the on mode, while storing the diagnosis result in a storage unit without displaying it on the display unit; An inspection support program that causes a computer to execute the above.

Images