Inspection support device, inspection support method, and inspection support program

Abstract

To provide an inspection support device, etc. for appropriately diagnosing an object place in which a doctor, who is a user, has an interest of an image captured by an endoscope.SOLUTION: An inspection support device includes: an acquisition part for sequentially acquiring images captured by a camera unit inserted into the body of a subject; a diagnosis part for performing diagnosis of the body on the basis of a set partial region of the captured images; and a display control part for sequentially displaying, in a display part, the captured images with a region wider than the partial region as a display region in a mode that enables a user to recognize the partial region. Since the partial region is displayed in the display part like this, the user has only to operate the camera unit so that an object place to be diagnosed is placed within the region, thereby achieving both easiness of operation and certainty of diagnosis.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an inspection support device, an inspection support method, and an inspection support program. [Background technology]

[0002] An endoscope system that uses an endoscope to capture images of, for example, the inside of a subject's stomach and displays the images on a monitor. Recently, the images captured by the endoscope system have been analyzed. Examination support devices that inform doctors of the results of examinations are also becoming more widespread (see, for example, Patent Document 1). The endoscope system is connected to the examination support device, and the images taken by the endoscope system are displayed in near real time. The use of test support devices to perform analysis in real time is also approaching the practical stage. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-42156 A Summary of the Invention [Problem to be solved by the invention]

[0004] When the presence or absence of a lesion is diagnosed for the entire image area of ​​the acquired image, the examination support device However, due to multiple lesions, body fluid stains, and artifacts caused by lighting, etc., errors occur one after another. Such erroneous diagnosis results may be included in the output of a single image. However, if the diagnostic results for multiple locations are output and displayed, the displayed content becomes too complicated, making it difficult for the doctor performing the examination. On the other hand, if only diagnostic results with a high probability of pathology are displayed, Incorrect diagnosis results are displayed, and lesions that should be recognized are not displayed and are overlooked. Doctors can easily spot suspicious areas by observing real-time images. In such cases, it is often desirable to have the examination support device diagnose only that part. and adjust the angle of view of the camera unit so that only that part is captured in the entire image area. However, this may significantly impair workability or may be physically impossible.

[0005] The present invention has been made to solve such problems, and This is an examination support device that properly diagnoses the areas of interest to the doctor who uses the images. This provides facilities, etc. [Means for solving the problem]

[0006] The examination support device according to the first aspect of the present invention includes a camera unit inserted into the body of a subject. an acquisition unit that sequentially acquires the captured images captured by the camera, and a diagnostic unit for diagnosing the inside of the body using the partial area and a diagnostic unit for detecting the partial area in a manner that allows a user to recognize the partial area. The display control unit sequentially displays captured images on the display unit using a larger area as a display area.

[0007] The examination support method according to a second aspect of the present invention includes a test support device including an acquisition unit, a display control unit, and a diagnosis unit. An examination support method using an examination support device, comprising: An acquisition step of sequentially acquiring images captured by the unit; The set partial area is displayed as a display area in a manner that allows the user to recognize the set partial area. a display step of sequentially displaying the captured image on a display unit; and a diagnosis step of performing a diagnosis inside the body based on the result.

[0008] The examination assistance program according to a third aspect of the present invention includes an acquisition unit, a display control unit, and a diagnosis unit. The examination support program controls an examination support device that can obtain an image by inserting an acquisition unit into the body of a subject. an acquisition step of sequentially acquiring images captured by the camera unit; and a display control step of The part of the captured image is displayed on the display unit 100 in such a manner that the user can recognize the set partial area of ​​the captured image. a display step of sequentially displaying the captured image on a display unit using the area as a display area; and a diagnosis step of causing the computer to perform a diagnosis of the inside of the body based on the partial region of the image. To make.

[0009] The examination support device according to the fourth aspect of the present invention includes a camera unit inserted into the body of a subject. An acquisition unit that sequentially acquires captured images captured by the camera, and a method of acquiring captured images using a set partial region of the captured images. The captured image is sequentially displayed on the display unit in a manner that can be recognized by a user, with an area larger than the partial area as the display area. a display control unit that performs a diagnosis on the partial region based on the captured image and displays the diagnosis result on the partial region; and a diagnostic unit that outputs the diagnostic results for the other regions separately.

[0010] A method for assisting an examination according to a fifth aspect of the present invention includes a diagnostic device including an acquisition unit, a display control unit, and a diagnosis unit. An examination support method using an examination support device, comprising: An acquisition step of sequentially acquiring images captured by the unit; The set partial area is displayed as a display area in a manner that allows the user to recognize the set partial area. A display step of sequentially displaying the captured images on a display unit; and a diagnosis step of diagnosing the inside of the body based on the captured images. The diagnosis result for the partial region is output separately from the diagnosis result for the other regions. and a cutting step.

[0011] The examination assistance program according to a sixth aspect of the present invention includes an acquisition unit, a display control unit, and a diagnosis unit. The examination support program controls an examination support device that can obtain an image by inserting an acquisition unit into the body of a subject. an acquisition step of sequentially acquiring images captured by the camera unit; and a display control step of The part of the captured image is displayed on the display unit 100 so that the user can recognize the set partial area of ​​the captured image. A display step of sequentially displaying the captured image on the display unit as a display area; Based on the results, the diagnosis of the internal body is performed, and the diagnosis result of the partial region is compared with the diagnosis result of the other regions. and a diagnostic step of discriminating between the detected and detected errors and outputting the results. Effect of the Invention

[0012] According to the present invention, it is possible to select images captured by an endoscope that are of interest to a doctor who is a user of the endoscope. It is possible to provide an examination support device etc. that can properly diagnose the target area. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram showing an endoscopic examination using an endoscope system and an examination support device according to an embodiment of the present invention. FIG. [Diagram 2] FIG. 2 is a hardware configuration diagram of the inspection support device. [Diagram 3] 13A to 13C are diagrams showing the state of a display screen that changes in response to the operation of the camera unit by a doctor. [Figure 4] FIG. 13 is a diagram illustrating the process up to displaying a diagnosis result. [Diagram 5] 11A to 11C are diagrams illustrating a procedure for determining a diagnostic target image from a change in a captured image. [Figure 6] FIG. 4 is a flowchart illustrating a processing procedure of a calculation processing unit. [Figure 7] 13A to 13C are diagrams illustrating step-by-step diagnosis processing of a divided area in the first modified example. [Figure 8]FIG. 11 is a hardware configuration diagram of a test support device according to a second modified example. [Figure 9] 13 is an example of a user interface screen relating to setting of a diagnostic region in the second modified example. [Figure 10] FIG. 13 is a diagram for explaining the process up to displaying the diagnosis result in the third modified example. [Figure 11] FIG. 13 is a flowchart illustrating a processing procedure of a calculation processing unit in a third modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The present invention will be described below through the embodiments of the invention. The present invention is not limited to the following embodiments. In addition, all of the configurations described in the embodiments can solve the problems. It is not necessarily essential that the method be used to determine the

[0015] FIG. 1 shows an endoscope using an endoscope system 200 and an examination support device 100 according to this embodiment. FIG. 1 is a diagram showing an examination state. The endoscope system 200 includes a camera unit 210. As shown in the figure, the laser unit 210 is used to deliver blood from the oral cavity of a lying subject to an internal organ such as the stomach. The camera unit transmits image signals of the images taken inside the organ to the system body. The insertion of the catheter 210 into the organ and the imaging operation are performed by a doctor.

[0016] The endoscope system 200 includes a system monitor 220 configured, for example, by a liquid crystal panel. The image signal sent from the camera unit 210 is processed to obtain a visible captured image. 221 on the system monitor 220. The endoscope system 200 also The inspection information 222 including the camera information of the camera unit 210 is displayed on the system monitor 2 Display at 20.

[0017] The examination support device 100 is connected to the endoscope system 200 by a connection cable 250. The endoscope system 200 transmits a display signal to the system monitor 220 via a connection cable. The signal is also transmitted to the test support device 100 via the cable 250. The display signal is an example of an image signal that the endoscope system 200 provides to an external device. The support device 100 includes a display monitor 120 configured as a liquid crystal panel as a display unit. The image corresponding to the captured image 221 is obtained from a display signal sent from the endoscope system 200. The image signals are extracted and displayed successively on a display monitor 120 as a visible captured image 121.

[0018] The doctor can view the captured images displayed in almost real time in response to the operation of the camera unit 210. The inspection will be carried out while visually checking the 121. The details will be described later, but the display is almost in real time. A diagnostic region frame 121a is superimposed on the captured image 121 to be displayed, and the doctor can The camera unit 21 is positioned so that the target area to be diagnosed by the diagnostic device is within the diagnostic area frame 121a. Manipulate 0.

[0019] When the examination support device 100 detects a diagnosis instruction as described later, Image data of a partial region corresponding to the diagnosis region frame 121a is generated. The diagnosis is performed based on the data, and the result, diagnostic information 122, is displayed on the display monitor 120. Note that FIG. 1 shows the results of sequentially generating the display signal in a situation where a diagnostic instruction is not detected. The figure shows a state in which a captured image 121 is displayed in almost real time, and diagnostic information 12 2 indicates that the device is waiting for diagnosis.

[0020] FIG. 2 is a hardware configuration diagram of the test support device 100. The test support device 100 mainly includes A calculation processing unit 110, a display monitor 120, an input / output interface 130, an input device The calculation processing unit 110 is configured of the test support device 100. A processor (CPU: Central Processing Unit) that controls and executes programs. The processor is ASIC (Application Specific Integrated Circuit) or GPU. The configuration may be such that the processor cooperates with a processing chip such as a Graphics Processing Unit (GPS). The processing unit 110 reads out the examination support program stored in the storage unit 150 and executes the examination support. It performs various processes related to support.

[0021] As described above, the display monitor 120 is a monitor equipped with, for example, a liquid crystal panel, and displays a captured image. The image 121 and diagnostic information 122 are displayed in a visually recognizable manner. Including a connector for connecting a cable 250, for sending and receiving information to and from an external device The input / output interface 130 is a connection interface for connecting, for example, a LAN unit. Including, update data of the inspection support program and the analytical neural network 151 described later. from an external device and delivers it to the arithmetic processing unit 110.

[0022] The input device 140 may be, for example, a keyboard, a mouse, or a keyboard superimposed on the display monitor 120. The doctor or assistant operates the touch panel to change the settings of the examination support device 100. and enter the information required for the test.

[0023] The storage unit 150 is a non-volatile storage medium, for example, a hard disk drive (HDD). The storage unit 150 is configured as a program for controlling and processing the examination support device 100. In addition to the graph, various parameter values, functions, display element data, etc. used for control and calculation are also included. The storage unit 150 may store, in particular, an analytical neural network. The analytical neural network 151 stores the work 151. When the captured image data is input by 210, the probability that a lesion exists in the image is calculated. This is a trained model. Note that the storage unit 150 may be configured with multiple pieces of hardware. For example, a storage medium for storing a program and an analytical neural network 151 may be The storage medium for storing the information may be configured as separate hardware.

[0024] The calculation processing unit 110 executes various calculations according to the processing instructed by the inspection support program. The calculation processing unit 110 also serves as a functional calculation unit. The acquisition unit 111 can function as a display control unit 113. The display signals are acquired one by one and each is expanded into a signal reproduction image. By searching for a change area where the difference between the raw images is equal to or greater than a reference amount, the camera unit 2 The image area representing the captured image captured by the image capture unit 10 is determined. The acquisition unit 111 first acquires the image area. Once the image is confirmed, the captured image 121 is displayed from the image area for the display signal acquired thereafter. Generate.

[0025] The diagnosis unit 112 extracts a predetermined partial area from the captured image 121 and generates a diagnosis area image. The image is input to an analyzing neural network 151 read from a memory unit 150, and the lesion is detected. The display control unit 113 calculates the probability that the display monitor 1 The display signal of the display screen to be displayed on the display monitor 120 is generated and transmitted to the display monitor 120. The display control unit 113 controls the display on the display monitor 120. In this situation, a display in which a diagnostic region frame 121a is superimposed on a captured image 121 that is successively updated and displayed is displayed. The device generates an indication signal.

[0026] FIG. 3 is an example of a display screen of the display monitor 120, in which the doctor is 3(a) and 3(b) are diagrams showing how the image changes with the operation. In a situation where a disconnection instruction is not detected, The image 121 is displayed as a sequence of captured images 121. The captured image 121 is currently captured by the camera unit 210 inserted into the organ. The image is substantially identical to that shown in

[0027] The diagnosis unit 112 diagnoses a partial region set in the center of the captured image 121 as a diagnosis region. The display control unit 113 displays the diagnosis area so that the doctor who is the user can recognize the diagnosis area. A diagnostic region frame 121a is superimposed on the captured image 121 so as to surround the region, and is displayed on the display monitor 120. In this situation, the diagnostic information 122 is displayed in a standby state for accepting a diagnostic instruction. This indicates that the

[0028] FIG. 3(a) shows an observation tag indicating a suspected lesion in a captured image 121 being observed by a doctor. This is an example of the display screen when the observation target 900 is found. 0 is located in the periphery of the captured image 121. It occurs in corner 1.

[0029] When the diagnosis unit 112 extracts candidates for a lesion from the entire image area of ​​the captured image 121, There is a possibility that not only the target 900 but also the artifact 910 may be extracted. For example, The analytical neural network 151 also detects artifacts for the observation target 900. If we calculate the estimated probability for 910 that is greater than or equal to the threshold, the probability will be displayed for each of them. The display screen becomes cluttered and visibility is reduced. If the probability is displayed only for candidates for which the rate was calculated, the artifact 910 In addition, the number of candidates extracted may be incorrect. Although the supplement is definitely a candidate for a lesion, it is not a candidate that the doctor is interested in. If the probability of being a candidate lesion of interest is high, then the probability of being a candidate lesion of interest will not be displayed.

[0030] In addition, the captured image 121 includes elements other than organ tissue, such as artifacts 910. In some cases, the entire image area of ​​the captured image 121 is processed by the neural network 151 for analysis. It is expected that the calculation results will be incorrect if the input is The network 151 is a ground truth image with annotations indicating whether it is diseased or not. Since this is a trained model, it is difficult to apply it to images of organ tissues that have not been trained. However, it tends to output incorrect calculation results when a doctor selects a region of concern from the entire image area. When a target 900 is found, the target 900 is expanded to cover the entire image area. Adjusting the angle of view of the camera unit 210 so that the image can be captured by the camera unit 210 significantly deteriorates the workability. In addition, the relative positional relationship of the observation target 900 in the target organ may be lost. be.

[0031] Therefore, as described above, the diagnosis unit 112 selects a diagnosis image set in the center of the captured image 121. The display control unit 113 diagnoses only the diagnostic region, and draws a diagnostic region frame 1 so as to surround the diagnostic region. The doctor places an observation target on the periphery of the captured image 121 as shown in FIG. When the observation target 900 is found, as shown in FIG. 3(b), the observation target 900 is placed in the diagnostic region frame 12. Operate the camera unit 210 so that it fits into the camera unit 1a. This can be done quickly and easily because it only requires a shift of the observation target 210. When the area is within the diagnosis area frame 121a, a diagnosis instruction is issued to the diagnosing unit 112 to execute a diagnosis.

[0032] In this way, when the diagnostic region is set in the center, the doctor can easily see which part of the organ is to be examined. In addition, the center of the captured image 121 is an archi- The incidence of cases is low, and even if they do occur, doctors can easily diagnose the condition if they are aware of it. By shifting the timing of the cut-off instruction and adjusting the angle of the camera unit 210, In this embodiment, the display control unit 113 The diagnostic area frame 121a is displayed superimposed on the entire image area of ​​the captured image 121. If an area wider than the diagnostic area is to be displayed as the display area, the captured image 121 For example, if the entire image area of ​​the captured image 121 is rectangular, In this case, a part of the periphery may be cut out to adjust the display to a square or circle.

[0033] In this embodiment, the diagnosis region frame 121a is represented by a dotted line graphic as shown in the figure. However, the display format is not limited to this. For example, a colored frame or a blinking frame may be used. Any visual effect may be used. This is not limited to the case of overlaying a graphic frame. Any visual effect may be used as long as it allows the doctor to recognize the diagnostic area. For example, a visual field may be created by brightening a diagnostic area in the captured image 121 and relatively darkening other areas. Effects may also be employed.

[0034] FIG. 4 is a diagram for explaining the process from detection of a diagnosis instruction to displaying a diagnosis result. The acquisition unit 111 acquires a display signal sent from the endoscope system 200 and develops the signal The playback image 225 is a display image displayed on the system monitor 220 of the endoscope system 200. The signal replay image 225 includes the captured image 221 and the inspection information 222. The inspection information 222 is, for example, text information.

[0035] Which area of ​​the image area of ​​the signal reproduction image 225 represents the captured image is determined. When the image area is determined as described above, the acquisition unit 111 cuts out the determined image area. The captured image 121 is a photographed image captured by the camera unit 210. It can be said that the captured image is reproduced by the inspection support device 100.

[0036] The diagnostic unit 112 cuts out an image of a diagnostic region from the captured image 121 to generate a diagnostic image. Then, the image data of the generated diagnostic image is input to the neural network 151 for analysis. The estimated probability that a lesion exists in the image is calculated, and the result is passed to the display control unit 113. The display control unit 113 receives the captured image data of the captured image 121 from the acquisition unit 111. and the diagnostic information including the estimated probability received from the diagnostic unit 112 in a preset display mode. The images are expanded and arranged according to the above and displayed on the display monitor 120.

[0037] Specifically, for example, as shown in the lower diagram of FIG. 4, the captured image 121 is arranged on the right side, and The probabilistic information 122 is expressed as numerical information indicating the estimated probability, a pie chart-like graphic, or the like. Also, it is possible to visually confirm which region of the captured image 121 is the diagnostic region. The diagnosis region frame 121a is displayed overlapping each other so that the diagnosis region frame 121a can be seen. 21a is superimposed on the captured image 121 before diagnosis so that it can be recognized as a diagnosed area. It is preferable that the display form is different from the diagnostic region frame 121a in FIG.

[0038] As described above, the examination support device 100 according to this embodiment receives the image transmitted from the endoscope system 200. The acquisition unit 111 acquires the display signal. The diagnosis unit 112 sequentially generates a display signal from the display signal. The change in the captured image 121 is used to infer the doctor's diagnostic instructions.

[0039] FIG. 5 illustrates a procedure for determining whether a freeze has occurred based on a change in the captured image 121 and for determining an image to be diagnosed. The test support device 100 receives a display signal at a cycle of, for example, 60 fps. However, the acquisition unit 111 cuts out an image area from the signal reproduction image 225 as described above each time. Thus, the captured images 121 are generated in sequence. The sequentially generated frames can be treated as a single frame image Fr that can change with the time. Image Fr 1 , Fr 2 , Fr 3 , Fr 4 , Fr 5 , ...

[0040] The camera unit 210 continuously captures images and outputs image signals as frame images. The image signals are sequentially transmitted to the endoscope system 200. The acquisition unit 111 receives the image signals including the image signals. While receiving the display signal from the endoscope system 200, the captured image of the camera unit 210 The corresponding frame images Fr that change over time are generated in sequence. do.

[0041] The doctor carefully observed the captured image 121 displayed on the display monitor 120. When the camera unit 210 is not in use, the freeze button provided on the operation unit of the camera unit 210 is pressed down. The system 200 detects that the freeze button is pressed down and starts the camera at that timing. The image signal captured by the unit 210 is frozen. That is, the freeze button is pressed. From that moment on, a freeze will begin, displaying a series of still images.

[0042] The acquisition unit 111 acquires a still image while the freeze button of the camera unit 210 is pressed. The frame image Fr corresponding to each of the above is generated in sequence. This period is called the still image period. In the example of FIG. , frame image Fr 2 The video period is up to the frame image Fr 3 The still image period starts from A still image period continues for a while after that.

[0043] For example, the diagnosis unit 112 determines that the motion is still when the difference between the previous and next frame images falls below a threshold. The diagnosis unit 112 can detect that a still image period has started by detecting the start of the still image period. The diagnosis unit 112 recognizes that a diagnosis instruction has been given by a doctor. The frame image at the time point is set as the diagnostic target image IM.

[0044] Next, a series of processes from determining the image area for the signal playback image to completing the inspection The procedure will be described below. FIG. 6 is a flow diagram illustrating the processing procedure of the arithmetic processing unit 110. .

[0045] In step S101, the acquisition unit 111 acquires a display signal from the endoscope system 200. In step S102, the captured image 121 is cut out from the signal reproduction image obtained by expanding the display signal. The display control unit 113 transfers the acquired information to the display control unit 113 in step S103. The diagnostic region frame 121a is superimposed on the captured image 121 received from the The image is displayed on the display monitor 120 as shown below.

[0046] In step S104, the diagnosis unit 112 performs the process of determining whether the previous frame The image is compared with the current frame image to determine whether the current frame image is a diagnostic image. If it is determined that the image is not a diagnostic image, the process proceeds to step S108. If it is cut off, the process proceeds to step S105.

[0047] In step S105, the diagnosis unit 112 detects the captured image that is determined to be a diagnosis target image. Then, in step S106, the image of the extracted diagnostic area is The image is input to an analytical neural network 151 read from a memory unit 150, and the image is The estimated probability that a lesion exists in the display area is calculated, and the result is passed to the display control unit 113. In step S107, the control unit 113 transfers the calculation result received from the diagnosis unit 112 to the acquisition unit 114. The image is displayed on the display monitor 120 together with the captured image 121 received from the camera 11. After a certain period of time (eg, 3 seconds) has elapsed, the process proceeds to step S108.

[0048] When the process proceeds to step S108, the calculation processing unit 110 checks whether or not an instruction to end the test has been received. If the instruction to end the examination has not been received, the process returns to step S101. If an instruction to end the examination is received, the series of processes is ended.

[0049] Next, some modified examples of this embodiment will be described. FIG. 7 shows a first modified example. FIG. 1 is a diagram for explaining a step-by-step diagnosis process for a divided region. The diagnosis unit 112 diagnosed the presence or absence of a lesion only in the diagnosis region set in the center. Although the central area of ​​the area that the doctor was interested in is unlikely to be a lesion, There are often cases where there are areas that are highly likely to be lesions in the area. In the embodiment, the diagnosis unit 112 detects a lesion exceeding a preset criterion in the central diagnosis region. If no abnormality is found, the adjacent region adjacent to the central portion is diagnosed.

[0050] Specifically, in the captured image 121, an expanded diagnostic region is defined so as to surround the diagnostic region frame 121a. Then, the area surrounded by the diagnosis area frame 121a is set as the first priority area. The area surrounded by the extended diagnostic area frame 121b and excluding the first priority area is set as the second priority area. The first priority area corresponds to the diagnosis area in the above embodiment.

[0051] The diagnosis unit 112 first cuts out the first priority region from the captured image 121 and applies a neural network for analysis to the first priority region. The input to the network 151 is a probability that the estimated probability exceeds a reference value (e.g., 50%). Once calculated, the display control unit 113 displays the result as shown in the lower left diagram of FIG. The display screen may be arranged so that the first priority region is visually recognized as the diagnostic region, for example. A captured image 121 in which the front region is highlighted and diagnostic information 122 which is the diagnosis result are displayed in parallel. In this case, the observation target 9 adjusted by the doctor so as to fit within the diagnosis region frame 121a is 00 is included in the highlighted first priority area.

[0052] If the diagnosis result of the first priority area is equal to or lower than the reference value, the diagnosis unit 112 continues to the second priority area. A region is cut out of the captured image 121 and input to an analytical neural network 151 . As a result, if an estimated probability exceeding the reference value is calculated, the display control unit 113 displays the result as an example. For example, the display is as shown in the lower right of Figure 7. The display screen shows that the second priority area is the diagnosis area. For example, the captured image 121 in which the second priority area is highlighted so that the second priority area can be visually recognized is displayed. The resulting diagnostic information 122 is collated. In this case, some The observation target 900 that is not within the diagnostic region frame 121a is highlighted. In the second priority area, if the diagnostic result is below the standard value, For example, the display control unit 113 may skip displaying the diagnosis result or may display a probability of the diagnosis result exceeding the reference value. This indicates that no lesions were present.

[0053] In addition, the display control unit 113 may receive the following from the display signal when a diagnosis instruction is not detected: In addition to the diagnostic region frame 121a, an expanded image 121 is generated in real time. The diagnostic region frame 121b may also be displayed in a superimposed manner. By including the second priority area adjacent to the first priority area as a diagnostic target, doctors will gain new insights. It is also possible to provide a better image quality and to compensate for insufficient adjustment of the imaging position. By setting a second priority area adjacent to the first priority area, rather than the entire area of ​​1, This makes it possible to prevent erroneous results due to color artifacts and the like from being displayed.

[0054] Next, a second modified example will be described. FIG. 8 shows the inspection support device 1 in the second modified example. 1 is a diagram showing a hardware configuration of the inspection support device 100′. The hardware configuration of the inspection support device 100 according to the embodiment includes a modification unit 114. The other configurations are the same, so the description will be omitted.

[0055] The change unit 114 sets a partial region of the captured image 121 to be diagnosed by the diagnosis unit 112. In the above embodiment, the diagnosis area is set at the center of the captured image 121. However, depending on the organ or area to be observed and the structure of the camera unit 210, In some cases, it may be difficult to guide the observation target 900 to the center of the captured image 121. Therefore, the examination support device 100' in this modified example is provided with a change unit 114 to change the The system allows the nurse to predefine diagnostic areas.

[0056] FIG. 9 is a diagram showing a user interface screen for setting a diagnostic region in the second modified example. This is an example. The diagnostic area is set before the examination starts. The doctor selects the diagnostic area from the menu. When you select "Diagnosis area setting", a setting screen will open. The title 123 of "Settings" and a selection button 124 and a selection item 125 for each option are displayed. The options are "Standard", which fixes the diagnostic area to the center, "Select Observation Area", "Select Frame", and " A "direct selection" is provided, and the doctor can select one of them by pressing the selection button 124. In this example, four options are available. Two or more of these may be provided, and other options may be added. .

[0057] When the doctor selects the observation site selection button 124, the observation site selection window 12 6 appears. The observation area selection window 126 has the title "Observation Area Selection" 123 For each option, a selection button 124 and a selection item 125 are displayed. The names of organs used as a target, such as "stomach," "esophagus," and "duodenum," are provided, and further selection is made. A scroll indicator 127 is displayed to indicate that there are further options below. Either one can be selected by pressing the button 124, and then the decision button 128 is pressed. The selected item is set. The partial area as the diagnostic area is set to each observation part. The change unit 114 changes the area according to the doctor's selection and decision. In response to this, the display control unit 113 changes the setting of the partial region for the captured image 121. The superimposition position of the diagnostic region frame 121a to be examined is changed.

[0058] When the physician selects the Select Frame button 124, a Select Frame window 126 appears. The frame selection window 126 has a title 123 of "Frame Selection" and a selection button for each option. The options 124 and 125 are displayed. The options are based on the position relative to the captured image 121. There are options such as "right," "left," and "top" to indicate the position of the mouse, and there are other options below. The doctor can select the desired one by pressing the selection button 124. Then, when the decision button 128 is pressed, the selected item is set. The partial area as the diagnostic area is determined in advance according to the relative position indicated by each item. The change unit 114 changes the setting of the partial region according to the selection and decision of the doctor. In response to this, the display control unit 113 adjusts the superimposition position of the diagnosis region frame 121a on the captured image 121. Change the location.

[0059] When the doctor selects the direct selection button 124, a direct selection window appears. The Direct Selection window 126 appears with the title "Direct Selection". A sample image 123 and a sample image 121' corresponding to the captured image 121 are displayed. The user can directly set the diagnostic region by tracing the desired area on the sample image 121' with his / her fingertip. Specifically, the area is designated by a touch panel provided on the display monitor 120. The contact position of the fingertip is detected by the touch sensor 124, and a trace line 129 is drawn on the graphics display 125 according to the contact trajectory. When the trace line closes, the image is A partial area of ​​the entire area of ​​the image 121 is determined, and the area designation is completed. If the selected diagnostic region is too small or too large, the diagnostic unit 112 may not be able to make an appropriate diagnosis. In such a case, the change unit 114 changes the designated diagnostic region to a preset one. The display control unit 113 adjusts the size so that the image falls within the diagnosable range. The position where the diagnostic region frame 121a is superimposed on the captured image 121 is changed. When the size of the diagnostic region frame 121a is changed, the size of the diagnostic region frame 121a to be superimposed is also changed. The designation is not limited to a method of detecting the trajectory of a touching fingertip, and other methods may be used.

[0060] Next, a third modified example will be described. In both the first and second modified examples, the diagnosis unit 112 determines a part set in advance from the captured image 121. A diagnostic region image is generated by extracting the divided region, and the diagnostic region image is fed to an analytical neural network. That is, the diagnosis is performed only on a partial region of the captured image 121. However, in the third modified example, the entire area of ​​the captured image 121 is used as a diagnostic image. The result is input to the analytical neural network 151. FIG. 13A to 13C are diagrams illustrating the process up to displaying the diagnosis result.

[0061] The acquisition unit 111 acquires a display signal sent from the endoscope system 200 and determines The process of extracting the image area and creating the captured image 121 is the same as in the embodiment of FIG. The captured image 121 is essentially an image captured by the camera unit 210 and transmitted to the inspection support device. It can be said that the image is reproduced at 100%.

[0062] The diagnosis unit 112 uses the entire captured image 121 as a diagnostic image and performs analysis using a neural network. The neural network 151 calculates the estimated probability that a lesion exists in the image. The network 151 identifies suspicious areas in the image along with their estimated probability. In the example of Figure 10, the upper left area indicates an estimated probability of 60%, and the estimated The area near the center, which indicates a probability of 85%, is output. The diagnosis result output by the network 151 is compared with the diagnosis region frame 121a set in advance. The diagnosis result is distinguished from the diagnosis result for the area other than the diagnosis area frame 121a, and the display control is performed for the former. Hand it over to Division 113.

[0063] The display control unit 113 receives the captured image data of the captured image 121 from the acquisition unit 111. and the diagnostic information including the estimated probability received from the diagnostic unit 112 in a preset display mode. Therefore, the images are expanded, arranged, and displayed on the display monitor 120. The display control unit 113 is Similarly, the captured image 121 is arranged on the right side, and the diagnostic information 122 is arranged in the diagnostic area frame 121a. Numerical information showing the estimated probability of Also, the area of ​​the captured image 121 that is the diagnosis area can be visually confirmed. The diagnosis area frames 121a are displayed overlapping each other as shown in FIG.

[0064] The diagnosis unit 112 evaluates the diagnosis result output by the analytical neural network 151 as follows: A diagnosis result for a preset diagnosis region frame 121a and a region outside the diagnosis region frame 121a are The diagnosis result for the patient may be distinguished from the diagnosis result for the patient, and both may be delivered to the display control unit 113. The display control unit 113 performs diagnosis by adopting a diagnosis result for a preset diagnosis region frame 121a. The diagnostic unit 112 may generate a display of the information 122 for later review by the physician. The captured image 121 is also stored with the diagnosis result for the diagnostic region frame 121a. The diagnosis results for the areas other than the above may be stored separately in the storage unit 150.

[0065] FIG. 11 is a flowchart illustrating a processing procedure of the arithmetic processing unit in the third modified example. The same steps as those in the flow chart of Figure 6 are explained by using the same step numbers. Omitted.

[0066] If the diagnosis unit 112 determines in step S104 that the current frame image is an image to be diagnosed Then, the process proceeds to step S201. In step S201, the diagnosis unit 112 The whole captured image 121 that is judged to be a diagnostic image is subjected to an analysis using a neural network. The diagnosis unit 112 inputs the result of the test to the diagnostic unit 151, which calculates the estimated probability that a lesion exists in the image. The process proceeds to step S202, where the prediction of the diagnosis results output by the analytical neural network 151 is performed. The diagnostic result for the diagnostic region frame 121a set for the above purpose is extracted and handed over to the display control unit 113. vinegar.

[0067] When the process proceeds from step S202 to step S107, the display control unit 113 The diagnostic result received from the acquisition unit 111 is displayed on the monitor together with the captured image 121 received from the acquisition unit 111. Display to 120.

[0068] In this third modified example, the doctor also moves the target area of ​​interest into the diagnostic region frame. By moving the target area, it is possible to estimate the probability that the target area is a lesion. The first and second modifications may be combined with the third modification. In the modified example, when the diagnosis result of the set diagnosis area falls below a preset judgment criterion, In the third modification, the diagnosis is performed on the adjacent region adjacent to the diagnostic region. The unit 112 does not perform diagnosis in a stepwise manner, but performs diagnosis and diagnosis for the diagnosis region frame 121a. The diagnostic results for the area other than the cross-sectional area frame 121a are output separately. When the modification example 1 is combined with the third modification example, the display control unit 113 displays the second priority area When displaying the diagnosis result, the diagnosis unit 112 outputs the diagnosis area frame 121a. Among the diagnostic results, the diagnostic results corresponding to the second priority area are extracted and displayed. When displaying the diagnostic result corresponding to the second priority area, the threshold value (e.g., 50%) is exceeded. If an estimated probability is calculated and it is less than the standard value, it will not be displayed. .

[0069] When the second modification is combined with the third modification, the diagnosis unit 112 causes the modification unit 114 The diagnosis result is differentiated according to the position of the diagnosis region frame 121a changed by the For example, when "Right" is selected by the selection box, the diagnosis unit 112 performs the analysis of the neural network. The diagnosis result output by the network 151 is shown in the diagnosis region frame 121 set to the right of the center. a and the diagnosis results for the area other than the diagnosis area frame 121a are output separately. Moreover, the first and second modified examples may be combined into the third modified example.

[0070] Although the above describes each of the modified examples, the modified examples are not limited to these. For example, The settings can be accepted during the test, not just before the test starts. It is acceptable to use a real-time image 121 so that a doctor can directly specify the By changing the above, it is possible to deal with the artifacts that appear appropriately. The real-time captured image 121 is input to a neural network for part diagnosis. The current observation area is estimated by the above, and the part appropriate for the estimated observation area is You can also change the settings to regions. In this case, you can change the settings to match each recognized observation area. The partial regions may be associated in advance as described above. In response to such processing in 114, the superimposition position of the diagnostic region frame 121a on the captured image 121 is In this way, the partial area is automatically and sequentially changed according to the observation area. If you do this, it is a good idea to set a certain interval (for example, 1 minute) to determine whether or not the change is allowed. .

[0071] In the embodiment described above, the diagnosis unit 112 A diagnostic process is performed on the diagnostic region, and the display control unit 113 displays the results on the display monitor 120. However, the diagnostic process and result display are not performed until the doctor has finished operating the camera unit 210. The diagnostic region frame 1 may be included in at least the captured image 121 in almost real time. If the images 21a and 21b are displayed in an overlapping manner, the doctor can appropriately diagnose the target area using the endoscope system 200. The release button for recording the image can be pressed, and the image can be sent to the diagnostic unit 112 after the fact. You can find out the diagnosis results.

[0072] In the embodiment described above, the endoscope system 200 and the examination support device 10 0 is assumed to be connected via a connection cable 250, but it is a wireless connection rather than a wired connection. The endoscope system 200 may output a display signal to the outside and provide an examination support Although the device 100 has been described as utilizing this display signal, the endoscope system 200 may be configured to The image signal provided to the external device includes an image signal of an image captured by the camera unit 210. As long as it includes the above, the format of the output signal does not matter. It is assumed that the camera unit 210 included in the endoscope system 200 is a flexible endoscope. However, even if the camera unit 210 is a rigid endoscope, the examination support device 100 There is no difference in the configuration or processing procedure. It is merely intended to assist the doctor's diagnosis, and the final decision is made by the doctor. [Explanation of symbols]

[0073] 100, 100'... inspection support device, 110... calculation processing unit, 111... acquisition unit, 112... diagnosis 113: display control unit; 114: change unit; 120: display monitor; 121: captured image; 21'... sample image, 121a... diagnostic region frame, 121b... extended diagnostic region frame, 122... diagnostic Cut information, 123...title, 124...selection button, 125...selection item, 126...window 127... scroll indicator, 128... decision button, 129... trace line, 130... Input / output interface, 140... input device, 150... memory unit, 151... analysis menu A network, 200, an endoscope system, 210, a camera unit, 220, a system monitor, 221...captured image, 222...inspection information, 225...signal replay image, 250...connection Cable, 900...observation target, 910...artifact

Claims

1. An acquisition unit that sequentially acquires images captured by a camera unit inserted into the body of a subject. and, a diagnosis unit that performs a diagnosis of the inside of the body based on a set partial region of the captured image; A region larger than the partial region is displayed as a display region in such a manner that the user can recognize the partial region. a display control unit that sequentially displays the captured images on a display unit; An inspection support device comprising:

2. An acquisition unit that sequentially acquires images captured by a camera unit inserted into the body of a subject. and, A region larger than the set partial region of the captured image is arranged in such a manner that the user can recognize the set partial region of the captured image. a display control unit that sequentially displays the captured images on a display unit using the area as a display area; A diagnosis of the inside of the body is performed based on the captured image, and a diagnosis result for the partial region is A diagnostic section that outputs the diagnostic results separately from those for the region; An inspection support device comprising:

3. The inspection support device according to claim 1 or 2, further comprising a change unit for changing the setting of the partial region. 。

4. The change unit changes one of the partial regions selected by a user from a plurality of partial regions prepared in advance. The inspection support device according to claim 3, wherein the inspection support device is changed to:

5. The change unit responds to one selected by a user from a plurality of observation regions prepared in advance.

5. The inspection support device according to claim 3, wherein the setting of the partial region is changed depending on the situation.

6. When a partial area of ​​the display area is designated by a user, the change unit changes the partial area 6. The partial region is determined by adjusting the region to a region that can be diagnosed by the diagnostic unit.

2. The inspection support device according to claim 1.

7. The change unit is configured to change the observation site of the captured image acquired by the acquisition unit. The method according to any one of claims 3 to 6, wherein the setting of the partial region is changed depending on whether the Inspection support equipment.

8. When the diagnosing unit diagnoses the partial region, the display control unit displays a diagnosis result together with and displaying the captured image on the display unit for a certain period of time in such a manner that the partial region can be recognized. The inspection support device according to claim 1 .

9. The diagnosis unit detects a lesion exceeding a preset judgment criterion in the set partial region. If not, a diagnosis is made of an adjacent region adjacent to the partial region; When the diagnosing unit diagnoses the adjacent region, the display control unit displays a diagnosis result together with The captured image is frozen for a certain period of time and displayed on the display unit in a manner that allows the adjacent area to be recognized.

9. The inspection support device according to claim 8.

10. An examination support method using an examination support device including an acquisition unit, a display control unit, and a diagnosis unit, The acquiring unit sequentially acquires images captured by a camera unit inserted into the body of a subject. The acquisition steps that will be obtained The display control unit displays the partial area of ​​the captured image in a manner that allows a user to recognize the partial area. A display step for sequentially displaying the captured images on a display unit using an area larger than the partial area as a display area. P and The diagnostic section performs a diagnosis of the inside of the body based on the partial region of the captured image. Tep and The inspection support method includes the steps of:

11. An examination support program for controlling an examination support device having an acquisition unit, a display control unit, and a diagnosis unit. So, The image capturing unit sequentially captures images captured by a camera unit inserted into the body of the subject. an acquiring step of acquiring the The display control unit is configured to display a partial area of ​​the captured image in a manner that allows a user to recognize the partial area. A display stage for sequentially displaying the captured images on a display unit using an area larger than the partial area as a display area. Top and A diagnostic unit is configured to diagnose the inside of the body based on the partial region of the captured image. Cutting step and An inspection support program that causes a computer to execute the above.

12. An examination support method using an examination support device including an acquisition unit, a display control unit, and a diagnosis unit, The acquiring unit sequentially acquires images captured by a camera unit inserted into the body of a subject. The acquisition steps that will be obtained The display control unit displays the partial area of ​​the captured image in a manner that allows a user to recognize the partial area. A display step for sequentially displaying the captured images on a display unit using an area larger than the partial area as a display area. P and The diagnostic unit performs a diagnosis of the inside of the body based on the captured image, and a diagnostic step of outputting the diagnostic result for the target area separately from the diagnostic results for other areas; The inspection support method includes the steps of:

13. An examination support program for controlling an examination support device having an acquisition unit, a display control unit, and a diagnosis unit. So, The image capturing unit sequentially captures images captured by a camera unit inserted into the body of the subject. an acquiring step of acquiring the The display control unit is configured to display a partial area of ​​the captured image in a manner that allows a user to recognize the partial area. A display stage for sequentially displaying the captured images on a display unit using an area larger than the partial area as a display area. Top and The diagnostic unit performs a diagnosis of the inside of the body based on the captured image, a diagnostic step of outputting a diagnostic result for the target area separately from diagnostic results for other areas; An inspection support program that causes a computer to execute the above.

Images