Image selection assistance device, image selection assistance method, and recording medium

Abstract

The present invention provides an image selection assisting device capable of easily selecting an appropriate image from various images acquired manually and automatically. The image selection assisting device assists in selecting a still image based on captured image data acquired by an endoscope. The image selection assisting device uses a first still image acquired based on captured image data when an operator of an endoscope performs a still image acquisition operation and one or more second still images acquired from captured image data at a time different from when the acquisition operation was performed, extracts a third still image having a shooting time satisfying a preset condition between the shooting time of the first still image and the shooting time of the first still image from the second still image, and associates the first still image with the third still image.

Description

Technical Field

[0001] The present invention relates to an image selection assisting device, an image selection assisting method and a recording medium for assisting selection of an image obtained by photographing an endoscope. Background Art

[0002] Conventionally, there is known an image management system that stores images acquired by endoscopes. The images stored in the image management system are inserted as key images in a report prepared for an examination using an endoscope, for example.

[0003] The acquisition of images is performed, for example, when an operator of the endoscope performs an operation to acquire images, that is, is performed manually.

[0004] Alternatively, the acquisition of images is performed automatically without relying on the image acquisition operation performed by the operator of the endoscope. For example, Patent Document 1 discloses a structure in which, in order to store the most appropriate images for diagnosis, such as lesion images, without omission and reliably, an index value of mucosal information is calculated based on an image signal obtained by photographing an observation object with an endoscope, and a specific image signal that is judged to satisfy a specific condition based on the index value is automatically stored.

[0005] Previous technical literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2016-67782 Summary of the invention

[0008] Technical issues to be solved by the invention

[0009] An image used as a key image or the like of a desired report can be selected from images acquired manually or automatically.

[0010] For example, manually acquired images may be images taken at inappropriate times, such as large jitters, due to a delay in the operator's acquisition operation. Also, with only manually acquired images, the operator may forget the acquisition operation or fail to obtain the desired image.

[0011] On the other hand, when automatically acquiring images, it is difficult to determine a complete reference for automatically acquiring the desired images. Therefore, the automatically acquired images sometimes contain unnecessary images. In addition, the desired images may not be obtained only from the automatically acquired images.

[0012] However, if the image is selected from each of the manually and automatically acquired images, it is difficult to select an appropriate image from these images. For example, for a manually acquired image, it is difficult for the image selector to determine which image was automatically acquired at a shooting time close to the shooting time of the image, and it is difficult to compare these images.

[0013] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image selection assistance device, an image selection assistance method, and an image selection assistance program that can easily select an appropriate image from various images acquired manually and automatically.

[0014] Means for solving technical problems

[0015] The image selection assistance device of the present invention has a processor and a memory, and assists in the selection of a still image based on captured image data obtained by an endoscope, wherein the memory records a first still image acquired based on the captured image data when an operator of the endoscope performs a still image acquisition operation and one or more second still images acquired from the captured image data at a time different from when the acquisition operation was performed, and the processor performs the following processing: extracting a third still image having a shooting time that satisfies a preset condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images, and associating the first still image with the third still image.

[0016] The image selection assisting method of the present invention assists in the selection of a still image based on captured image data obtained by an endoscope. In the above-mentioned image selection assisting method, a first still image acquired based on the captured image data when an operation of acquiring a still image is performed by an operator of the above-mentioned endoscope and one or more second still images acquired from the captured image data at a time different from when the above-mentioned acquisition operation is performed are used. A third still image having a shooting time satisfying a preset condition between the shooting time of the above-mentioned first still image and the shooting time of the above-mentioned first still image is extracted from the above-mentioned one or more second still images, and the above-mentioned first still image is associated with the above-mentioned third still image.

[0017] The image selection assistance program of the present invention assists in the selection of a still image based on captured image data obtained by an endoscope, and the image selection assistance program is used to cause a computer to perform the following processing: using a first still image acquired based on the captured image data when an operator of the endoscope performed an operation to acquire a still image and one or more second still images acquired from the captured image data at a time different from when the acquisition operation was performed, extracting a third still image having a shooting time that satisfies a preset condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images, and associating the first still image with the third still image.

[0018] Effects of the Invention

[0019] According to the present invention, it is possible to provide an image selection support device, an image selection support method, and an image selection support program capable of easily selecting an appropriate image from among images acquired manually and automatically. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a diagram showing an example of the overall structure of the hospital internal system.

[0021] Figure 2 Yes means Figure 1 A diagram showing a schematic configuration of an endoscope department system in the system shown.

[0022] Figure 3 Yes means Figure 2 FIG. 1 is a block diagram of the internal structure of a client PC in the endoscopy department system.

[0023] Figure 4 This is a diagram showing an example of a basic screen of an application of the client PC 22 .

[0024] Figure 5 It is a diagram showing an example of the endoscope device 24 .

[0025] Figure 6 Yes means Figure 5 A schematic diagram of the internal structure of the endoscope device 24 is shown.

[0026] Figure 7 It means by Figure 6 FIG. 5 is a diagram showing an example of a spectrum of light generated by the light source device 5 shown.

[0027] Figure 8 This is a diagram showing an example of accumulation of the first still image and the second still image.

[0028] Fig. 9 This is a diagram showing an example of an image selection screen of the client PC 22 .

[0029] Fig.10 1 is a diagram showing an example of an easily-missed image display screen of the client PC 22 .

[0030] Fig.11 This is a diagram showing an example of back-end AI analysis by the image processing device 23 .

[0031] Fig.12 This is a timing chart showing an example of normal light imaging based on automatic acquisition during special light observation. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0033] (Example of Implementation)

[0034] Figure 1 This is a diagram showing an example of the overall structure of the hospital internal system. Figure 1 The system shown includes a hospital information system (HIS) 10, an endoscopy department system 20, a pathology department system 30, a medical image storage system 40, and other department systems 50, which can be connected to a hospital internal LAN (Local Area Network) 60 and can cooperate with each other.

[0035] The HIS 10 is an integrated system including a medical affairs accounting system, a medical appointment system, a medical information system, etc., and has an electronic medical record database, etc. The electronic medical record database stores electronic medical records recording medical information of patients.

[0036] When information on a request (instruction) for an examination is issued from another medical department to the endoscopy department (hereinafter referred to as examination request information), the information is transmitted to the endoscopy department system 20 via the HIS 10 .

[0037] The examination request information includes, for example, patient information, instruction key information ("instruction number", "date and time of occurrence", etc.), request source information ("request department name", "request doctor name", "request date", etc.), instruction information ("request disease name", "examination purpose", "examination type", "examination items", "examination site", "comments", etc.), examination reservation information ("examination date", "implementation time", etc.). Patient information is information related to the patient, and includes "patient ID", "patient name", "date of birth", "age", "sex", "inpatient / outpatient classification", etc., which are information inherent to the patient.

[0038] The endoscope department system 20 is a system for managing the endoscope department.

[0039] The pathology department system 30 is a system for managing the pathology department.

[0040] The medical image storage system 40 is a system for electronically storing, searching, and analyzing inspection images from medical image diagnostic devices such as endoscopes, CT (Computed Tomography), and MRI (Magnetic Resonance Imaging). The medical image storage system 40 may be, for example, a PACS (Picture Archiving and Communication Systems) or other systems capable of storing medical images.

[0041] The other department system 50 is a system for managing other departments.

[0042] Figure 2 Yes means Figure 1 The endoscope department includes a reception desk 20A, a pre-treatment room 20B, a plurality of endoscope inspection rooms (hereinafter referred to as inspection rooms) 20C, a cleaning room 20D, and a conference room 20E.

[0043] The reception desk 20A is a place where the examination is accepted. The pre-treatment room 20B is a room for consultation and pre-treatment before the endoscope examination. The inspection room 20C is a room for endoscope examination. The cleaning room 20D is a room for cleaning the scope used in the endoscope examination.

[0044] Figure 2 The endoscope department system 20 shown includes an endoscope department server 21, a plurality of client PCs (Personal Computers) 22, an image processing device 23, an endoscope device 24, a cleaning management device 25, and a cleaning machine 26. The endoscope department server 21, the plurality of client PCs 22, the image processing device 23, and the cleaning management device 25 are connected to an internal department LAN 27. The internal department LAN 27 is connected to an internal hospital LAN 60.

[0045] The image processing device 23 and the endoscope device 24 are installed in each of the plurality of inspection rooms 20C. The image processing device 23 and the endoscope device 24 communicate with each other and perform synchronization processing to synchronize the internal times of each other.

[0046] The endoscope device 24 includes an insertion portion (scope) having an imaging element at the distal end, and inputs continuous image data obtained by continuous imaging by the imaging element into the image processing device 23. Figure 5 In the embodiment of the present invention, a configuration example of the endoscope device 24 will be described later.

[0047] The endoscope device 24 generates a first still image based on the captured image data obtained by the imaging element when the operator of the endoscope device 24 performs a still image acquisition operation (for example, pressing a button). The first still image is a manually acquired image acquired by the operator performing the acquisition operation. The endoscope device 24 adds information of the shooting time, which is the internal time of the endoscope device 24 when the first still image is acquired, to the generated first still image. Then, the endoscope device 24 inputs the generated first still image to the image processing device 23.

[0048] The image processing device 23 is connected to the endoscope device 24 of the inspection room 20C in which the image processing device 23 is installed. The image processing device 23 constitutes an analysis device that receives the captured image data from the endoscope device 24.

[0049] The captured image data obtained by continuous shooting by the endoscope device 24 is continuously input to the image processing device 23. The image processing device 23 generates a second still image based on the captured image data at a time different from when the operator of the endoscope device 24 performs the acquisition operation of the still image, among the captured image data continuously input from the endoscope device 24. The second still image is an automatically acquired image acquired independently of the acquisition operation from the operator. In addition, the image processing device 23 adds information of the shooting time as the internal time of the image processing device 23 when the second still image is acquired to the generated second still image.

[0050] For example, the image processing device 23 analyzes the continuously input captured image data by AI (Artificial Intelligence). Then, the image processing device 23 generates a second still image based on the captured image data when the analyzed result is based on the captured image data continuously input from the endoscope device 24. When the analyzed result is based on an example of a time different from when the above-mentioned acquisition operation is performed, for example, when an abnormal finding is detected based on the captured image data. The abnormal finding is, for example, a lesion such as a malignant tumor, but it is not limited to this, and can be set to various abnormal findings.

[0051] Furthermore, the image processing device 23 transmits the first still image input from the endoscope device 24 and the generated second still image to the endoscope department server 21 via the department internal LAN 27. The first still image and the second still image transmitted by the image processing device 23 are processed by the endoscope department server 21 under the control of the endoscope department server 21. Figure 1 The medical image archiving system 40 shown stores the images.

[0052] In addition, although the configuration in which the first still image generated by the endoscope device 24 is sent to the endoscope department server 21 via the image processing device 23 has been described, the endoscope device 24 may be configured to send the first still image to the endoscope department server 21 without passing through the image processing device 23. Furthermore, as the endoscope device 24, an endoscope device in which the image processing device 23 is integrated may be used.

[0053] The cleaning machine 26 and the cleaning management device 25 are installed in the cleaning room 20D. The cleaning machine 26 is a device that cleans the scope and the like used in the endoscope inspection.

[0054] The cleaning management device 25 is connected to the reprocessor 26 and is a computer for storing information such as the cleaning history by the reprocessor 26 in the endoscope department server 21 .

[0055] The endoscope department server 21 is a computer that centrally controls the client PC 22, the image processing device 23, and the cleaning management device 25. The endoscope department server 21 has a built-in database DB, and various information (inspection request information, inspection result information, etc.) is stored in the database DB.

[0056] A predetermined application program is installed in the client PC 22 , and the program enables reference and editing of data recorded in the database DB, storage of data in the database, and the like.

[0057] Figure 3 Yes means Figure 2 The block diagram of the internal structure of the client PC in the endoscopy department system is shown in FIG. Figure 3 As shown, the client PC 22 is composed of an input unit 22a, a display unit 22b, a recording unit 22c, a transceiver unit 22d and a control unit 22e.

[0058] The input unit 22 a is an input mechanism for performing various inputs, and is composed of an input device such as a keyboard and a touch panel, or a pointing device such as a mouse and a trackball.

[0059] The display unit 22 b is a display for performing various displays such as images and reports, and is composed of an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube).

[0060] The recording unit 22c is composed of a hard disk or the like for recording various data.

[0061] The transceiver unit 22d is composed of a transceiver interface circuit and the like, and executes processing for transmitting and receiving various instructions, various requests, and various data via the department internal LAN 27.

[0062] The control unit 22e includes various processors that execute programs to perform processing, RAM (Random Access Memory), and ROM (Read Only Memory).

[0063] Various processors include general-purpose processors such as CPUs (Central Processing Units) that execute programs and perform various processes, FPGAs (Field Programmable Gate Arrays) and other processors whose circuit structures can be changed after manufacturing, as well as programmable logic devices (PLDs) and ASICs (Application Specific Integrated Circuits) that are processors with circuit structures specially designed to perform specific processes, i.e., dedicated circuits.

[0064] More specifically, the structure of these various processors is a circuit formed by combining circuit elements such as semiconductor elements.

[0065] The control unit 22e may be composed of one of various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA).

[0066] The control unit 22e controls each unit of the client PC 22 according to various requests sent from the outside via the above-mentioned program and the department internal LAN 27, instruction information input from the input unit 22a, and the like.

[0067] Figure 4 2 is a diagram showing an example of a basic screen of an application of the client PC 22. When a user starts an application in the client PC 22 and logs in, the control unit 22e obtains basic screen data from the database DB and uses Figure 4 The basic screen shown is displayed on the display unit 22b. In addition, the client PC 22 is, for example, the client PC 22 of the inspection room 20C, but is not limited to this. Figure 2 The other client PC 22 is shown.

[0068] This basic screen is composed of an area A displaying a list of inspection request information (partially extracted information), an area B displaying various operation buttons, and an area C displaying a launcher effective for the inspection request information selected in the list in area A.

[0069] In the list of area A, processing items such as "consultation", "acceptance", "inspection", and "accounting" are set for each examination request information. When the processing indicated in each processing item for each examination request information is completed, an "○" mark is displayed in each processing item.

[0070] The data for displaying the mark is stored in the database DB by the endoscope department server 21 when each process is completed. For example, when the inspection based on the inspection request information is completed, the endoscope department server 21 associates the information indicating the end of the inspection with the inspection request information and stores it in the database DB. As a result, the "○" mark is displayed in the processing item of "inspection". The information on whether each process is completed can be manually input, or it can be set to be automatically notified from the client PC 22 or the endoscope device 24.

[0071] The application has a "Report" button C1 as an action button. The "Report" button C1 is displayed in Figure 4 The basic screen shown in FIG. 1 is in area C. The button C1 is a button for creating a report on the inspection based on the inspection request information selected from the list in area A.

[0072] The control unit 22e of the client PC 22 periodically acquires data for displaying the basic screen from the database DB, and displays the data on the display unit 22b.

[0073] exist Figure 2 In the illustrated system, when an instruction to create a report for a performed examination is given, the control unit 22e of the client PC 22 assists the report creator (eg, a doctor) in selecting a still image to be inserted into the report.

[0074] Figure 5 2 is a diagram showing an example of an endoscope device 24. Figure 5 As shown, the endoscope device 24 includes an endoscope 1 , and a control device 4 and a light source device 5 connected to the endoscope 1 .

[0075] The control device 4 is connected to a display device 7 that displays a captured image obtained by the endoscope 1 inside the subject, and an input unit 6 that is an interface for inputting various information to the control device 4. The control device 4 controls the endoscope 1, the light source device 5, and the display device 7.

[0076] The display device 7 has a display surface in which display pixels are arranged two-dimensionally, and displays an image based on the image data by drawing pixel data constituting the image data on each display pixel of the display surface. The display device 7 constitutes a display unit that switches the display image according to the command from the control device 4.

[0077] The endoscope 1 includes: an insertion portion 110, which is a tubular component extending in one direction and inserted into a subject; an operating portion 11, which is arranged at the base end of the insertion portion 110 and is provided with operating components for performing observation mode switching operations, video recording operations, forceps operations, air and water supply operations, and suction operations; an angle button 12, which is arranged adjacent to the operating portion 11; and a universal cord 13, which includes connector portions 13A and 13B for freely and detachably connecting the endoscope 1 to the control device 4 and the light source device 5, respectively.

[0078] In addition, although Figure 5 Although omitted in the figure, various channels such as forceps holes for inserting forceps for collecting biological tissues such as cells or polyps, channels for supplying air and water, and channels for suction are provided inside the operation portion 11 and the insertion portion 110.

[0079] The insertion portion 110 includes a flexible flexible portion 10A, a curved portion 10B provided at the distal end of the flexible portion 10A, and a hard distal end portion 10C provided at the distal end of the curved portion 10B.

[0080] The bending portion 10B is configured to be bendable by rotating the angle knob 12. The bending portion 10B can be bent in any direction and at any angle depending on the part of the subject to which the endoscope 1 is applied, and the distal end portion 10C can be directed in a desired direction.

[0081] Figure 6 Yes means Figure 5 A schematic diagram of the internal structure of the endoscope device 24 is shown. Figure 7 It means by Figure 6 FIG. 5 is a diagram showing an example of a spectrum of light generated by the light source device 5 shown.

[0082] The light source device 5 can switch and irradiate normal light and special light as illumination light. Normal light is light having a light spectrum suitable for recognition by humans such as doctors, such as white light. Special light is light having a light spectrum different from that of normal light and suitable for analysis by computers such as IEE (Image-Enhanced Endoscopy).

[0083] Specifically, the light source device 5 includes a light source processor 151 , a light source unit 152 , and an optical path coupling unit 154 . The light source processor 151 is connected to the system control unit 44 of the control device 4 , and controls the light source unit 152 according to a command from the system control unit 44 .

[0084] The light source unit 152 includes, for example, a plurality of semiconductor light sources, which are turned on or off respectively. When turned on, the light emission of each semiconductor light source is controlled to emit illumination light that illuminates the object of observation. In the present embodiment, the light source unit 152 includes four colors of LEDs, namely, V-LED (Violet Light Emitting Diode) 152a, B-LED (Blue Light Emitting Diode) 152b, G-LED (Green Light Emitting Diode) 152c, and R-LED (Red Light Emitting Diode) 152d.

[0085] The light source processor 151 can independently control the V-LED 152a, B-LED 152b, G-LED 152c and R-LED 152d to emit purple light V, blue light B, green light G or red light R by changing the light quantity independently. Figure 7 As shown, V-LED 152a generates purple light V with a central wavelength of 405±10nm and a wavelength range of 380 to 420nm. B-LED 152b generates blue light B with a central wavelength of 450±10nm and a wavelength range of 420 to 500nm. G-LED 152c generates green light G with a wavelength range of 480 to 600nm. R-LED 152d generates red light R with a central wavelength of 620 to 630nm and a wavelength range of 600 to 650nm.

[0086] When irradiating normal light, the light source processor 151 controls the LEDs 152a to 152d to emit white light with a light quantity ratio of Vc:Bc:Gc:Rc among the purple light V, blue light B, green light G and red light R. In addition, Vc, Bc, Gc, Rc>0.

[0087] When irradiating special light, the light source processor 151 controls each LED 152a to 152d so that the light intensity ratio of purple light V, blue light B, green light G and red light R, which are short-wavelength narrow-band light, becomes Vs:Bs:Gs:Rs.

[0088] The light intensity ratio Vs:Bs:Gs:Rs is different from the light intensity ratio Vc:Bc:Gc:Rc used when irradiating normal light, and is appropriately determined according to the purpose of observation. For example, when emphasizing superficial blood vessels, it is preferable to make Vs larger than the other Bs, Gs, and Rs, and when emphasizing medium and deep blood vessels, it is preferable to make Gs larger than the other Vs, Gs, and Rs.

[0089] The optical path coupling unit 154 couples the lights emitted from the V-LED 152a, the B-LED 152b, the G-LED 152c, and the R-LED 152d, and emits the coupled lights as illumination lights. The illumination lights emitted from the optical path coupling unit 154 of the light source unit 152 are incident on the light guide 153, which is described later and is built into the universal cord 13, and are irradiated onto the subject through the illumination lens 150 provided at the front end portion 10C of the insertion unit 110.

[0090] The distal end portion 10C of the endoscope 1 is provided with an imaging optical system including an objective lens 121 and a lens group 122, an imaging element 123 for imaging a subject through the imaging optical system, a memory 125 such as a RAM, a communication interface (I / F) 126, an imaging drive unit 127, and a light guide 153 for guiding illumination light emitted from a light source unit 152 to an illumination lens 150. The imaging element 123 constitutes an imaging unit of the present invention.

[0091] The light guide 153 extends from the front end portion 10C to the connector portion 13A of the universal cord 13. When the connector portion 13A of the universal cord 13 is connected to the light source device 5, illumination light emitted from the light source portion 152 of the light source device 5 can enter the light guide 153.

[0092] The imaging element 123 may be a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, etc. In the present embodiment, the imaging element 123 is a CMOS using a rolling shutter.

[0093] The imaging element 123 has a light receiving surface with a plurality of pixels arranged in a two-dimensional shape, and converts the optical image formed on the light receiving surface by the imaging optical system into an electrical signal (imaging signal) in each pixel. Then, the imaging element 123 converts the converted imaging signal from an analog signal into a digital signal with a predetermined number of bits, and outputs the imaging signal converted into a digital signal to the memory 125. For example, the imaging element 123 can use an imaging element equipped with a color filter such as a primary color or a complementary color.

[0094] The imaging element 123 may be disposed at the distal end portion 10C with the light receiving surface being perpendicular to the optical axis Ax of the objective lens 121 , or may be disposed at the distal end portion 10C with the light receiving surface being parallel to the optical axis Ax of the objective lens 121 .

[0095] The imaging optical system provided in the endoscope 1 is composed of optical components such as lenses and prisms (including the lens group 122) located on the optical path of light from the subject between the imaging element 123 and the objective lens 121, and the objective lens 121. The imaging optical system may be composed of only the objective lens 121.

[0096] The memory 125 temporarily records the digital imaging signal output from the imaging element 123 .

[0097] The communication I / F 126 is connected to the communication interface (I / F) 41 of the control device 4 . The communication I / F 126 transmits the imaging signal recorded in the memory 125 to the control device 4 via the signal line in the universal cord 13 .

[0098] The imaging drive unit 127 is connected to the system control unit 44 of the control device 4 via the communication I / F 126 . The imaging drive unit 127 drives the imaging element 123 and the memory 125 according to the command received from the system control unit 44 via the communication I / F 126 .

[0099] The control device 4 includes a communication I / F 41 connected to the communication I / F 126 of the endoscope 1 via the universal cord 13 , a signal processing unit 42 , a display controller 43 , a system control unit 44 , and a recording medium 45 .

[0100] The communication I / F 41 receives the imaging signal transmitted from the communication I / F 126 of the endoscope 1 and transfers the image pickup signal to the signal processing unit 42 .

[0101] The signal processing unit 42 has a built-in memory for temporarily recording the image signal received from the communication I / F 41, and processes the image signal, which is a collection of image signals recorded in the memory (image processing such as demosaic processing or gamma correction processing) to generate image information in a form that can be recognized, etc. The image information generated by the signal processing unit 42 is recorded in a recording medium 45 such as a hard disk or a flash memory.

[0102] Display controller 43 displays a captured image based on the captured image information generated by signal processing unit 42 on display device 7. Coordinates of each pixel data constituting the captured image information generated by signal processing unit 42 are associated with coordinates of any display pixel constituting the display surface of display device 7 and managed.

[0103] The system control unit 44 controls each unit of the control device 4 and sends instructions to the imaging drive unit 127 of the endoscope 1 and the light source processor 151 of the light source device 5, thereby centrally controlling the entire endoscope device 24. For example, the system control unit 44 controls the imaging element 123 via the imaging drive unit 127. In addition, the system control unit 44 controls the light source unit 152 via the light source processor 151.

[0104] The system control unit 44 or the signal processing unit 42 includes various processors, RAM, and ROM that execute programs to perform processing.

[0105] Various processors include general-purpose processors such as CPUs that execute programs to perform various processes, FPGAs that can change the circuit structure after manufacturing, programmable logic devices, and ASICs that have circuit structures specially designed to perform specific processes, i.e., dedicated circuits.

[0106] More specifically, the structure of these various processors is a circuit formed by combining circuit elements such as semiconductor elements.

[0107] The system control unit 44 or the signal processing unit 42 may be composed of one of various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0108] Next, a specific example of a method for switching the emission spectrum of the illumination light in the endoscope device 24 and a method for generating the second still image in the image processing device 23 in accordance with the switching of the emission spectrum of the illumination light in the endoscope device 24 will be described.

[0109] For example, the endoscope device 24 periodically switches the emission spectrum of the illumination light when the endoscope device 24 continuously takes pictures. As an example, the endoscope device 24 periodically switches as follows: the illumination light is set to normal light such as white light for only 9 frames, and then the illumination light is set to special light such as narrow-band light for only 1 frame. The frame mentioned here is a shooting frame based on the endoscope device 24. In addition, the ratio of the time of normal light to the time of special light in each cycle is not limited to 1:9, and can be set arbitrarily.

[0110] At this time, the captured image data obtained by continuous shooting by the endoscope device 24 includes captured image data obtained by shooting using normal light (captured image data of normal light) and captured image data obtained by shooting using special light (captured image data of special light).

[0111] The endoscope device 24 displays, for example, a normal light image based on the normal light captured image data and a special light image based on the special light captured image data on the display device 7. Alternatively, the endoscope device 24 may analyze or process the special light image based on the special light captured image data without displaying it on the display device 7 (i.e., in the background).

[0112] Then, the endoscope device 24 inputs at least one of the captured image data of normal light and the captured image data of special light among the captured image data obtained by continuous imaging to the image processing device 23 .

[0113] The image processing device 23 may detect abnormal findings based on the normal light image data among the image data continuously input from the endoscope device 24, may detect abnormal findings based on the special light image data, or may detect abnormal findings based on both image data.

[0114] Furthermore, for example, if an abnormal sight is detected based on the captured image data of normal light, the image processing device 23 generates a second still image of normal light based on the captured image data. Alternatively, if an abnormal sight is detected based on the captured image data of special light, the image processing device 23 generates a second still image of special light based on the captured image data.

[0115] Alternatively, if an abnormal sight is detected based on the special light captured image data, the image processing device 23 may generate two, namely, a second still image of special light based on the special light captured image data and a second still image of normal light based on the normal light captured image data. The normal light captured image data at this time is, for example, captured image data having a capture time closest to the capture time of the special light captured image data at which the abnormal sight is detected, among the normal light captured image data input from the endoscope device 24 to the image processing device 23.

[0116] The case where the endoscope device 24 alternately switches the illumination light to the normal light and the special light has been described, but the switching of the illumination light is not limited to this. That is, the endoscope device 24 can periodically switch the illumination light to two or more illumination lights having different emission spectra. For example, the endoscope device 24 can periodically switch to the normal light, the first special light, and the second special light having an emission spectrum different from the first special light. The image processing device 23 detects abnormal findings based on the captured image data of any emission spectrum among the captured image data continuously input from the endoscope device 24.

[0117] Figure 8 FIG. 1 is a diagram showing an example of accumulation of a first still image and a second still image. Figure 8 , the horizontal axis represents time.

[0118] For example, during diagnosis using the endoscope device 24, the doctor observes the display (e.g., Figure 5 The endoscope device 24 can be operated to obtain a still image at any time, such as when an abnormal situation is found, while viewing the real-time image on the display device 7).

[0119] As a result, a first still image is generated in the endoscope device 24. Figure 8 In the example shown, manually acquired images 51 and 52 are generated as first still images. The manually acquired images 51 and 52 are stored in the medical image storage system 40 together with time information indicating the time when the manually acquired images 51 and 52 were respectively generated.

[0120] In parallel with the doctor's diagnosis, the image processing device 23 detects abnormal findings (e.g., lesions such as malignant tumors) by AI analysis based on the captured image data continuously input from the endoscope device 24. This AI analysis is performed, for example, in a back-end AI analysis that is invisible to the doctor or the like (at the back end).

[0121] For example, the image processing device 23 calculates the index value of the mucosal information by AI based on the captured image data obtained by capturing the observation object by the endoscope device 24. The index value is, for example, a value indicating the state of at least one of the blood vessels, glandular ducts, redness, swelling, wrinkles, and mucus of the observation object displayed in the image signal. Then, the image processing device 23 determines whether the mucosal information (mucosal state) of the observation object corresponds to a lesion such as a malignant tumor based on the calculated index value, thereby detecting a lesion such as a malignant tumor (abnormal findings).

[0122] The image processing device 23 generates a second still image based on the captured image data continuously input from the endoscope device 24 when abnormal findings are detected through back-end AI analysis. Figure 8 In the example shown, automatically acquired images 53 to 57 are generated as second still images. The automatically acquired images 53 to 57 are stored in the medical image storage system 40 together with time information indicating the time when the automatically acquired images 53 to 57 were generated, respectively.

[0123] Fig. 9 2 is a diagram showing an example of an image selection screen of the client PC 22. When the first still image and the second still image are stored in the medical image storage system 40 by diagnosis using the endoscope device 24, the "screen display" button C2 included in the area C of the basic screen is indicated, and the areas D to F are displayed instead. Figure 2 Areas A and B are shown.

[0124] In area F, operation units such as buttons for performing various operations on areas D and E are arranged.

[0125] A first still image (manually acquired image) and a second still image (automatically acquired image) generated for the target patient are generated in regions D and E. The control unit 22 e of the client PC 22 acquires these still images from the medical image storage system 40 and displays them.

[0126] In area D, the first still images generated for the target patient are displayed side by side. Fig. 9 In the example shown, four first still images are displayed in the area D. Alternatively, more first still images may be displayed in the area D by scrolling the screen or the like.

[0127] The thick frame D1 is displayed so as to surround the first still image selected from among the first still images displayed in the area D. Fig. 9 In the example shown, the first still image on the upper left is selected. For example, by operating the operation unit in area F, the first still image selected among the first still images displayed in area D is switched, and the thick frame D1 moves accordingly.

[0128] In region E, automatically acquired images having a shooting time satisfying a preset condition between the shooting time of the first still image selected in region D and among the second still images generated for the target patient are displayed side by side as third still images.

[0129] Specifically, the control unit 22e extracts from the second still image a second still image having a time information whose difference from the time information of the selected first still image is less than a threshold value (a certain value), and displays the extracted second still image as a third still image in parallel in the area E. The threshold value is, for example, a value in the range of several seconds to several tens of seconds, but is not limited to a value in this range. Fig. 9 In the example shown, five second still images are displayed in the area E as the third still images.

[0130] In this way, the control unit 22e associates the first still image with the third still image and displays them on the display unit 22b. The association of the first still image with the third still image means that the first still image and the third still image are displayed in a manner that allows comparison, for example, Fig. 9 As shown in FIG. 1 , the first still image and the third still image are simultaneously displayed in respective different areas of the display unit 22 b.

[0131] The report creator can select a key image from the first still image and the third still image displayed in association by operating the operation unit in area F. A key image is one or more still images inserted in a report. Fig. 9 In the example shown, the report generator can select a key image from the first still image at the upper left of area D and the five third still images of area E. The selected key image constitutes a selected still image selected by the user from the first still image and the third still image, and becomes a candidate image for an image to be inserted into a report on an examination based on the endoscope device 24.

[0132] These first still images and third still images are still images that are taken at close times. Thus, the report maker can select either the first still image acquired manually or the third still image acquired automatically at the same time as the first still image as the key image. Therefore, for example, if the quality of the first still image is poor (for example, the image has a large shake), an image with good quality can be selected from the third still image and replaced.

[0133] Furthermore, when the report maker operates the operation unit of the area F to switch the selected first still image in the area D to the first still image in the upper right, the control unit 22e extracts the second still image having time information whose difference from the time information of the first still image in the upper right is less than the threshold value from the second still images, and displays the extracted second still image as the third still image in parallel in the area E. In this case as well, the report maker can select a key image from the first still image and the third still image displayed in association.

[0134] Similarly, the report generator selects a key image from the first and third still images for the other first still images in the area D. In addition, by operating the operation unit in the area F, a part of the first still images in the area D can be deleted.

[0135] Fig.10 2 is a diagram showing an example of an easily overlooked image display screen of the client PC 22. Fig. 9 The control unit 22e operates the operation unit in the area F shown, and causes the display unit 22b to display the area G instead of the areas D and E.

[0136] The fourth still image extracted by the control unit 22e from the second still images is displayed in parallel in region G. The fourth still image is a second still image having a shooting time that does not meet the above condition between the shooting times of all the first still images in region D and the second still images.

[0137] Specifically, when displaying area G instead of areas D and E, the control unit 22e extracts a second still image from the second still image having moment information whose difference from the moment information of all the first still images in area D is greater than a threshold, and displays the extracted second still image as the fourth still image in parallel in area G.

[0138] That is, the fourth still image is the second still image obtained by automatic shooting based on back-end AI analysis during a time period when no still image acquisition operation is performed. Therefore, the fourth still image includes an image that is easily missed during examination by the endoscope device 24.

[0139] The report generator can select a key image from the fourth still images displayed in the area G by operating the operation unit in the area F.

[0140] Thus, even if the operator (e.g., doctor) of the endoscope device 24 forgets to perform the still image acquisition operation or fails to perform the acquisition operation, the key image can be selected from the fourth still image automatically acquired and interpolated. Furthermore, the operator of the endoscope device 24 can perform the still image acquisition operation by focusing only on a clear lesion or the like discovered by the operator.

[0141] exist Fig.10 In the example shown, the case where only the extracted fourth still image is displayed in the region G is described, but the display method of the fourth still image is not limited to this. For example, the control unit 22e may display a plurality of fourth still images in parallel in chronological order together with the time information. Alternatively, the control unit 22e may display the plurality of fourth still images in chronological order together with the time information. Fig. 9 The screen shown displays the fourth still image in chronological order together with the selected key image. This allows the report creator to easily understand at which stage of the examination by the endoscope device 24 the fourth still image, which is easily overlooked, is a still image.

[0142] If you choose based on Fig. 9 , Fig.10 After the key image of the screen is displayed, if the report generator points to the "report" button C1 in the area C, a report generation screen for generating a report is displayed on the display unit 22b.

[0143] Then, on the report creation screen, Fig. 9 Each still image selected as a key image by the operations described in the above can be selected as an image to be inserted into the report. Thus, the report operator can easily create a report in which an appropriate key image selected from the first still image (manually acquired image) and the second still image (automatically acquired image) is inserted.

[0144] The created report is stored in, for example, the database DB of the endoscope department server 21 .

[0145] Fig.11 FIG. 2 is a diagram showing an example of back-end AI analysis by the image processing device 23. Fig.11 The horizontal axis represents time. The captured image data group 80 is each captured image data that is continuously input from the endoscope device 24 to the image processing device 23 .

[0146] As an example of back-end AI analysis, the image processing device 23 takes each of the captured image data included in the captured image data group 80 as an object and calculates the correct diagnosis rate based on the captured image data of the object. The correct diagnosis rate is the probability of making a correct diagnosis based on a still image based on the captured image data. The correct diagnosis rate based on the captured image data can be performed using, for example, an AI model that has been learned using a plurality of samples of a combination of captured image data and the correct diagnosis rate.

[0147] Then, when a certain number of consecutive (for example, 6 frames) of captured image data have a calculated correct diagnosis rate that is greater than a threshold value (for example, 90%), the image processing device 23 generates a second still image based on at least a portion of the consecutive captured image data. Fig.11 In the illustrated example, the image processing device 23 generates a second still image based on the captured image data 81 in the captured image data group 80. Then, the image processing device 23 stores the generated second still image in the medical image storage system 40 via the endoscope department server 21.

[0148] Fig.12 1 is a timing diagram showing an example of normal light photography automatically acquired during special light observation. First, the endoscope device 24 sets the illumination light based on the light source unit 152 to special light (step S1). Next, the endoscope device 24 starts continuous photography by the imaging element 123, and starts inputting the continuous photographed image data obtained by continuous photography into the image processing device 23 (step S2). The photographed image data is photographed image data obtained by photography in a state of irradiating special light.

[0149] Then, the image processing device 23 starts the above-mentioned back-end AI analysis based on the captured image data from the endoscope device 24 (step S3 ).

[0150] Next, it is assumed that the image processing device 23 detects abnormal findings based on the captured image data from the endoscope device 24 (step S4). At this time, the image processing device 23 stops the back-end AI analysis (step S5) and generates a still image (special light) based on the captured image data in which the abnormal findings were detected (step S6).

[0151] Next, the image processing device 23 transmits a detection signal indicating that an abnormal finding has been detected to the endoscope device 24 (step S7 ). In response, the endoscope device 24 sets the illumination light from the light source unit 152 to normal light (for example, white light) (step S8 ).

[0152] Next, the endoscope device 24 inputs the captured image data obtained by capturing the image with the imaging element 123 to the image processing device 23 (step S9). The captured image data is captured by capturing the image in a state of irradiating normal light.

[0153] Next, the image processing device 23 generates a still image (normal light) based on the captured image data input from the endoscope device 24 (step S10 ).

[0154] Next, the endoscope device 24 sets the illumination light by the light source unit 152 as special light (step S11). Next, the endoscope device 24 starts continuous shooting by the imaging element 123 again, and starts inputting continuous captured image data obtained by continuous shooting into the image processing device 23 (step S12). The captured image data is captured image data obtained by shooting in a state of irradiating special light.

[0155] Then, the image processing device 23 restarts the above-mentioned back-end AI analysis based on the image data captured from the endoscope device 24 (step S13). Thereafter, if an abnormal finding is detected again in the image processing device 23, steps S5 to S12 are repeated.

[0156] exist Fig.12 In the process shown, the image processing device 23 stores the normal light still image generated in step S10 in the medical image storage system 40 as the second still image.

[0157] In this way, when observing with special light, if a still image is automatically acquired through back-end AI analysis, a still image of normal light for reporting can be simultaneously captured and recorded in conjunction with the still image without the operator of the endoscope device 24 having to perform a switching operation.

[0158] Thus, the operator of the endoscope device 24 does not need to perform switching operations of the illumination light, and the operation load and trouble of the operation can be reduced. In addition, since there is no switching operation of the illumination light, a still image of normal light at almost the same time as the special light shooting can be obtained, so a still image of the special light and a still image of normal light for the same scene can be obtained. In addition, the operator of the endoscope device 24 can focus on the inspection or diagnosis without worrying about the recording of the report or the still image.

[0159] exist Fig.12In the example shown, the following processing is described: in steps S8 to S10, the endoscope device 24 sends the normal light captured image data to the image processing device 23, and the image processing device 23 generates a normal light still image based on the captured image data, but the present invention is not limited to this processing. For example, it is also possible to set the endoscope device 24 to generate a normal light still image based on the normal light captured image data and send the generated normal light still image to the image processing device 23.

[0160] Furthermore, the description has been given of the process in which the image processing device 23 stops the back-end AI analysis when an abnormal finding is detected based on the captured image data. However, the image processing device 23 may continue the back-end AI analysis when an abnormal finding is detected.

[0161] (Variation Example)

[0162] In the above description, as the use of the selected still image, a key image is inserted into a report generated for an examination using an endoscope. However, the use of the selected still image is not limited to this, and various uses are possible.

[0163] Furthermore, as an example of back-end AI analysis, the calculation of the correct diagnosis rate is described, but the back-end AI analysis is not limited to this, and various analyses that can detect abnormal findings can be set. Furthermore, as an analysis for automatically acquiring still images, the analysis using AI is described, but the analysis for automatically acquiring still images can also be set to be performed according to a specific algorithm that does not use AI.

[0164] Furthermore, although the configuration in which the first and second still images are stored in the medical image storage system 40 has been described, the first and second still images may be stored in the endoscopy department server 21, the client PC 22, or the image processing device 23.

[0165] Furthermore, as an example of displaying the first still image and the third still image in a manner that allows comparison, in Fig. 9 In the description, a case where the first still image and the third still image are simultaneously displayed in different areas of the display unit 22b is described, but the control unit 22e of the client PC 22 can display the first still image and the third still image in a comparative manner by switching the first still image and the third still image in time while displaying them on the display unit 22b.

[0166] As described above, the following matters are disclosed in this specification. (1)

[0168] An image selection assisting device having a processor and a memory and assisting selection of a still image based on captured image data obtained by an endoscope, wherein:

[0169] The memory stores a first still image acquired based on the captured image data when the operator of the endoscope performs a still image acquisition operation and one or more second still images acquired from the captured image data at a time different from when the acquisition operation is performed.

[0170] The above processor performs the following processing:

[0171] extracting a third still image having a shooting time satisfying a preset condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images,

[0172] The first still image is associated with the third still image. (2)

[0174] The image selection support device according to (1), wherein:

[0175] The processor associates the first still image with the third still image and displays the associated still image on a display. (3)

[0177] The image selection support device according to (2), wherein:

[0178] The processor displays the first still image and the third still image on the display in a manner that allows comparison. (4)

[0180] The image selection support device according to (2) or (3), wherein:

[0181] The processor displays the first still image and the third still image simultaneously in respective different areas of the display. (5)

[0183] The image selection support device according to (2) or (3), wherein:

[0184] The processor displays the first still image and the third still image on the display while switching them in time. (6)

[0186] The image selection support device according to any one of (2) to (5), wherein:

[0187] The processor selects, as a selected still image, a still image selected by a user from among the first still image and the third still image associated and displayed on the display. (7)

[0189] The image selection support device according to any one of (2) to (6), wherein:

[0190] The above processor performs the following processing:

[0191] extracting a fourth still image having a shooting time that does not satisfy the above condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images,

[0192] displaying the fourth still image on the display,

[0193] A still image selected by the user is selected from the fourth still images as a selected still image. (8)

[0195] The image selection support device according to (6) or (7), wherein:

[0196] The selected still image is a candidate image to be inserted into a report of the examination by the endoscope. (9)

[0198] The image selection support device according to any one of (1) to (8), wherein:

[0199] The second still image is a still image acquired from the captured image data based on detection of an abnormal finding based on analysis of the captured image data. (10)

[0201] The image selection support device according to (9), wherein:

[0202] The analysis includes calculating a probability of a correct diagnosis based on the second still image based on the captured image data, that is, a correct diagnosis rate.

[0203] The second still image is a still image acquired from the captured image data when a certain number or more of the captured image data for which the calculated correct diagnosis rate is greater than a certain value are continuously acquired. (11)

[0205] The image selection support device according to (9) or (10), wherein:

[0206] The analysis is performed by an analysis device that receives the captured image data from the endoscope.

[0207] The second still image is a still image acquired by the analyzing device from the captured image data. (12)

[0209] The image selection support device according to any one of (9) to (11), wherein:

[0210] The second still image includes a still image acquired from captured image data obtained by the endoscope in a state where the emission spectrum of illumination light captured by the endoscope is switched according to detection of the abnormal finding based on analysis of the captured image data. (13)

[0212] The image selection support device according to (12), wherein:

[0213] The above-mentioned second still image includes the following still images: a still image obtained from the above-mentioned captured image data in which the above-mentioned abnormal sight is detected; and a still image obtained from the captured image data obtained by the above-mentioned endoscope in a state in which the luminous spectrum of the illumination light captured by the above-mentioned endoscope is switched according to the detection of the above-mentioned abnormal sight. (14)

[0215] The image selection support device according to any one of (1) to (13), wherein:

[0216] The processor extracts the third still image having a shooting time whose difference from the shooting time of the first still image is less than a certain value from the one or more second still images. (15)

[0218] The image selection support device according to any one of (1) to (14), wherein:

[0219] When a plurality of the first still images are acquired, the processor extracts the third still image for a first still image selected by a user from among the acquired first still images. (16)

[0221] An image selection assisting method for assisting selection of a still image based on captured image data obtained by an endoscope, wherein:

[0222] using a first still image acquired based on the captured image data when an operator of the endoscope performs a still image acquisition operation and one or more second still images acquired from the captured image data at a time different from when the acquisition operation was performed,

[0223] extracting a third still image having a shooting time satisfying a preset condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images,

[0224] The first still image is associated with the third still image. (17)

[0226] An image selection assisting program assists selection of a still image based on captured image data obtained by an endoscope, the image selection assisting program causing a computer to execute the following processing:

[0227] using a first still image acquired based on the captured image data when an operator of the endoscope performs a still image acquisition operation and one or more second still images acquired from the captured image data at a time different from when the acquisition operation was performed,

[0228] extracting a third still image having a shooting time satisfying a preset condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images,

[0229] The first still image is associated with the third still image.

[0230] In the above, various embodiments are described with reference to the accompanying drawings, but the present invention is not limited to these examples, which is a matter of course. Obviously, if one is skilled in the art, it is clear that various variations or modifications can be thought of within the scope described in the claims, and these variations or modifications should also be understood to be within the technical scope of the present invention. In addition, the constituent elements in the above embodiments can be arbitrarily combined within the scope of the purpose of the invention.

[0231] In addition, this application is based on the Japanese patent application (Japanese patent application 2020-036270) applied on March 3, 2020, and the content is used in this application as a reference.

[0232] Explanation of symbols

[0233] 1-endoscope, 4-control device, 5-light source device, 6, 22a-input unit, 7-display device, 10-HIS, 10A-flexible unit, 10B-bending unit, 10C-front end, 11-operating unit, 12-angle button, 13-universal cord, 13A-connector unit, 13B-connector unit, 20-endoscope department system, 20A-reception desk, 20B-pre-treatment room, 20C-inspection room, 20D-cleaning room, 20E-negotiation room, 21-endoscope department server, 22-client PC, 22b-display unit, 22c-recording unit, 22d-transmitting and receiving unit, 22e-control unit, 23-image processing device, 24-endoscope device, 25-cleaning management device, 26-cleaning machine, 27-department internal LA N, 30-pathology department system, 40-medical image storage system, 41, 126-communication I / F, 42-signal processing unit, 43-display controller, 44-system control unit, 45-recording medium, 50-other department systems, 51, 52-manual image acquisition, 53~57-automatic image acquisition, 60-internal LAN of the hospital, 80-captured image data group, 81-captured image data, 110-insertion unit, 121-objective lens, 122-lens group, 123-camera element, 125-memory, 127-camera drive unit, 150-illumination lens, 151-light source processor, 152-light source unit, 152a-V-LED, 152b-B-LED, 152c-G-LED, 152d-R-LED, 153-light guide, 154-optical path coupling unit, D1-thick frame.

Claims

1. An image selection assisting device having a processor and a memory, and assisting selection of a still image based on captured image data obtained by an endoscope, wherein: The memory stores a first still image acquired based on the captured image data when an operator of the endoscope performs a still image acquisition operation and one or more second still images acquired from the captured image data at a time different from when the acquisition operation is performed. The processor performs the following processing: extracting a third still image having a shooting time satisfying a preset condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images, The first still image is associated with the third still image and displayed on a display, A still image selected by a user is selected as a selected still image from among the first still image and the third still image associated and displayed on the display.

2. The image selection assisting device according to claim 1, wherein: The processor displays the first still image and the third still image on the display in a manner that allows comparison.

3. The image selection assisting device according to claim 1, wherein: The processor displays the first still image and the third still image simultaneously in respective different areas of the display.

4. The image selection assisting device according to claim 1, wherein: The processor temporally switches the first still image and the third still image and displays the first still image and the third still image on the display.

5. The image selection assisting device according to any one of claims 1 to 4, wherein: The processor performs the following processing: extracting a fourth still image having a shooting time that does not satisfy the condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images, displaying the fourth still image on the display, A still image selected by the user is selected from the fourth still images as a selected still image.

6. The image selection assisting device according to claim 1, wherein: The selected still image is a candidate image for an insertion image in a report of the inspection based on the endoscope.

7. The image selection assisting device according to claim 1, wherein: The second still image is a still image acquired from the captured image data based on detection of an abnormal finding based on analysis of the captured image data.

8. The image selection assisting device according to claim 7, wherein: The analysis includes calculating a probability of a correct diagnosis being made based on the second still image based on the captured image data, that is, a correct diagnosis rate, The second still image is a still image acquired from the captured image data when a certain number or more of the captured image data for which the calculated correct diagnosis rate is greater than a certain value are continuously acquired.

9. The image selection assisting device according to claim 7, wherein: The analysis is performed by an analysis device that receives the captured image data from the endoscope, The second still image is a still image acquired by the analyzing device from the captured image data.

10. The image selection assisting device according to claim 7, wherein: The second still image includes a still image acquired from captured image data obtained by the endoscope in a state where the emission spectrum of illumination light captured by the endoscope is switched according to detection of the abnormal finding based on analysis of the captured image data.

11. The image selection assisting device according to claim 10, wherein: The second still image includes the following still images: a still image obtained from the captured image data in which the abnormal sight is detected; and a still image obtained from the captured image data obtained by the endoscope in a state in which the emission spectrum of the illumination light captured by the endoscope is switched according to the detection of the abnormal sight.

12. The image selection assisting device according to claim 1, wherein: The processor extracts the third still image having a shooting time whose difference from the shooting time of the first still image is less than a certain value from the one or more second still images.

13. The image selection assisting device according to claim 1, wherein: When a plurality of the first still images are acquired, the processor extracts the third still image for a first still image selected by a user from among the acquired first still images.

14. An image selection assisting method for assisting selection of a still image based on captured image data obtained by an endoscope, wherein: using a first still image acquired based on the captured image data when an operator of the endoscope performs a still image acquisition operation and one or more second still images acquired from the captured image data at a time different from when the acquisition operation was performed, extracting a third still image having a shooting time satisfying a preset condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images, The first still image is associated with the third still image and displayed on a display, A still image selected by a user is selected as a selected still image from among the first still image and the third still image associated and displayed on the display.

15. A computer-readable recording medium storing an image selection assisting program for assisting selection of a still image based on captured image data obtained by an endoscope, the image selection assisting program for causing a computer to execute the following processing: using a first still image acquired based on the captured image data when an operator of the endoscope performs a still image acquisition operation and one or more second still images acquired from the captured image data at a time different from when the acquisition operation was performed, extracting a third still image having a shooting time satisfying a preset condition between the shooting time of the first still image and the shooting time of the first still image from the one or more second still images, The first still image is associated with the third still image and displayed on a display, A still image selected by a user is selected as a selected still image from among the first still image and the third still image associated and displayed on the display.

Images