Medical image processing apparatus, endoscope system, diagnosis support method, and recording medium

Abstract

The present application provides a kind of medical image processing device, endoscope system, diagnostic aid method and program, can be in the observation of using the determination of using image processing is completed at the same time processing error determination, can inhibit the oversight of physician to observation object site.Medical image processing device has at least one processor.At least one processor obtains medical image, based on medical image, whether observation of object site is completed observation completion determination, carries out the display control of the determination result of observation completion determination is displayed on display, accepts the user input including the instruction of the display content of the determination result indicating observation completion determination is modified, and the content modified by user input is reflected in display.

Description

Technical Field

[0001] This invention relates to medical image processing devices, endoscope systems, diagnostic aid methods and procedures. Background Technology

[0002] In endoscopic examinations, it is required to comprehensively observe multiple parts of the organs or other organs to be examined. Patent Document 1 proposes an auxiliary technique to prevent omissions in imaging when determining the necessary imaging sites for the organs to be examined. The image display device described in Patent Document 1 detects prescribed anatomical landmarks from the endoscopic image, generates a map image showing the imaged and unimaged areas of the organ to be examined, and displays the map image on a monitor.

[0003] Patent Document 2 describes an image recognition technique using machine learning. The image recognition device described in Patent Document 2 processes input image data into categories and displays the category recognition results on a display device. The image recognition device described in Patent Document 2 accepts user evaluations of the category recognition results, uses the user's evaluation results and the input image data as learning data, and performs additional learning on the recognition unit, thereby improving recognition accuracy. According to paragraph 0057 of Patent Document 2, the image recognition device is mounted on an endoscope control unit.

[0004] Previous technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2018-50890

[0007] Patent Document 2: Japanese Patent Application Publication No. 2012-174222 Summary of the Invention

[0008] The technical problem to be solved by the invention

[0009] In examinations using medical equipment such as endoscopes, a comprehensive diagnosis of all locations within the target area is necessary depending on the purpose of the examination. For example, if the internal structure of the stomach is divided into more than ten smaller parts, there is a possibility that doctors may miss some areas when observing the stomach's interior. In the field of medical imaging, there is ongoing development of various technologies that utilize artificial intelligence (AI) to assist in the examination. The aim is to use image recognition technology on endoscopic and other medical images to automatically determine whether doctors have completed the observation of specific areas and display the results, thereby reducing the likelihood of missed observations.

[0010] However, the criteria for determining whether an observation of a region has been completed by AI, such as deep learning used to suppress missed observations, sometimes differ from those used by doctors. Because the judgment of whether an observation of a region has been completed also includes subjective factors such as whether the doctor consciously observed it, it can be difficult to determine in typical image classification tasks based on image analysis as described in Patent Document 2. Even if the AI ​​determines that an observation of a region has been completed, if the doctor has not actually observed it or the observation was insufficient, the AI's determination of completion may differ from human judgment.

[0011] The aforementioned issues are not limited to endoscopic examinations, but are general issues assumed when determining whether an observation of a certain area has been completed in the processing of various medical images.

[0012] The present invention was made in view of the following circumstances, and its object is to solve at least one of the above-mentioned problems by providing a medical image processing device, endoscope system, diagnostic aid method and procedure that can handle misjudgments while making a judgment using observations that have undergone image processing, and can suppress doctors from missing parts of the observed object.

[0013] means for solving technical problems

[0014] The medical image processing apparatus according to one aspect of this disclosure has at least one processor, wherein the at least one processor acquires a medical image, determines whether the observation of an object part has been completed based on the medical image, performs display control to display the determination result of the observation completion on a display, accepts user input including an instruction to correct the display content representing the determination result of the observation completion, and reflects the content corrected by the user input in the display.

[0015] According to this medical image processing apparatus, a determination result for the observation completion of a site, obtained through processing medical images, is displayed on a screen. The user can provide user input to correct the display of the determination result. The corrections indicated by the user input are reflected on the screen, changing the display to indicate whether the observation of the reported site is complete or incomplete.

[0016] The phrase "to reflect the corrected content in the display" can be understood as displaying the corrected content, that is, displaying the corrected content that reflects the correction.

[0017] In this medical image processing apparatus, if the determination of the observation completion of a part obtained through processing medical images differs from the doctor's (user's) determination, at least one processor can accept user interaction and correct the display of the determination result based on user input. This allows for the display of accurate information related to the observed part and assists in preventing missed parts.

[0018] Medical image processing apparatuses can be configured as a single device or as a combination of multiple devices. For example, a medical image processing apparatus can be implemented using one or more computers. The term "apparatus" includes the concepts of "system" and "module".

[0019] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: it further includes a display for showing the determination result of the observation completion determination.

[0020] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: when there is a mark of a specific area on the medical image, at least one processor determines that the observation of the specific area has been completed.

[0021] “Marker” refers to a boundary marker, which can be a characteristic shape and / or style of a part, or it can include non-human parts such as a part of an endoscope that is captured in an image along with a specific part.

[0022] A "specific location" can be a part of the organ being examined, i.e., a small part, or the entire organ. When observing multiple locations comprehensively, each of the multiple locations can be pre-defined as a "specific location".

[0023] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: when a specific part is present on a medical image for a fixed period of time or more, at least one processor determines that the observation of the specific part is complete.

[0024] For example, in time-series medical images such as moving images, the presence of the same area at a fixed time on the image can be inferred to be conscious observation of that specific area by a physician. Here, "fixed time" refers to a predetermined length of time.

[0025] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: when a specific region is marked in the central part of a medical image, at least one processor determines that observation of the specific region is complete.

[0026] The term "central part" is not limited to the strict center, but includes the approximate central portion of the image area in a medical image. When a doctor is focusing on a specific area and consciously observing it, the object can be considered to be captured in the central part of the image.

[0027] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: if the presence of a mark of a specific part on the medical image is taken as a first condition, the presence of the specific part on the medical image for a fixed period of time is taken as a second condition, and the presence of a mark of a specific part in the center of the medical image is taken as a third condition, if at least one of the first, second, and third conditions is met, at least one processor determines that the observation of the specific part is completed.

[0028] At least one processor may determine that the observation is complete when any one of the first, second, and third conditions is met, or when two or more conditions are met.

[0029] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor receives an input of a shooting instruction signal indicating the shooting timing of a still image, and upon receiving the input of the shooting instruction signal, determines that the observation of the part of the still image that is the subject of the shooting is completed.

[0030] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: it further includes a memory in which medical images and determination result information representing the determination of observation completion are stored. When at least one processor receives user input, the user input information, the determination result information which is the object of correction of the user input, and the medical image in which the determination result information has been obtained are associatedly saved into the memory.

[0031] According to this method, information about at least one medical image that the processor erroneously determined to be complete in the observation process, and the correct determination result for that medical image, is saved to memory. The data saved in this memory can be used as learning data, enabling additional learning to improve the inference accuracy of the inference model used in the observation completion determination process and / or the development of new models.

[0032] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: it further includes: a memory; and a communication device, wherein the memory stores a medical image and determination result information representing the determination result of observation completion, and when at least one processor receives user input, the communication device sends the user input information, the determination result information as the correction object of the user input, and the medical image for which the determination result information has been obtained to an external device.

[0033] According to this method, information regarding at least one medical image that the processor has incorrectly determined as observation completion, and the correct determination result for that medical image, is sent to an external device. The data sent to the external device can be used for learning data, enabling additional learning and / or the development of new models to improve the inference accuracy of the inference model used in the observation completion determination process, either on the external device or in other devices.

[0034] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor accepts user input that modifies the observation incomplete display of the part where the observation is not completed to the observation completion display, and based on the user input, changes the display content of the observation incomplete display to the display content of the observation completion display.

[0035] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor receives user input that modifies the observation completion display of the part where the observation completion determination result is observation completion to an observation incomplete display indicating that the observation is not completed, and based on the user input, changes the display content of the observation completion display to the display content of the observation incomplete display.

[0036] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor accepts user input that modifies the display of location information of a location determined to be observed as observed as completed by observation completion determination to display location information of another location, and modifies the display content of location information of the observed completed location based on the user input.

[0037] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor displays a medical image in a first display area of ​​a display, and displays the determination result of the observation completion judgment in a second display area of ​​the display that is different from the first display area.

[0038] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be employed: at least one processor displays part information about the observed part by schematically representing a model graphic of a part of the human body including the object part.

[0039] Model graphics can be, for example, figurative images that simulate the shape of organs.

[0040] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor distinguishes between observed and unobserved areas in a model graphic using different colors.

[0041] Using this method, doctors can easily and clearly distinguish between areas that have been observed and areas that have not.

[0042] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor displays location information about the observed site as text information.

[0043] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: it further includes an input device for use when a user makes an instruction input, and at least one processor accepts user input from the input device.

[0044] It can also have the following structure: the input device includes at least one of a keyboard, a touch panel, a voice input device, a switch located on the endoscope observer, and a foot switch.

[0045] In another aspect of the medical image processing apparatus disclosed herein, the structure may also be as follows: at least one processor sets multiple body parts as objects of observation, determines the completion of observation for each of the multiple body parts, and displays information indicating the body parts that have been observed and the body parts that have not been observed based on the determination result of the observation completion.

[0046] According to this method, it is possible to clearly distinguish between the parts that have been observed (completed observation parts) and the parts that have not been observed (incomplete observation parts) and display this information. When it is required to observe multiple parts comprehensively, it can effectively suppress the omission of observation (missed parts).

[0047] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor uses a neural network to observe and complete a determination.

[0048] At least one processor is capable of performing observation and judgment using a learned model that has been learned in a manner that enables tasks such as part identification and observation judgment through machine learning, such as deep learning.

[0049] In another aspect of the medical image processing apparatus disclosed herein, the following structure may be adopted: at least one processor acquires time-series medical images.

[0050] Time-series medical images can be dynamic images or groups of images captured at specific time intervals using methods such as burst shooting or interval shooting. Furthermore, the time intervals between images captured in a time-series format do not necessarily need to be constant.

[0051] Medical images can be endoscopic images taken using an endoscope. An endoscope can be an endoscopic observer or a capsule endoscope.

[0052] Another aspect of the endoscopic system disclosed herein includes an endoscopic observer and at least one processor, wherein the at least one processor acquires endoscopic images obtained by taking pictures inside the body using the endoscopic observer, determines whether the observation of the object site has been completed based on the endoscopic images, performs display control to display the determination result of the observation completion on a display, accepts user input including instructions to correct the display content indicating the determination result of the observation completion, and reflects the content corrected by the user input in the display.

[0053] According to this method, the endoscope system can display the judgment result of the observation completion in real time on the endoscopic image during observation. If an incorrect judgment result is displayed, the user can correct the display to the correct content in a timely manner.

[0054] Another aspect of this disclosure relates to a diagnostic aid method implemented by at least one processor, comprising the following steps: at least one processor acquires a medical image; determines whether an observation of a target site has been completed based on the medical image; performs display control to display the determination result of the observation completion on a display; receives user input including an instruction to correct the display content representing the determination result of the observation completion; and reflects the corrected content obtained through the user input in the display.

[0055] Another aspect of this disclosure relates to a program that enables a computer to: acquire medical images; determine whether observation of a body part has been completed based on the medical images; perform display control to display the result of the observation completion determination on a display; accept user input including instructions to correct the display content indicating the result of the observation completion determination; and reflect the content corrected by the user input in the display.

[0056] Invention Effects

[0057] According to the present invention, based on image processing of medical images, it is possible to determine whether a body part has been observed and display the determination result. If the determination result is incorrect, it can be corrected to the correct content by user input. Therefore, it is possible to prevent missed areas and assist in the implementation of appropriate examinations. Attached Figure Description

[0058] Figure 1 This is a perspective view illustrating the appearance of an endoscope system according to an embodiment of the present invention.

[0059] Figure 2 This is a block diagram illustrating a structural example of an endoscope system.

[0060] Figure 3 This is a functional block diagram illustrating the functions of the medical image processing apparatus according to the first embodiment.

[0061] Figure 4 This is a flowchart illustrating the operation of the medical image processing apparatus according to the first embodiment.

[0062] Figure 5 Example 1 is an image displayed on a monitor screen.

[0063] Figure 6 Example 2 is an image displayed on a monitor screen.

[0064] Figure 7 Example 3 shows the image displayed on the monitor screen.

[0065] Figure 8 Example 4 shows an image displayed on a monitor screen.

[0066] Figure 9 This is an example of a display image that is corrected from the observation completion information based on the observation completion determination to the observation incomplete information.

[0067] Figure 10 This is an example of a display image that corrects the display of "observation incomplete" information based on the observation completion determination to "observation completed" information.

[0068] Figure 11 This is an example of a display image that is modified from displaying observation completion information for a part based on the judgment result of observation completion to displaying observation completion information for another part.

[0069] Figure 12 This is an example of an image obtained when observing the cardia of the stomach.

[0070] Figure 13 This is another example of an image obtained when observing the cardia of the stomach.

[0071] Figure 14 This is a block diagram illustrating an example of the hardware structure of the medical image processing apparatus according to the first embodiment.

[0072] Figure 15 This is a functional block diagram illustrating the functions of the medical image processing apparatus according to the second embodiment.

[0073] Figure 16 This is a block diagram illustrating an example of the hardware structure of the medical image processing apparatus according to the second embodiment. Detailed Implementation

[0074] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. In this specification, the same reference numerals are used to denote the same constituent elements, and repeated descriptions are omitted where appropriate.

[0075] An Overview of Endoscopic Systems

[0076] Figure 1 This is a perspective view illustrating the appearance of an endoscope system 10 according to an embodiment of the present invention. The endoscope system 10 includes an endoscope observer 100, a processor device 200, a light source device 300, and a monitor 400.

[0077] The endoscope observer 100 is an electronic endoscope, such as a flexible endoscope. The endoscope observer 100 includes a handheld operating unit 102, an insertion unit 104, and a universal cable 106. The handheld operating unit 102 includes a bend button 140, an air / water supply button 141, a suction button 142, a function button 143, and a recording button 144.

[0078] The bend button 140 is used to indicate the bending direction and amount of the bend in the bend 114 of the insertion part 104. The bend button 140 includes two types of buttons: an up-down bend button that bends the bend 114 in the up-down direction and a left-right bend button that bends the bend 114 in the left-right direction.

[0079] Gas and water supply buttons 141 receive gas and water supply instructions. Suction button 142 receives suction instructions. Function buttons 143 are assigned various functions and receive instructions for each function. Shooting button 144 receives shooting instructions. The term "shooting" encompasses both still image shooting and moving image shooting. Shooting instructions include timing for still image shooting and timing for starting and ending moving image shooting.

[0080] The user operates the endoscope 100 by holding the control unit 102 and observes the body by inserting the insertion unit 104 into the body of the subject. Here, "user" refers to the doctor performing the procedure. The term "subject" is synonymous with patient, examinee, or person being examined.

[0081] The insertion part 104 is the part that is inserted into the body of the subject. The insertion part 104 is connected to the hand operation part 102 and has a soft part 112, a curved part 114 and a hard tip part 116 in sequence from the hand operation part 102 side.

[0082] The flexible part 112 is a flexible portion provided between the hand-operated part 102 and the bending part 114. The bending part 114 is a part that includes a mechanism that can be bent by operating the hand-operated part 102. The user can bend the bending part 114 by operating the bending knob 140 and change the direction of the rigid part 116 at the top, moving it up, down, left, and right.

[0083] exist Figure 1 The diagram shows an enlarged view of a portion of the top rigid section 116. The top rigid section 116 includes an imaging unit including a photographic lens 132, an illumination unit including illumination lenses 123A and 123B, and a jaw opening 126. Furthermore, the imaging unit... Figure 2 The symbol 130 is used for illustration. Additionally, the lighting section... Figure 2 The symbols 123 are used to illustrate the diagram.

[0084] During observation and handling, white light and / or narrowband light can be irradiated via illumination lenses 123A and 123B according to the user's operation. The narrowband light includes at least one of red narrowband light, green narrowband light, blue narrowband light, and violet narrowband light.

[0085] Additionally, when the air / water supply button 141 is operated, cleaning water is released from the water supply nozzle (not shown), or gas is released from the air supply nozzle (not shown). The cleaning water and gas can be used for cleaning the photographic lens 132 and the illumination lenses 123A, 123B, etc. Furthermore, the water supply nozzle and the air supply nozzle can be interchangeable.

[0086] The forceps port 126 communicates with a treatment instrument insertion channel (not shown) disposed inside the insertion section 104. A treatment instrument (not shown) is inserted into the treatment instrument insertion channel. A treatment instrument inlet (not shown) is provided on the hand operation section 102 for inserting the treatment instrument into the treatment instrument insertion channel. Treatment instruments may include, for example, biopsy forceps, catheters, high-frequency coils, etc. Other treatment instruments may include guide tubes, cannulas, slide tubes, etc. The treatment instrument is supported in the treatment instrument insertion channel to allow for appropriate movement. When performing procedures such as tumor resection, the user can use the treatment instrument to perform necessary treatments on the subject.

[0087] The universal cable 106 is used to connect the endoscope observer 100 to the processor device 200 and the light source device 300. A cable and a light guide extending from the insertion portion 104 are inserted into the universal cable 106. The cable includes a communication cable for signal transmission and a power supply cable for power supply. The endoscope observer 100 is connected to the processor device 200 and the light source device 300 via the universal cable 106.

[0088] In addition to the handheld operating unit 102, the endoscope system 10 may also include a foot switch (not shown) and / or a voice input device as input devices for user instructions, etc. The foot switch includes a pedal and a cable. The cable of the foot switch is connected to the processor device 200.

[0089] Structural Examples of Endoscopic Systems

[0090] Figure 2 This is a block diagram showing an example of the structure of the endoscope system 10. The following description will describe each example in the order of the endoscope observer 100, the light source device 300, and the processor device 200.

[0091] [Instructions for the endoscopic observation device]

[0092] The endoscope observer 100 includes an imaging unit 130 and an illumination unit 123. The imaging unit 130 is disposed inside the top rigid part 116. The imaging unit 130 includes a photographic optical system including a photographic lens 132, an imaging element 134, a drive circuit 136, and an analog front end (AFE) 138.

[0093] A photographic lens 132 is disposed on the top side face 116A of the top rigid portion 116. An image sensor 134 is disposed on the inner side of the photographic lens 132 (on the side closer to the curved portion 114 than the top side face 116A). The image sensor 134 is, for example, a CMOS (Complementary Metal-Oxide Semiconductor) type image sensor. A CCD (Charge Coupled Device) type image sensor may also be used as the image sensor 134.

[0094] The imaging element 134 is, for example, a color imaging element. Multiple pixels, each consisting of a plurality of light-receiving elements equipped with a color filter (not shown), are arranged in a specific pattern on the light-receiving surface (imaging surface) of the imaging element 134 in a two-dimensional configuration. Each pixel of the imaging element 134 includes a microlens, a color filter, and a photoelectric conversion unit (photodiode, etc.). For example, a primary color filter including red (R), green (G), and blue (B) is used as the color filter. The arrangement of the color pattern of the color filter is not particularly limited; for example, a Bayer arrangement can also be used.

[0095] Alternatively, the camera element 134 may also include pixels having a purple color filter corresponding to a purple light source (not shown) and / or an infrared filter corresponding to an infrared light source (not shown).

[0096] The drive circuit 136 supplies various timing signals required for the operation of the camera element 134 to the camera element 134 based on the control signals sent from the processor device 200.

[0097] The optical image of the subject being observed is imaged onto the light-receiving surface of the imaging element 134 via the photographic lens 132. The imaging element 134 converts the optical image of the subject into an electrical signal. The electrical signal output from the imaging element 134 is processed by the analog front-end 138 and converted into a digital image signal.

[0098] The analog front-end 138 includes an amplifier, a filter, and an analog-to-digital converter. The analog front-end 138 amplifies, denoises, and performs analog-to-digital conversion on the output signal of the imaging element 134. The output signal of the analog front-end 138 is then sent to the processor device 200. Furthermore, the imaging element 134, the drive circuit 136, and the analog front-end 138 can be configured as a single integrated circuit, and these circuit elements can be mounted on a single imaging chip.

[0099] The illumination unit 123 includes illumination lenses 123A and 123B. The illumination lenses 123A and 123B are disposed on the top side end face 116A of the top rigid part 116, adjacent to the photographic lens 132. The emission end of the light guide 170 is disposed inside the illumination lenses 123A and 123B.

[0100] Optical guide 170 plug-in Figure 1 The insertion part 104, the hand-operated part 102, and the universal cable 106 are shown. The incident end of the light guide 170 is disposed inside the light guide connector 108 provided at the end of the universal cable 106.

[0101] [Description of the light source device]

[0102] The light source device 300 supplies illumination light to the light guide 170 via the light guide connector 108. The illumination light is selected from various wavelength bands corresponding to the observation objective, such as white light (light in the white wavelength band or light in multiple wavelength bands), one or more specific wavelength bands, or combinations thereof. Furthermore, the specific wavelength band is a band narrower than the white wavelength band. The illumination light illuminating the observation area is sometimes referred to as the observation light.

[0103] The light source device 300 includes a light source 310 for illumination, an aperture 330, a condenser lens 340, and a light source control unit 350. The light source device 300 directs observation light into the light guide 170. The light source 310 includes a red light source 310R, a green light source 310G, and a blue light source 310B. The red light source 310R, the green light source 310G, and the blue light source 310B emit narrowbands of red, green, and blue light, respectively.

[0104] The light source 310 can generate observation light by arbitrarily combining narrowband lights of red, green, and blue. For example, the light source 310 can combine narrowband lights of red, green, and blue to generate white light. In addition, the light source 310 can combine any two colors of narrowband lights of red, green, and blue to generate narrowband light.

[0105] The light source 310 can generate narrowband light using any one of the narrowband colors of red, green, and blue. The light source 310 can selectively switch between emitting white light and narrowband light. Furthermore, narrowband light is synonymous with special light. The light source 310 can be an infrared light source emitting infrared light or an ultraviolet light source emitting ultraviolet light, etc.

[0106] The light source 310 can be configured to emit white light, pass through a filter for white light, and pass through a filter for narrowband light. The light source 310 can selectively emit either white light or narrowband light by switching between the filter that passes through white light and the filter that passes through narrowband light.

[0107] The filters that allow narrowband light to pass through may include multiple filters corresponding to different frequency bands. The light source 310 can selectively switch between multiple filters corresponding to different frequency bands to selectively emit multiple narrowband lights with different frequency bands.

[0108] The light source 310 can be applied to various types and wavelength bands, depending on the type of object being observed and the purpose of observation. Examples of types of light sources 310 include laser light sources, xenon light sources, and LED (Light-Emitting Diode) light sources.

[0109] By connecting the light guide connector 108 to the light source device 300, the incident end of the light guide 170 is positioned in the optical path of the emitted light from the condenser lens 340. The observation light emitted from the light source 310 passes through the aperture 330 and the condenser lens 340 to the incident end of the light guide 170. The observation light is then transmitted via the light guide 170 to the illumination lenses 123A and 123B, illuminating the observation area from the illumination lenses 123A and 123B.

[0110] The light source control unit 350 sends control signals to the light source 310 and the aperture 330 based on the instruction signal sent from the processor device 200. The light source control unit 350 controls the illuminance of the observation light emitted from the light source 310, the switching of the observation light, and the on / off state of the observation light.

[0111] [Structure of the processor device]

[0112] The processor device 200 includes an image input controller 202, an image processing unit 204, a communication control unit 205, a video output unit 206, and a storage unit 207. Additionally, the processor device 200 includes a CPU 210, a ROM 211, a RAM 212, an operation unit 208, a sound processing unit 209, and a speaker 209A.

[0113] The image input controller 202 acquires camera signals from the endoscope observer 100. The image processing unit 204 processes the camera signals acquired via the image input controller 202 to generate an endoscopic image of the observed object. Furthermore, the term "image" includes both the image itself and the image data representing the image. An image encompasses both the concepts of a moving image and a still image. The camera signals output from the endoscope observer 100 can be understood as an "endoscopic image."

[0114] The image processing unit 204 can perform image quality correction by applying digital signal processing such as white balance processing and shadow correction processing to the input camera signal. The image processing unit 204 can be constructed using a dedicated digital signal processing circuit for image processing. Alternatively, some or all of the processing functions of the image processing unit 204 can be implemented by executing a program via the CPU 210. Based on the camera signal obtained from the endoscope observer 100, the image processing unit 204 can generate one or more spectral images. Furthermore, the image processing unit 204 can also attach supplementary information specified by the DICOM (Digital Imaging and Communications in Medicine) standard to the endoscopic images.

[0115] Storage unit 207 stores endoscopic images generated using endoscope observer 100. Storage unit 207 can store various information (ancillary information) accompanying the endoscopic images.

[0116] The video output unit 206 sends various display signals, including the image generated by the image processing unit 204, to the monitor 400. The monitor 400 displays the image of the observed object, etc., according to the display signals output from the video output unit 206.

[0117] The communication control unit 205 controls communication with devices connected via communicable means such as a hospital local area network (LAN) and a hospital information system (HIS). The communication control unit 205 can apply communication protocols compliant with the DICOM standard.

[0118] CPU 210 functions as the overall control unit that controls all components within processor device 200 and provides unified control over the endoscope system 10 as a whole. CPU 210 also functions as a memory controller that controls ROM (Read Only Memory) 211 and RAM (Random Access Memory) 212. ROM 211 stores various programs and control parameters used to control the operation of processor device 200.

[0119] RAM 212 serves as a temporary storage area for data during various processes and as a processing area for computations performed by CPU 210. Programs executed by CPU 210 are stored in RAM 212. RAM 212 can also be used as a buffer memory when acquiring camera signals or endoscopic images.

[0120] The operation unit 208 accepts user operations and outputs indication signals corresponding to the user operations. The operation unit 208 may be configured using one or more of the following: a keyboard, mouse, gamepad, touch panel, foot switch, and voice input device. Alternatively, a switch such as the shooting button 144 located on the endoscope observer 100 may be included in the operation unit 208.

[0121] CPU210 acquires the instruction signal (user input signal) sent from the operation unit 208 and performs processing or control corresponding to the acquired user input signal.

[0122] The sound processing unit 209 generates an audio signal representing information as an audio notification. The speaker 209A converts the audio signal generated by the sound processing unit 209 into sound. Examples of sounds output from the speaker 209A include messages, audio prompts, and warning sounds.

[0123] The processor device 200 performs various processes on the endoscopic images generated by the endoscope observer 100 or acquired via the communication control unit 205, so that the endoscopic images and various information accompanying the endoscopic images are displayed on the monitor 400. In addition, the processor device 200 can store the endoscopic images and various information accompanying the endoscopic images in the storage unit 207.

[0124] Additionally, a diagnostic assistance system for endoscopic images using AI is installed on the processor device 200. Details will be described later. The processor device 200 includes: an observation completion determination function that identifies which part of the body was captured in the endoscopic image of the observed object and determines whether the observation of that part is complete; a display control function that displays the observation completion determination result on the monitor 400; a user input acceptance function that accepts user input including instructions to correct the information in the displayed determination result; and a display correction function that reflects the corrections made by the user input in the display. The processor device 200 is an example of a "medical image processing apparatus" in this disclosure. The monitor 400 is an example of a "display" in this disclosure. The term "recognition" includes concepts such as identification, discrimination, reasoning, inference, detection, and classification.

[0125] Overview of the Medical Image Processing Apparatus According to the First Embodiment

[0126] Figure 3 This is a functional block diagram illustrating the functions of the medical image processing apparatus 20 according to the first embodiment of the present invention. The medical image processing apparatus 20 includes an image acquisition unit 22, an observation completion determination unit 24, a display control unit 26, and a user input receiving unit 28.

[0127] The image acquisition unit 22 acquires an endoscopic image 18 captured by the endoscope observer 100. The endoscopic image 18 may be an image represented by a camera signal output from the endoscope observer 100, or it may be an image obtained through the implementation of... Figure 2 The image is generated by the processing performed by the image processing unit 204 shown.

[0128] Figure 2 The image input controller 202 shown can function as the image acquisition unit 22. Furthermore, the image acquisition unit 22 may be configured to include a communication interface for acquiring endoscopic images 18 from an external device via a communication line, or it may include a media interface for acquiring endoscopic images stored in a portable information storage medium such as a memory card. Figure 2 The communication control unit 205 shown can function as the image acquisition unit 22.

[0129] Alternatively, the image acquisition unit 22 can also be from... Figure 2 The processor device 200 shown has an internal processing circuit that accepts data input interfaces and / or data input terminals for inputting endoscopic images 18. For example, the image acquisition unit 22 may also accept... Figure 2 The image processing unit 204 inputs the endoscope image 18 to the terminal. The endoscope image 18 is an example of a "medical image" in this disclosure.

[0130] The image acquisition unit 22 can acquire a dynamic image 18A composed of time-series frame images 18B captured by the endoscope observer 100. Furthermore, when the image acquisition unit 22 receives a still image capture instruction from the user and performs still image capture 19 during the capture of the dynamic image 18A, it can acquire a still image 19 captured according to the still image capture instruction.

[0131] The observation completion determination unit 24 is a processing unit that identifies the scene of the endoscopic image 18 acquired by the image acquisition unit 22 and determines whether the observation of the target area has been completed. Observation completion determination refers to the process of determining whether the observation of the target area has been completed. The observation completion determination unit 24 presupposes whether the doctor, as the user, actually consciously observed the target area and completed the observation of that area.

[0132] The observation completion determination unit 24 may be configured using a part recognizer, which identifies the scene in the endoscopic image 18 and performs image classification processing by assigning labels to the parts captured in the endoscopic image 18. The part recognizer may be configured using a learned model, such as a neural network learned through machine learning, such as deep learning. The observation completion determination unit 24 may be configured using a convolutional neural network (CNN). The observation completion determination unit 24 may also be an AI module using an AI model, which performs the task of identifying object parts and determining the completion of observation regarding the object parts based on the endoscopic image 18. The observation completion determination unit 24 may be configured as a combination of a part recognizer and a determination of whether observation is complete, or it may be configured using an AI model learned through machine learning that takes the endoscopic image 18 as input and outputs part classification and whether observation is complete.

[0133] The observation completion determination unit 24 can perform part identification and observation completion determination for each frame image of a portion or all of the multiple frame images 18B acquired in a time sequence.

[0134] The display control unit 26 controls the display content on the monitor 400. That is, the display control unit 26 generates display signals required for display output to the monitor 400. In addition to the display signal representing the endoscopic image 18, the display signals also include display signals for notification related to information indicating the determination result of the observation completion.

[0135] The display control unit 26 includes a judgment result information display control unit 26A and a display correction processing unit 26B. The judgment result information display control unit 26A generates a display signal for displaying the judgment result information obtained by the observation completion judgment unit 24 on the monitor 400.

[0136] The display signal generated by the determination result information display control unit 26A is output to the monitor 400. The monitor 400 displays information indicating the determination result of the observation completion and the endoscope image 18, etc., according to the display signal from the display control unit 26A.

[0137] The user input receiving unit 28 receives user input signals input via the input device 234. The user input signals include signals instructing the correction of information displayed on the monitor 400 indicating the completion of the observation judgment. The input device 234 is equivalent to... Figure 2 The operation unit 208 is shown. The input device 234 may be, for example, a keyboard, mouse, touch panel, foot switch, or voice input device, or a suitable combination thereof. The user can input various instructions by operating the input device 234.

[0138] The display correction processing unit 26B processes the information (display content) displayed on the monitor 400 according to the user input signal, and generates a display signal that reflects the correction content on the display. The display correction processing unit 26B can rewrite the information of the judgment result output from the observation completion judgment unit 24, and can also correct the display content on the monitor 400 while retaining the information of the judgment result of the error judgment output from the observation completion judgment unit 24 in the device.

[0139] The content corrected by the display correction processing unit 26B based on the user input signal is reflected in the display of the monitor 400. The display control unit 26B and... Figure 2 This corresponds to the video output unit 206 shown.

[0140] in addition, Figure 2 The CPU 210 shown can function as the observation completion determination unit 24. Figure 2 The combination of the image processing unit 204 and the CPU 210 shown can also appropriately share the functions of the observation completion determination unit 24.

[0141] Operational Example of Medical Image Processing Device 20

[0142] Figure 4 This is a flowchart illustrating the operation of the medical image processing device 20. Figure 4 Each step of the flowchart is implemented by a processor such as CPU 210 mounted on the medical image processing device 20.

[0143] In the image acquisition process of step S12, the medical image processing device 20 acquires an endoscopic image.

[0144] In the observation completion determination process in step S14, the medical image processing device 20 identifies the part of the object to be observed and determines whether the observation of the part has been completed.

[0145] In the observation completion display process of step S16, the medical image processing device 20 displays the information of the observation completion area that has been determined to be completed on the monitor 400 based on the determination result of the observation completion determination.

[0146] In step S18, the medical image processing device 20 determines whether there is user input indicating a correction to the displayed content of the observed area. If the user provides input indicating a correction to the display, the medical image processing device 20 proceeds to the user input information display step S20.

[0147] In step S20, the medical image processing device 20 reflects the user-inputted corrections on the display of the monitor 400.

[0148] In step S18, if no corrective input is displayed from the user's instruction, the medical image processing device 20 skips step S20, maintains the display of information about the observation completion site indicated by the observation completion determination result, and ends the process. Figure 4 The flowchart. The medical image processing device 20 can implement a process based on each of a plurality of frame images 18B acquired sequentially in a time series. Figure 4 The processing of flowcharts.

[0149] Can Figure 4 The working method of the medical image processing device 20 shown in the flowchart can be understood as a diagnostic assistance method implemented by the medical image processing device 20, or as an image processing method implemented by the medical image processing device 20.

[0150] <Example 1 of displaying an image>

[0151] Figure 5 This is example 1 of an image displayed on screen 40 of monitor 400. In Figure 5 Examples of endoscopic images taken using a flexible video gastroduodenoscope are shown. Figure 5 This is an example of an observation image of the upper part of the stomach body. The monitor 400's screen 40 includes a main screen 42 and a sub-screen 44. The endoscopic image 18, which is the observation image, is displayed on the main screen 42. The endoscopic image 18 being captured is displayed in real time as a moving image on the main screen 42. The main screen 42 functions as the observation image display area.

[0152] Subscreen 44 displays information related to the completed observation area. The user operates the endoscope observer 100 while observing the internal image (endoscopic image 18) displayed on the main screen 42. Information regarding the completion of the observation of the area is displayed on subscreen 44. The medical image processing device 20, based on scene recognition processing of the endoscopic image 18, determines whether the observation of a certain area is complete and displays the determination result on subscreen 44 through a graph and / or list display method. The main screen 42 is an example of a "first display area" in this disclosure. Subscreen 44 is an example of a "second display area" in this disclosure.

[0153] exist Figure 5 The first sub-screen 44A shows a schematic image 46 representing the stomach as the organ being examined, with different colors used to distinguish between the completed observation area 47 and the uncompleted observation area 48. Figure 5 The image shown is an example of the observation completed when the upper part of the stomach body is identified. The stomach is an example of a "body part" in this disclosure, and the image 46 is an example of a "model graphic" in this disclosure.

[0154] The second sub-screen 44B displays text information indicating the determination result of the observation completion determination unit 24. That is, on the second sub-screen 44B, the information of the part identified as the object of observation and the determination result information indicating whether the observation is completed are displayed as string-based information. Here, an example is shown where "upper part of the stomach body" is displayed as the part information and "observation completed" is displayed as the determination result information.

[0155] <Example 2 of displaying an image>

[0156] Figure 6 Example 2 is a display image shown on screen 40 of monitor 400. Figure 6 Examples of images showing the rectum. In Figure 6 The image shown is an example of a colonoscopy where the examination of the rectum has been completed. Figure 6 The first sub-screen 44A displays an image 46 representing the large intestine, showing the rectum as the observed portion 47. Figure 6 The second sub-screen 44B shows an example of a text-based display of "rectum" as information about the part of the body being observed and an example of a display of "observation completed" as information about the result of the observation.

[0157] <Example 3 of displaying an image>

[0158] Figure 7 Example 3 shows the image displayed on screen 40 of monitor 400. It can also replace the image displayed on... Figure 5The imagery of 46 described in the text is displayed, and as... Figure 7 As shown, the information of the observed parts is listed and displayed as text information.

[0159] exist Figure 7 The first sub-screen 44A displays a list of observed areas, while the second sub-screen 44B displays information about the area currently being observed. Figure 7 In the middle section, the observed parts are listed as the fundus, upper part, middle part, and lower part of the stomach.

[0160] Displayed Figure 7 The endoscopic image 18 on the main screen 42 is an observation image of the vestibular region of the stomach. When the observation completion determination unit 24 determines that the observation of the vestibular region is complete, text information indicating that the observation of the vestibular region is complete is displayed on the second sub-screen 44B.

[0161] exist Figure 7 The first sub-screen 44A displays a list of observed areas, while the second sub-screen 44B displays information about the area currently being observed. Figure 7 In the middle section, the observed parts are listed as the fundus, upper part, middle part, and lower part of the stomach.

[0162] Displayed Figure 7 The endoscopic image 18 on the first sub-screen 44A is an observation image of the vestibular region of the stomach. When the observation completion determination unit 24 determines that the observation of the vestibular region is complete, text information indicating that the observation of the vestibular region is complete is displayed on the second sub-screen 44B.

[0163] <Example 4 of displaying an image>

[0164] Figure 8 This is another example of a displayed image shown on screen 40 of monitor 400. Figure 8 In the first sub-screen 44A, no observed area exists, and the information related to the observed area is blank. Figure 8 The image shows the situation where the observation of the rectum is determined to be complete by the observation completion determination unit 24. In this case, text information indicating that the observation of the rectum has been completed is displayed on the second sub-screen 44B.

[0165] <Example 1 of the modified display based on user input>

[0166] Figure 9 This is an example of an image displayed when the observation completion information based on the observation completion determination is corrected to an observation incomplete information display. Figure 9 In the image, the left side shows the content before the correction, and the right side shows the content after the correction.

[0167] The observation completion determination unit 24 determines the completion of the observation of the vestibular region, and the result is displayed on the screen 40 of the monitor 400. Figure 9 The left image shows the completion of the vestibular observation.

[0168] In the image 46 of the first sub-screen 44A, the area block 51 corresponding to the part that has been confirmed as observed is displayed, for example, by filling it with a color indicating that the observation of the part has been completed (e.g., blue). Furthermore, when the vestibular part is re-determined as observed and completed, the area block 52 corresponding to the vestibular part in the image 46 is displayed, for example, by filling it with a color indicating that the part has been re-determined as observed and completed (e.g., orange). Additionally, text information indicating that the vestibular part observation has been completed is displayed on the second sub-screen 44B.

[0169] exist Figure 9 In the display state shown in the left figure, if the user does not input any correction instructions and a specified time has elapsed, it is determined that the observation information of the vestibular region is correct, and the display color of area block 52 changes from orange to blue. The "specified time" can be, for example, a few seconds. In addition, here, the display of the observation completed area in the image image 46 is shown as an example of a confirmed display via a change from "orange" to "blue", but it is also possible to display the observation completed without such a color change. Figure 9 The content displayed in sub-screen 44 shown in the left figure is an example of the "observation completion display" in this disclosure.

[0170] for Figure 9 As shown in the left figure, if the user does not complete the observation of the vestibular region during actual observation, the user can operate the input device 234 to input an instruction to correct the display indicating "observation completed" regarding the vestibular region to the display indicating "observation incomplete".

[0171] The user can input instructions for corrections by either typing the text information in the second sub-screen 44B or by selecting "Observation incomplete" from a drop-down menu that prompts for corrections.

[0172] When user input is modified to reflect the displayed content, the modified content reflects the user input. Figure 9 The display shown in the left image has been changed to... Figure 9 The corrected display content is shown in the right-hand image. Figure 9 In the right image, the area block 52 corresponding to the vestibular region in the imagery 46 is displayed using a color (e.g., gray) to indicate incomplete observation, and the text message indicating "incomplete observation" of the vestibular region is displayed on the second sub-screen 44B. Furthermore, in Figure 9In the image 46 on the right, for ease of illustration, the outline of the area block 52 is represented by a dashed line to clearly indicate the area block 52 that has been modified. However, in the actual screen display, such an outline may not be displayed. Figure 9 The content displayed in sub-screen 44 shown in the right figure is an example of "incomplete observation display" in this disclosure.

[0173] <Revised Example 2 of User Input-Based Display>

[0174] Figure 10 This is an example of a display image that corrects the display of "incomplete observation" information based on the observation completion determination result to "observation completed" information. The observation completion determination unit 24 determines whether the observation of the vestibular region is incomplete, and this is displayed on screen 40 of the monitor 400, as shown below. Figure 10 As shown in the left image, no information regarding the completion of observation regarding the vestibular region was reported. Furthermore, in Figure 10 In the image 46, for ease of illustration, the outline of the area block 52 is represented by a dashed line to clearly indicate that the area block 52 corresponding to the vestibular part is "observation incomplete", but such an outline is not displayed in the actual screen display.

[0175] for Figure 10 As shown in the left figure, when the user has completed the observation of the vestibular region during actual observation, the user can operate the input device 234 to input an instruction to correct the display indicating "observation not completed" regarding the vestibular region to the display indicating "observation completed".

[0176] When user input is modified to reflect the displayed content, the modified content reflects the user input. Figure 10 The display shown in the left image has been changed to... Figure 10 The corrected display content is shown in the right-hand image. Figure 10 In the right image, the area block 52 corresponding to the vestibular region in the image 46 is displayed by a display color (e.g., orange or blue) indicating the completion of observation, and the text information indicating "observation of the vestibular region completed" is displayed on the second sub-screen 44B.

[0177] <Example 3 of the revised display based on user input>

[0178] Figure 11 This is an example of a display image where the display of observation completion information for a region based on the observation completion determination result is corrected to the display of observation completion information for another region. The observation completion determination unit 24 determines the observation completion of the vestibular region, and on the screen 40 of the monitor 400, as shown... Figure 11 The left image shows information about the completion of observation regarding the vestibular region. Additionally, in... Figure 11In the imagery of image 46, it becomes a display indicating that the observation of the lower part of the stomach body is incomplete.

[0179] for Figure 11 As shown in the left figure, if the user fails to complete the observation of the vestibule but completes the observation of the lower part of the stomach body during the actual observation, the user can operate the input device 234 to input an instruction to correct the display of information related to the completed observation site from "vestibule" to "lower part of stomach body".

[0180] When user input is modified to reflect the displayed content, the modified content reflects the user input. Figure 11 The display shown in the left image has been changed to... Figure 11 The corrected display content is shown in the right-hand image. Figure 11 In the right image, the region 53 corresponding to the lower part of the stomach body in the image 46 is displayed by a color indicating completion of observation (e.g., orange or blue), and the region 52 corresponding to the vestibule is displayed by a color indicating incomplete observation (e.g., gray).

[0181] Additionally, the second sub-screen 44B displays the text message indicating "Observation complete" for the lower part of the stomach.

[0182] like Figures 9 to 11 As illustrated, the medical image processing apparatus 20 according to this embodiment can, in the event of an error in the determination result of observation completion, change the information displayed as "observation completed" or "observation not completed" based on the determination result, or change the information of the area where observation was completed.

[0183] Examples of judgment criteria in observation completion determination

[0184] As a criterion for determining whether an observation is complete in the observation completion determination unit 24, the following conditions can be used, for example.

[0185] [Condition 1] The markings (landmarks) of the object location are captured in the image.

[0186] [Condition 2] The observation site is fixed for more than one time.

[0187] [Condition 3] The subject is captured in the center of the image.

[0188] When at least one of these conditions 1 to 3 is met, the observation completion determination unit 24 determines that the observation is complete. The observation completion determination unit 24 may also determine that the observation is complete when two or more of these multiple conditions are met.

[0189] The "mark" in condition 1 can be an anatomical landmark, such as a specific pattern of the mucosa and / or a characteristic structure of a region. A mark is assigned to each region. If the observation completion determination unit 24 identifies a mark in the endoscopic image 18, it determines that the observation of the region associated with that mark is complete.

[0190] Figure 12 These are examples of images obtained when observing the cardia of the stomach. Figure 12 The diagram shows an example where observation is deemed complete due to fulfillment of condition 1. Since the endoscopic observer 100 is bent at the bend 114 to observe the cardia, the cardia opening 70 and the endoscopic observer 100 are captured on the endoscopic image 18 during observation. The cardia opening 70 and the endoscopic observer 100 can serve as markers for the cardia. The observation completion determination unit 24 identifies (detects) these markers from the endoscopic image 18, and determines that observation is complete if a marker is present on the image.

[0191] In obtaining such Figure 12 When the endoscopic image 18 is shown on the main screen 42, the observation completion determination unit 24 determines that the observation of the cardia has been completed as condition 1 is met. When the observation completion determination unit 24 determines that the observation of the cardia is complete, the area block 54 corresponding to the cardia is displayed in the image image 46 of the first sub-screen 44A using a display color indicating observation completion. In addition, text information indicating that the observation of the cardia is complete is displayed on the second sub-screen 44B.

[0192] Regarding condition 2, the observation completion determination unit 24 identifies the affected area from the endoscopic image 18. If the same affected area appears on the image for a fixed period of time or more, the observation is deemed complete. For example, for frame images 18B acquired sequentially at specific time intervals, the observation completion determination unit 24 detects the affected area in each frame image 18B. If the same affected area is detected in a specified number of consecutive frame images 18B, the observation of that area is deemed complete. The "fixed period of time" mentioned in condition 2 can be preset to an appropriate value, for example, approximately a few seconds.

[0193] Figure 13 This is another example of an image obtained when observing the cardia of the stomach. Figure 13 The example shown illustrates a situation where observation is deemed complete due to fulfillment of condition 3. Figure 13In the endoscope image 18, the opening 70 of the cardia is captured in the center. The observation completion determination unit 24 detects a marker as an object in the endoscope image 18, identifies the position and category (classification) of the marker within the image, and determines whether the marker exists in the center of the image. If the marker of the object part exists in the center of the endoscope image 18, the observation completion determination unit 24 can determine that the observation of that part has been completed.

[0194] The cardia is an example of a "specific part" in this disclosure, and the orifice 70 of the cardia is an example of a "marker" in this disclosure. Condition 1 is an example of a "first condition" in this disclosure. Condition 2 is an example of a "second condition" in this disclosure. Condition 3 is an example of a "third condition" in this disclosure.

[0195] The criteria for determining observation completion are not limited to conditions 1 to 3. For example, when a still image is captured by operating the capture button 144 of the endoscope observer 100, the observation completion determination unit 24 can also determine that the observation of the part of the still image that is the subject has been completed. That is, when the observation completion determination unit 24 receives an input of a capture instruction signal indicating the capture time of the still image, it can also determine that the observation of the part of the still image that is the subject of the capture has been completed. Alternatively, condition 4, such as "capturing a still image," can be used as the criterion for determining observation completion. When one or more of conditions 1 to 4 are met, the observation completion determination unit 24 determines that the observation is complete.

[0196] Example of the hardware structure of medical image processing device 20

[0197] Figure 14 This is a block diagram illustrating the hardware structure of a medical image processing apparatus 20. The medical image processing apparatus 20 is not limited to being mounted on a processor device 200, but can also be applied to an information processing device different from the processor device 200. For example, the processing function of the medical image processing apparatus 20 can also be installed on an image processing server or the like connected to a hospital intranet.

[0198] The medical image processing apparatus 20 can be implemented using a computer system consisting of one or more computers. That is, the medical image processing apparatus 20 is implemented by installing programs on a computer.

[0199] The medical image processing device 20 includes a processor 222, a computer-readable medium 224 which is a non-transitory tangible object, an image input interface 225, an image processing processor 226, a communication interface 227, an input / output interface 228, and a bus 230.

[0200] Processor 222 includes a CPU. Processor 222 may also include a GPU (Graphics Processing Unit). Processor 222 is connected via bus 230 to computer-readable medium 224, image input interface 225, image processing processor 226, communication interface 227, and input / output interface 228. Medical image processing apparatus 20 may also include input device 234 and display device 236. Input device 234 and display device 236 are connected to bus 230 via input / output interface 228.

[0201] Computer-readable medium 224 includes a memory as a primary storage device and a storage device as an auxiliary storage device. Computer-readable medium 224 may be, for example, a semiconductor memory, a hard disk drive (HDD) device, a solid-state drive (SSD) device, or a combination thereof.

[0202] Image input interface 225 can be used as Figure 3 The image acquisition unit 22 shown functions. The medical image processing device 20 is connected to the endoscope observer 100 via the image input interface 225.

[0203] Image processing processor 226 is equivalent to Figure 2 The image processing unit 204 shown is a dedicated processor for image processing.

[0204] Communication interface 227 is equivalent to Figure 2 The communication control unit 205 is shown. The medical image processing device 20 is connected to a communication line (not shown) via a communication interface 227. This communication line may be, for example, a local area network (LAN) built within the hospital. This hospital-wide communication network is referred to as the hospital intranet. The hospital intranet may also be further connected to a wide area network (WAN) such as the Internet via a router. An image storage server 500, such as a PACS (Picture Archiving and Communication Systems) server, is connected to the hospital intranet.

[0205] A PACS server is a computer that stores and manages various data, including medical images captured using various modalities. It features large-capacity external storage and database management software. The PACS server communicates with other devices via the hospital's intranet, sending and receiving various data, including image data. A PACS server can also be a DICOM server, operating based on the DICOM protocol.

[0206] The medical image processing device 20 can also acquire endoscopic images from the image storage server 500 connected via the communication interface 227.

[0207] The computer-readable medium 224 stores an observation completion determination program 240 and a display control program 260. Additionally, the computer-readable medium 224 includes an image storage unit 271, a determination information storage unit 272, and a user input information storage unit 273. The computer-readable medium 224 may store information including functions for causing the processor 222 to execute... Figure 2 The image processing program (not shown) describes some or all of the commands processed by the image processing unit 204.

[0208] The image storage unit 271 is a storage area that stores endoscopic images acquired via the image input interface 225 or the communication interface 227 in advance.

[0209] The observation completion determination program 240 includes a function to cause the processor 222 to execute as in Figure 3 The procedure for processing the command of the observation completion determination unit 24 is described in the text.

[0210] Display control program 260 includes functions for causing processor 222 to execute as... Figure 3 The program described herein is the procedure for processing commands by the display control unit 26. The display control program 260 includes a main screen control module 261, a sub-screen control module 262, and a display correction module 263.

[0211] The main screen control module 261 is a program module that performs controls related to the display of the main screen 42. The sub-screen control module 262 is a program module that performs controls related to the display of the sub-screen 44. The sub-screen control module 262 includes a program module for causing the processor 222 to execute as... Figure 3 The determination result information displayed in the description is the command processed by the control unit 26A. The display correction module 263 includes commands for causing the processor 222 to execute as... Figure 3 The program module described herein displays the commands processed by the correction processing unit 26B. Alternatively, part or all of the display control program 260 may be incorporated into the observation completion determination program 240.

[0212] The determination information storage unit 272 is a storage area that stores information about the determination results obtained by observing the execution of the determination procedure 240. The user input information storage unit 273 is a storage area that stores user input information input from the input device 234 via the input / output interface 228. In the computer-readable medium 224, information related to the organ being examined and multiple small parts (regions) to be observed in a series of examinations is stored for each type of examination performed using the endoscope system 10. Specifically, for example, in the case of examining all internal parts of the stomach, the areas to be observed include the cardia, fundus, angular esophagus, upper body, middle body, lower body, vestibule, and pylorus.

[0213] Here, a small part of the stomach is illustrated, but the classification labels for parts are not limited to this example. Furthermore, the site for determining the completion of observation is not limited to a single organ; it can span multiple organs. For example, observation and determination can be performed not only on the stomach but also on various parts of the examination area, including the esophagus and duodenum. For the esophagus, classifications such as upper esophagus, middle esophagus, and lower esophagus can be used. For the duodenum, classifications such as duodenal bulb and postbulbar duodenum can be used.

[0214] The display device 236 may be, for example, a liquid crystal display, an organic electroluminescence (OEL) display, a projector, or a suitable combination thereof. In addition to the recognition result, the display device 236 may also display the image of the object being processed, various setting information required for processing, and other information. The display device 236 is equivalent to... Figure 2 The monitor 400 shown is an example of a "display" in this disclosure. The display device 236 is an example of a "display".

[0215] The observation completion determination result performed by the observation completion determination procedure 240 is sent to the sub-screen control module 262 and displayed on the sub-screen 44. The display correction module 263 accepts user input from the input device 234 and corrects the display content of the sub-screen 44 according to the user input.

[0216] According to the medical image processing apparatus 20 of this embodiment, it is possible to automatically determine whether the observation of the target area has been completed based on the endoscopic image 18 being observed, and to display the information of the completed observation area on the monitor 400. In addition, if the determination result of the completion of the observation of the area using AI differs from the judgment of the human user (doctor), it is possible to accept user interaction and correct the display indicating the completion of the observation result.

[0217] According to the medical image processing device 20, during an examination in which multiple parts are observed sequentially, the doctor's observation behavior can be determined based on the acquired endoscopic images 18, and information on the parts that have been observed can be provided in real time. Therefore, in examinations requiring comprehensive observation of multiple parts, information on the parts that have been observed and the parts that have not been observed can be provided to the user in a timely manner, thus preventing the omission of observed parts.

[0218] Second Implementation Method

[0219] Figure 15 This is a functional block diagram illustrating the functions of the medical image processing apparatus 20B according to the second embodiment. Figure 15 In the middle, to and Figure 3 Elements with the same or similar structures are labeled with the same symbols, and their descriptions are omitted. Differences from the first embodiment are explained.

[0220] Figure 15 In addition to the processing functions of the medical image processing device 20 according to the first embodiment, the medical image processing device 20B shown also has the functions of saving the judgment result information of the observation completion judgment procedure 240 and the user's correction information, and sending the saved information to an external device not shown.

[0221] The medical image processing apparatus 20B includes a correction information storage unit 274 and a communication unit 277. The correction information storage unit 274 is an information storage unit that stores the observation image that is the object of the judgment processing performed by the observation completion judgment unit 24, the judgment result information output by the observation completion judgment unit 24 based on the observation image, and user input information indicating the correction content input by the user.

[0222] When a user's correction instruction is input from the user input receiving unit 28, the medical image processing device 20B associates the user input information, the judgment result information before correction as the correction object of the user input, and the observation image for which the judgment result information is obtained, i.e. the observation image for which the observation completion judgment unit 24 made an error judgment, and saves these data to the correction information storage unit 274.

[0223] The data saved in the correction information storage unit 274 can be used to improve the AI ​​model applied to the observation completion determination unit 24. For example, by using the observation image that the observation completion determination unit 24 incorrectly determined and the label of the correct determination result input by the user on the observation image as learning data, additional learning can be performed on the AI ​​model, thereby improving the determination accuracy.

[0224] Data stored in the correction information storage unit 274 can be transmitted to an external device (not shown) via the communication unit 277. Alternatively, data stored in the correction information storage unit 274 can also be stored in a portable storage medium (not shown) and retrieved from the outside of the medical image processing device 20B.

[0225] Communication unit 277 communicates with an external device (not shown) according to a prescribed communication protocol. Communication unit 277 is connected to... Figure 2 The communication control unit 205 shown and Figure 14 The structure corresponding to the communication interface 227 shown. The communication unit 277 is an example of a "communication device" in this disclosure. The external device may be, for example, a computer connected to the hospital's intranet. Alternatively, the external device may be a cloud server or the like, located in a facility outside the hospital. The external device may be a computer system that performs machine learning on the AI ​​model applied to the observation completion determination unit 24, or a data storage server that pre-stores learning datasets for machine learning.

[0226] The medical image processing device 20B can send correction information to an external device each time a user inputs a correction instruction for the display of the judgment result of the observation completion judgment unit 24, or it can send data sets at a predetermined communication time or when a certain amount of data sets are accumulated.

[0227] Figure 16 This is a block diagram illustrating the hardware structure of the medical image processing apparatus 20B according to the second embodiment. Figure 16 In the middle, to and Figure 14 Elements with the same or similar structures are labeled with the same symbols, and their descriptions are omitted.

[0228] The computer-readable medium 224 of the medical image processing apparatus 20B has a storage area serving as a correction information storage unit 274. When a user inputs a correction instruction regarding the display of the judgment result of the observation completion judgment obtained by the observation completion judgment program 240, the processor 222 associates the observation image obtained as the object of correction, the judgment result information, and the user input information indicating the correction content, and performs processing to save these data to the correction information storage unit 274.

[0229] The processor 222 can send data stored in the correction information storage unit 274 to an external device (not shown) via the communication interface 227.

[0230] Hardware Structure of Each Processing and Control Department

[0231] exist Figure 2 The image processing unit 204, communication control unit 205, and light source control unit 350 described herein are in... Figure 3 The image acquisition unit 22, observation completion determination unit 24, display control unit 26, determination result information display control unit 26A, display correction processing unit 26B, and user input receiving unit 28 described herein are as follows: Figure 15 The hardware structure of the processing unit, such as the communication unit 277 described herein, which performs various processes, consists of various processors as shown below.

[0232] Various processors include general-purpose processors that execute programs and function as various processing units, such as CPUs (Central Processing Units), dedicated processors for image processing, such as GPUs (Graphics Processing Units), processors such as FPGAs (Field Programmable Gate Arrays) whose circuit structure can be changed after manufacturing, such as Programmable Logic Devices (PLDs), and processors such as ASICs (Application Specific Integrated Circuits) with circuit structures specifically designed to perform specific processes, such as dedicated circuits.

[0233] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types. For example, a processing unit can also be composed of multiple FPGAs, a combination of CPU and FPGA, or a combination of CPU and GPU. Alternatively, a single processor can be used to construct multiple processing units. As examples of a single processor constructing multiple processing units, firstly, there are computer-like devices such as client or server computers, where a combination of one or more CPUs and software is used to construct a single processor, which functions as multiple processing units. Secondly, there are devices such as System-on-Chip (SoC), where a single IC (Integrated Circuit) chip implements the overall system functionality including multiple processing units. In this way, one or more of the aforementioned processors are used as hardware structures to construct various processing units.

[0234] Moreover, more specifically, the hardware structure of these various processors is a circuit that combines circuit elements such as semiconductor components.

[0235] "Observation Light of Endoscopic Systems"

[0236] The light selected for observation includes white light, one or more specific wavelength bands of light, or combinations thereof, among other wavelength bands appropriate to the observation objective. White light is light within the white wavelength band or multiple wavelength bands. A "specific wavelength band" is a band narrower than the white wavelength band. Specific examples related to specific wavelength bands are shown below.

[0237] 〈First example〉

[0238] The first example of a specific wavelength band is, for example, the blue or green band within the visible range. This first example's wavelength band includes a wavelength band of 390 nm to 450 nm or a wavelength band of 530 nm to 550 nm, and the light of the first example has a peak wavelength within either the 390 nm to 450 nm wavelength band or the 530 nm to 550 nm wavelength band.

[0239] <Second example>

[0240] A second example of a specific wavelength band is, for example, the red band within the visible range. This second example's wavelength band includes a wavelength band of 585 nm to 615 nm or a wavelength band of 610 nm to 730 nm, and the light of the second example has a peak wavelength within the wavelength band of 585 nm to 615 nm or within the wavelength band of 610 nm to 730 nm.

[0241] <Third Case>

[0242] A third example of a specific wavelength band includes wavelength bands where the absorption coefficient differs between oxidized and deoxygenated hemoglobin, and the light of the third example has a peak wavelength within these different wavelength bands. The wavelength bands of this third example include wavelength bands of 400±10 nm, 440±10 nm, 470±10 nm, or wavelength bands between 600 nm and 750 nm, and the light of the third example has a peak wavelength within these wavelength bands.

[0243] <Fourth Case>

[0244] A fourth example of a specific wavelength band is the wavelength band of the excitation light used to excite the fluorescent substance in a living organism (fluorescence observation), for example, 390 nm to 470 nm.

[0245] <Fifth Case>

[0246] The fifth example of a specific wavelength band is the wavelength band of infrared light. The wavelength band of this fifth example includes a wavelength band of 790nm to 820nm or a wavelength band of 905nm to 970nm, and the light of the fifth example has a peak wavelength in the wavelength band of 790nm to 820nm or the wavelength band of 905nm to 970nm.

[0247] On observing the switching of light

[0248] The light source can be a laser light source, a xenon light source, or an LED (Light-Emitting Diode) light source, or a suitable combination thereof. The type, wavelength, and presence or absence of filters of the light source are preferably configured according to the type of subject and the purpose of observation. Furthermore, during observation, it is preferable to combine and / or switch the wavelength of the illumination light according to the type of subject and the purpose of observation. When switching wavelengths, for example, the wavelength of the illuminated light can be switched by rotating a disc-shaped filter (rotating color filter) positioned in front of the light source and equipped with filters that transmit or block light of specific wavelengths.

[0249] Imaging elements used in electronic endoscopes are not limited to color imaging elements with color filters for each pixel; monochrome imaging elements can also be used. When using a monochrome imaging element, the wavelengths of the illumination light can be switched sequentially to capture images in a surface order (color order). For example, the wavelengths of the emitted illumination light can be switched sequentially between violet, blue, green, and red. Alternatively, broadband light (white light) can be used, and the wavelengths of the emitted illumination light can be switched by rotating color filters (red, green, blue, etc.). Furthermore, one or more narrowband lights can be used, and the wavelengths of the emitted illumination light can be switched by rotating color filters. Narrowband light can also be infrared light of two or more different wavelengths.

[0250] Examples of generating special light images

[0251] The processor device 200 can also generate a special light image with information of a specific wavelength band based on a normal light image obtained by shooting with white light. Furthermore, the term "generation" here includes the concept of "acquisition." In this case, the processor device 200 functions as a special light image acquisition unit. The processor device 200 can obtain a signal of a specific wavelength band by performing calculations based on the color information of red (R), green (G), and blue (B), or cyan (C), magenta (M), and yellow (Y) contained in the normal light image.

[0252] Example of generating feature quantity images

[0253] The processor device 200 can generate a feature image as a medical image by performing calculations based on at least one of ordinary light images obtained by illuminating light in the white frequency band or light in multiple wavelength bands as the white frequency band, and special light images obtained by illuminating light in a specific wavelength band. A feature image is one type of medical image.

[0254] Illumination using phosphors

[0255] Alternatively, a phosphor can be disposed between the output end of the light guide 170 and the illumination lenses 123A and 123B of the endoscope observer 100. For example, a blue laser passing through the light guide 170 irradiates the phosphor, exciting it, and a portion of it passes through the phosphor and is emitted as blue light from the illumination lenses 123A and 123B.

[0256] The phosphor is excited by a blue laser, emitting a wide range of light (yellow as the color) from the wavelength range near the boundary between blue and green to the red wavelength range. Alternatively, this yellow light can be mixed with the blue light passing through the phosphor to form white light, which then illuminates the subject through illumination lenses 123A and 123B. Furthermore, the blue light passing through the phosphor also includes a portion of the blue light emitted by the phosphor.

[0257] A phosphor can be, for example, a phosphor that emits yellow light when irradiated by a blue laser with a wavelength of 445 nm, allows blue light with a wavelength of 445 nm to pass through, and allows most of the blue light with a wavelength of 405 nm to pass through when irradiated by a blue laser with a wavelength of 405 nm. By using such a phosphor, the mixing ratio of the blue laser with a wavelength of 445 nm and the blue laser with a wavelength of 405 nm in the light source device can be controlled, thereby controlling the ratio of blue light passing through the phosphor to yellow light emitted by the phosphor.

[0258] Variation Example 1

[0259] In the above embodiments, an example using an endoscope observer 100 as a flexible endoscope has been described. However, the endoscope used for imaging inside the body is not limited to a flexible endoscope; it can also be a rigid endoscope or a capsule endoscope. The medical images processed by the medical image processing apparatus according to this disclosure are not limited to endoscopic images; they can also be images generated by other medical imaging devices such as ultrasound diagnostic devices. Medical imaging devices can be, for example, at least one of X-ray imaging devices, CT (Computed Tomography) imaging devices, MRI (Magnetic Resonance Imaging) imaging devices, and nuclear medicine diagnostic devices. The technology of this disclosure can be applied to apparatuses that process medical images acquired using these various medical imaging devices (modalities).

[0260] Variation Example 2

[0261] The medical image processing device disclosed herein can be used as a diagnostic aid to assist doctors in examination, treatment, or diagnosis. The term "diagnostic aid" includes the concepts of diagnostic aid, treatment aid, examination aid, observation aid, and lesion differentiation aid.

[0262] Variation Example 3

[0263] The display method used to inform the user whether multiple parts have been fully observed is not limited to the area display using model graphics such as imagery 46 or the list display of text information. It can also be a progress bar display indicating the progress of observation, or a check list display in the form of marking the observed parts with check marks.

[0264] Examples of the Application of Medical Information Management Systems

[0265] The medical image processing apparatus disclosed herein is not limited to applications in... Figure 1 The processor device 200 of the illustrated endoscope system 10 can be used for various applications. For example, the medical image processing device can be applied to a medical information management system that manages various medical information, including endoscopic images.

[0266] Information management devices equipped with the processing functions of the medical image processing apparatus disclosed herein can be installed in operating rooms, examination rooms, or conference rooms within hospitals, or in medical institutions or research institutions outside of hospitals. The information management device can be an auxiliary workstation for examination, treatment, and diagnosis, or a business support device to assist medical operations. The business support device can have functions such as accumulating clinical information, assisting in the creation of diagnostic documents, and assisting in report creation.

[0267] "Program for Enabling Computers to Perform Medical Image Processing Functions"

[0268] The program that enables the computer to perform the functions of the medical image processing apparatus 20, 20B described in the above embodiments can be recorded on a non-transitory information storage medium, such as an optical disc, magnetic disk, or semiconductor memory, and provided through that information storage medium. Alternatively, instead of storing the program on such a non-transitory information storage medium, a program signal can be provided as a download service using telecommunications lines such as the Internet.

[0269] Alternatively, it can provide some or all of the functions of the medical image processing apparatus described in the above embodiments as an application server, and provide processing functions via telecommunication lines.

[0270] Regarding combinations of implementation methods and variations, etc.

[0271] The constituent elements described in the above embodiments and the constituent elements described in the variations can be used in appropriate combinations. In addition, some constituent elements can be replaced.

[0272] Postscript

[0273] In addition to the various embodiments and modifications described above, this specification also includes the disclosure of the invention described below.

[0274] (Note 1)

[0275] A medical image processing apparatus includes a medical image analysis and processing unit and a medical image analysis result acquisition unit. The medical image analysis and processing unit detects regions of interest as regions of interest based on the feature quantities of pixels in a medical image, and the medical image analysis result acquisition unit acquires the analysis results of the medical image analysis and processing unit.

[0276] The medical image analysis and processing unit may include an image recognition unit.

[0277] (Note 2)

[0278] A medical image processing apparatus, wherein a medical image analysis and processing unit detects whether there is an object of interest based on the feature quantities of pixels in a medical image, and a medical image analysis result acquisition unit acquires the analysis result of the medical image analysis and processing unit.

[0279] (Note 3)

[0280] A medical image processing apparatus, wherein a medical image analysis result acquisition unit acquires analysis results of a medical image from a recording device, the analysis results being either or both of the regions of interest included in the medical image and the presence or absence of objects of interest.

[0281] (Note 4)

[0282] A medical image processing apparatus, wherein the medical image is a conventional light image obtained by irradiating light in the white frequency band, or by irradiating light in multiple wavelength bands as white frequency band light.

[0283] (Note 5)

[0284] A medical image processing apparatus, wherein the medical image is an image obtained by irradiating light with a specific wavelength band, the specific wavelength band being a band narrower than the white wavelength band.

[0285] (Note 6)

[0286] A medical image processing device wherein a specific wavelength band is the blue or green band of the visible range.

[0287] (Note 7)

[0288] A medical image processing device, wherein a specific wavelength band includes a wavelength band of 390nm to 450nm or 530nm to 550nm, and the light in the specific wavelength band has a peak wavelength within the wavelength band of 390nm to 450nm or 530nm to 550nm.

[0289] (Postscript 8)

[0290] A medical image processing device wherein a specific wavelength band is the red band of the visible range.

[0291] (Note 9)

[0292] A medical image processing device, wherein a specific wavelength band includes a wavelength band of 585nm to 615nm or 610nm to 730nm, and the light in the specific wavelength band has a peak wavelength within the wavelength band of 585nm to 615nm or 610nm to 730nm.

[0293] (Postscript 10)

[0294] A medical image processing device, wherein a specific wavelength band includes wavelength bands with different absorption coefficients in oxidized hemoglobin and deoxygenated hemoglobin, and the light in the specific wavelength band has a peak wavelength in the wavelength bands with different absorption coefficients in oxidized hemoglobin and deoxygenated hemoglobin.

[0295] (Postscript 11)

[0296] A medical image processing device, wherein a specific wavelength band includes wavelength bands of 400±10nm, 440±10nm, 470±10nm, or 600nm to 750nm, and the light in the specific wavelength band has a peak wavelength in the wavelength bands of 400±10nm, 440±10nm, 470±10nm, or 600nm to 750nm.

[0297] (Postscript 12)

[0298] A medical image processing device, wherein the medical image is an image of an organism inside a living organism, and the image of the organism inside the living organism contains information about the fluorescence emitted by fluorescent substances within the organism.

[0299] (Postscript 13)

[0300] A medical image processing device, wherein fluorescence is obtained by irradiating a living organism with an excitation light having a peak wavelength of 390 nm or higher and 470 nm or lower.

[0301] (Postscript 14)

[0302] A medical image processing device, wherein the medical image is an image of a living organism captured within a specific wavelength band, which is the wavelength band of infrared light.

[0303] (Postscript 15)

[0304] A medical image processing device, wherein a specific wavelength band includes a wavelength band of 790nm to 820nm or 905nm to 970nm, and the light in the specific wavelength band has a peak wavelength in the wavelength band of 790nm to 820nm or 905nm to 970nm.

[0305] (Postscript 16)

[0306] A medical image processing apparatus, wherein a medical image acquisition unit includes a special light image acquisition unit, which acquires a special light image with information of a specific wavelength band based on a normal light image obtained by irradiating light of a white frequency band or light of multiple wavelength bands as a white frequency band. The medical image is a special light image.

[0307] (Postscript 17)

[0308] A medical image processing device, wherein a signal in a specific wavelength band is obtained by calculation based on the RGB or CMY color information contained in a normal light image.

[0309] (Postscript 18)

[0310] A medical image processing apparatus includes a feature quantity image generation unit that generates a feature quantity image by performing calculations based on at least one of a general light image obtained by irradiating light in the white frequency band, or light in multiple wavelength frequency bands as white frequency bands, and a special light image obtained by irradiating light in a specific wavelength frequency band. The medical image is a feature quantity image.

[0311] (Postscript 19)

[0312] An endoscope device comprising: a medical image processing device as described in any one of Annex 1 to Annex 18; and an endoscope that acquires images by irradiating light in a white wavelength band or light in a specific wavelength band.

[0313] (Postscript 20)

[0314] A diagnostic aid device, comprising any one of Annex 1 to Annex 18, a medical image processing device.

[0315] (Postscript 21)

[0316] A medical business assistance device, comprising any one of Annex 1 to Annex 18, a medical image processing device.

[0317] "other"

[0318] The embodiments of the present invention described above may be modified, added to, or deleted as appropriate without departing from the spirit of the invention. The present invention is not limited to the embodiments described above; many modifications can be made by those skilled in the art within the scope of the technical concept of the present invention.

[0319] Symbol Explanation

[0320] 10 Endoscopic Systems

[0321] 18 Endoscopic images

[0322] 18A Dynamic Images

[0323] 18B frame image

[0324] 19 Still Images

[0325] 20, 20B Medical Image Processing Device

[0326] 22 Image Acquisition Unit

[0327] 24. Observation Completion Judgment Department

[0328] 26 Display Control Unit

[0329] 26A Judgment Result Information Display Control Unit

[0330] 26B Display Correction Processing Unit

[0331] 28 User Input Reception Section

[0332] 40 pictures

[0333] 42 Main screen

[0334] 44 sub-screens

[0335] 44A First Sub-Picture

[0336] 44B Second Sub-Picture

[0337] 46 Imagery

[0338] 47. Observe the completed area.

[0339] 48. Observe the unfinished parts.

[0340] Blocks 51, 52, 53, and 54

[0341] 70 holes

[0342] 100 Endoscopic Observation Devices

[0343] 102 Hands-on Operations Department

[0344] 104 Insertion Section

[0345] 106 General Purpose Cable

[0346] 108 Optical Wire Connector

[0347] 112 Soft parts

[0348] 114 Bend

[0349] 116 Hardened tip

[0350] 116A Top Side Face

[0351] 123 Lighting Department

[0352] 123A, 123B Illumination Lenses

[0353] 126 Pliers opening

[0354] 130 Camera Department

[0355] 132 Photographic Lens

[0356] 134 camera elements

[0357] 136 drive circuit

[0358] 138 Simulated Front End

[0359] 140° bend button

[0360] 141 Gas and water supply buttons

[0361] 142 Attraction Button

[0362] 143 Function Buttons

[0363] 144. Camera button

[0364] 170 optical waveguide

[0365] 200 processor devices

[0366] 202 Image Input Controller

[0367] 204 Image Processing Department

[0368] 205 Communications Control Department

[0369] 206 Video Output Department

[0370] 207 Storage Department

[0371] 208 Operations Department

[0372] 209 Sound Processing Department

[0373] 209A Speaker

[0374] 210 CPU

[0375] 211 ROM

[0376] 212 RAM

[0377] 222 processor

[0378] 224 Computer-readable media

[0379] 225 Image Input Interface

[0380] 226 Image Processing Processor

[0381] 227 Communication Interface

[0382] 228 Input / Output Interfaces

[0383] 230 bus

[0384] 234 Input Device

[0385] 236 Display Device

[0386] 240. Observation completion judgment procedure

[0387] 260 Display Control Program

[0388] 261 Main screen control module

[0389] 262 Sub-screen Control Module

[0390] 263 Display Correction Module

[0391] 271 Image Storage Unit

[0392] 272. Determine the information storage department.

[0393] 273 User Input Information Storage Department

[0394] 274. Correction of Information Storage Department

[0395] 277 Ministry of Communications

[0396] 300 Light Source Device

[0397] 310 Light Source

[0398] 310B Blue Light Source

[0399] 310G Green Light Source

[0400] 310R Red Light Source

[0401] 330 aperture

[0402] 340 Condensing Lens

[0403] 350 Light Source Control Unit

[0404] 400 monitor

[0405] S12~S20 Processing steps performed by the medical image processing device

Claims

1. A medical image processing apparatus, comprising at least one processor, wherein, The at least one processor Acquiring medical images, Based on the medical images, a determination is made as to whether the observation of the target area has been completed. The display control is performed to show the determination result of the observation completed on the display screen. Accept user input including instructions to correct the displayed content indicating the determination result of the observation completion. The content that has been corrected based on the user input is reflected in the display.

2. The medical image processing apparatus according to claim 1, wherein, The medical image processing device also includes a display that shows the determination result of the observation completion determination.

3. The medical image processing apparatus according to claim 1, wherein, If the object region is marked on the medical image, the at least one processor determines that the observation of the object region is complete.

4. The medical image processing apparatus according to any one of claims 1 to 3, wherein, If the object region is present on the medical image for a fixed period of time or more, the at least one processor determines that the observation of the object region is complete.

5. The medical image processing apparatus according to any one of claims 1 to 3, wherein, If a mark of the object region is present in the center of the medical image, the at least one processor determines that the observation of the object region is complete.

6. The medical image processing apparatus according to any one of claims 1 to 3, wherein, When the presence of a mark indicating the object region on the medical image is taken as a first condition, the presence of the object region on the medical image for a fixed period of time is taken as a second condition, and the presence of a mark indicating the object region in the center of the medical image is taken as a third condition, the following conditions apply. If at least one of the first condition, the second condition, and the third condition is met, the at least one processor determines that the observation of the object part is complete.

7. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor receives an input of a shooting instruction signal indicating the shooting timing of a still image, and upon receiving the input of the shooting instruction signal, determines that the observation of the part of the still image that is the subject of the shooting is completed.

8. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The medical image processing device also includes a memory. The memory stores the medical image and the determination result information indicating the completion of the observation. When the at least one processor receives the user input User input information, the determination result information before correction (which is the object of correction for the user input), and the medical image for which the determination result information is obtained are associatedly saved in the memory.

9. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The medical image processing device also includes: Memory; and communication device, The memory stores the medical image and the determination result information indicating the completion of the observation. When the at least one processor receives the user input The communication device sends user input information, the judgment result information before correction (which is the object of correction for the user input), and the medical image with the obtained judgment result information to an external device.

10. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor receives the user input that corrects the "observation incomplete" display for parts where the observation is incomplete to a "observation complete" display indicating that the observation is complete. Based on the user input, the display content for when the observation is incomplete is changed to the display content for when the observation is complete.

11. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor receives user input that modifies the observation completion display of the part where the observation completion determination result is observation completion to an observation incomplete display indicating that the observation is not completed. Based on the user input, the display content for when the observation is completed is changed to the display content for when the observation is incomplete.

12. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor Accept user input that requests a change in the display of location information for a location deemed complete by the observation completion determination to display location information for another location. Based on the user input, the displayed content of the location information for the observed area is corrected.

13. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor The medical image is displayed in the first display area of ​​the display. The determination result of the observation is displayed in a second display area on the display that is different from the first display area.

14. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor displays part information about the observed part by schematically representing a model graphic of a human body part including the object part.

15. The medical image processing apparatus according to claim 14, wherein, The at least one processor uses different colors to distinguish between observed completed areas and observed incomplete areas in the model graphic.

16. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor displays location information about the observed area as text information.

17. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The medical image processing device also includes an input device for the user to input the instructions. The at least one processor accepts the user input from the input device.

18. The medical image processing apparatus according to claim 17, wherein, The input device includes at least one of a keyboard, a touch panel, a voice input device, a switch located on the endoscope observer, and a foot switch.

19. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor Set multiple body parts as the objects of observation. The determination is completed by observing each of the multiple locations. Based on the determination result of the observation completion, information indicating the observed completed parts and information indicating the observed incomplete parts are displayed.

20. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor uses a neural network to perform the observation and complete the determination.

21. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The at least one processor acquires the time-series medical images.

22. The medical image processing apparatus according to any one of claims 1 to 3, wherein, The medical images are endoscopic images taken using an endoscope.

23. An endoscope system comprising an endoscope observer and at least one processor, wherein, The at least one processor Acquire endoscopic images obtained by taking pictures inside the body using the endoscopic observer. Based on the endoscopic images, a determination is made as to whether the observation of the target area has been completed. The display control is performed to show the determination result of the observation completed on the display screen. Accept user input including instructions to correct the displayed content indicating the determination result of the observation completion. The content that has been corrected based on the user input is reflected in the display.

24. A diagnostic assistance method, implemented by at least one processor, wherein, Includes the following steps performed by the at least one processor: Acquiring medical images; Based on the medical images, a determination is made as to whether the observation of the target area has been completed. Display control is performed to show the determination result of the observation completed on the display screen; Accept user input including instructions to correct the displayed content indicating the determination result of the observation completion; as well as The content that has been corrected based on the user input is reflected in the display.

25. A recording medium that is non-transitory and computer-readable, wherein, The computer contains computer commands that cause it to execute the diagnostic assistance method of claim 24.

Images